[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:59:52.242122   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38733
I20260709 12:59:52.244056   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:59:52.245234   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:52.264608   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:52.264704   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:52.267966   431 server_base.cc:1061] running on GCE node
W20260709 12:59:52.265278   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:53.508782   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:53.509047   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:53.509328   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601993509299 us; error 0 us; skew 500 ppm
I20260709 12:59:53.510211   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:53.517290   431 webserver.cc:533] Webserver started at http://127.0.107.254:46779/ using document root <none> and password file <none>
I20260709 12:59:53.518203   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:53.518428   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:53.519134   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:53.526955   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "2cff963597994b30a7d083542e68ccb8"
format_stamp: "Formatted at 2026-07-09 12:59:53 on dist-test-slave-kgkf"
I20260709 12:59:53.535422   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:59:53.541895   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:53.543118   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:59:53.543524   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "2cff963597994b30a7d083542e68ccb8"
format_stamp: "Formatted at 2026-07-09 12:59:53 on dist-test-slave-kgkf"
I20260709 12:59:53.543947   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:53.597223   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:53.598795   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:59:53.599319   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:53.682085   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38733
I20260709 12:59:53.682143   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38733 every 8 connection(s)
I20260709 12:59:53.690543   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:53.714411   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: Bootstrap starting.
I20260709 12:59:53.720969   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:53.722954   498 log.cc:826] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: Log is configured to *not* fsync() on all Append() calls
I20260709 12:59:53.728164   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: No bootstrap required, opened a new log
I20260709 12:59:53.749876   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER }
I20260709 12:59:53.750463   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:53.750676   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cff963597994b30a7d083542e68ccb8, State: Initialized, Role: FOLLOWER
I20260709 12:59:53.751503   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER }
I20260709 12:59:53.752082   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:53.752352   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:53.752652   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:53.757270   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER }
I20260709 12:59:53.758013   498 leader_election.cc:304] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cff963597994b30a7d083542e68ccb8; no voters: 
I20260709 12:59:53.759706   498 leader_election.cc:290] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:53.760120   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:53.763736   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 1 LEADER]: Becoming Leader. State: Replica: 2cff963597994b30a7d083542e68ccb8, State: Running, Role: LEADER
I20260709 12:59:53.764843   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER }
I20260709 12:59:53.765897   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:53.778558   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cff963597994b30a7d083542e68ccb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER } }
I20260709 12:59:53.778357   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cff963597994b30a7d083542e68ccb8. Latest consensus state: current_term: 1 leader_uuid: "2cff963597994b30a7d083542e68ccb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cff963597994b30a7d083542e68ccb8" member_type: VOTER } }
I20260709 12:59:53.779605   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:53.779722   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:53.784129   510 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:53.797024   510 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:53.799685   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:53.816820   510 catalog_manager.cc:1383] Generated new cluster ID: d30fb6273d954108ab6064a0cd323627
I20260709 12:59:53.817152   510 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:53.843720   510 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:53.845352   510 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:53.860214   510 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: Generated new TSK 0
I20260709 12:59:53.861266   510 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:53.867148   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:53.877607   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:53.878865   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:53.883045   431 server_base.cc:1061] running on GCE node
W20260709 12:59:53.883175   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:53.884202   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:53.884440   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:53.884640   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601993884622 us; error 0 us; skew 500 ppm
I20260709 12:59:53.885330   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:53.889662   431 webserver.cc:533] Webserver started at http://127.0.107.193:41679/ using document root <none> and password file <none>
I20260709 12:59:53.890228   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:53.890431   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:53.890733   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:53.891923   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "6ad1b341cd64494a810f03572475100a"
format_stamp: "Formatted at 2026-07-09 12:59:53 on dist-test-slave-kgkf"
I20260709 12:59:53.898002   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.005s
I20260709 12:59:53.902601   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:53.903586   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:59:53.903997   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "6ad1b341cd64494a810f03572475100a"
format_stamp: "Formatted at 2026-07-09 12:59:53 on dist-test-slave-kgkf"
I20260709 12:59:53.904381   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:53.949390   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:53.951215   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:53.953975   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:53.958076   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:53.958344   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:53.958621   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:53.958811   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.068898   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42163
I20260709 12:59:54.068969   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42163 every 8 connection(s)
I20260709 12:59:54.076742   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:54.088479   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:54.095463   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:54.102156   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38733
I20260709 12:59:54.102756   591 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:54.103832   591 heartbeater.cc:507] Master 127.0.107.254:38733 requested a full tablet report, sending...
W20260709 12:59:54.105437   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:54.105899   431 server_base.cc:1061] running on GCE node
I20260709 12:59:54.107954   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:59:54.107910   463 ts_manager.cc:194] Registered new tserver with Master: 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163)
W20260709 12:59:54.108292   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:54.108516   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601994108491 us; error 0 us; skew 500 ppm
I20260709 12:59:54.109333   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:54.111078   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52976
I20260709 12:59:54.113098   431 webserver.cc:533] Webserver started at http://127.0.107.194:40373/ using document root <none> and password file <none>
I20260709 12:59:54.113777   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:54.114040   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:54.114398   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:54.115756   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "d595fd28c9804e208dc6fbe9448860a3"
format_stamp: "Formatted at 2026-07-09 12:59:54 on dist-test-slave-kgkf"
I20260709 12:59:54.121034   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:54.125528   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.126346   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:59:54.126637   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "d595fd28c9804e208dc6fbe9448860a3"
format_stamp: "Formatted at 2026-07-09 12:59:54 on dist-test-slave-kgkf"
I20260709 12:59:54.126935   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:54.154871   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:54.156320   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:54.158918   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:54.162271   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:54.162501   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.162744   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:54.162923   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.272571   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:41575
I20260709 12:59:54.272864   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:41575 every 8 connection(s)
I20260709 12:59:54.280978   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:54.290983   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:54.292155   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:54.297899   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:54.298024   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38733
I20260709 12:59:54.298607   666 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:54.299711   666 heartbeater.cc:507] Master 127.0.107.254:38733 requested a full tablet report, sending...
I20260709 12:59:54.300273   431 server_base.cc:1061] running on GCE node
I20260709 12:59:54.301366   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:54.301577   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:54.301807   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601994301787 us; error 0 us; skew 500 ppm
I20260709 12:59:54.302307   463 ts_manager.cc:194] Registered new tserver with Master: d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:54.302539   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:54.304172   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52982
I20260709 12:59:54.308408   431 webserver.cc:533] Webserver started at http://127.0.107.195:41509/ using document root <none> and password file <none>
I20260709 12:59:54.308907   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:54.309098   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:54.309372   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:54.310405   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
format_stamp: "Formatted at 2026-07-09 12:59:54 on dist-test-slave-kgkf"
I20260709 12:59:54.314982   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:59:54.318342   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.319192   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:59:54.319537   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
format_stamp: "Formatted at 2026-07-09 12:59:54 on dist-test-slave-kgkf"
I20260709 12:59:54.319914   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:54.336833   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:54.338449   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:54.341506   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:54.345268   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:54.345538   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.345829   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:54.346038   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:54.438750   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:42801
I20260709 12:59:54.438812   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:42801 every 8 connection(s)
I20260709 12:59:54.457989   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38733
I20260709 12:59:54.458376   740 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:54.459250   740 heartbeater.cc:507] Master 127.0.107.254:38733 requested a full tablet report, sending...
I20260709 12:59:54.461439   463 ts_manager.cc:194] Registered new tserver with Master: 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:54.461547   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016341249s
I20260709 12:59:54.463295   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52992
I20260709 12:59:54.526467   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52998:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:54.531497   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:54.640364   556 tablet_service.cc:1511] Processing CreateTablet for tablet 38901755dc724684ab5f9407ea15a220 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:59:54.640245   555 tablet_service.cc:1511] Processing CreateTablet for tablet 20ece978b14a4aee944391b0dfc79c0e (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:59:54.642518   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38901755dc724684ab5f9407ea15a220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.642259   630 tablet_service.cc:1511] Processing CreateTablet for tablet 20ece978b14a4aee944391b0dfc79c0e (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:59:54.643180   629 tablet_service.cc:1511] Processing CreateTablet for tablet c4121df3cce444999297d205d8dedeeb (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:59:54.643654   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ece978b14a4aee944391b0dfc79c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.644989   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4121df3cce444999297d205d8dedeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.646256   631 tablet_service.cc:1511] Processing CreateTablet for tablet 38901755dc724684ab5f9407ea15a220 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:59:54.647529   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38901755dc724684ab5f9407ea15a220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.648357   628 tablet_service.cc:1511] Processing CreateTablet for tablet f1e626c407ad476d85798e0e811d7d12 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:59:54.654160   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ece978b14a4aee944391b0dfc79c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.654141   554 tablet_service.cc:1511] Processing CreateTablet for tablet c4121df3cce444999297d205d8dedeeb (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:59:54.657140   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1e626c407ad476d85798e0e811d7d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.661270   552 tablet_service.cc:1511] Processing CreateTablet for tablet e77caab168104bf9ac8a9e74fe110ec5 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:59:54.656136   553 tablet_service.cc:1511] Processing CreateTablet for tablet f1e626c407ad476d85798e0e811d7d12 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:59:54.665247   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4121df3cce444999297d205d8dedeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.666481   627 tablet_service.cc:1511] Processing CreateTablet for tablet e77caab168104bf9ac8a9e74fe110ec5 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:59:54.668119   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77caab168104bf9ac8a9e74fe110ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.671998   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1e626c407ad476d85798e0e811d7d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.673316   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77caab168104bf9ac8a9e74fe110ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.692322   551 tablet_service.cc:1511] Processing CreateTablet for tablet 55c144d08f7c4c76a755d2a49212b236 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:59:54.698444   626 tablet_service.cc:1511] Processing CreateTablet for tablet 55c144d08f7c4c76a755d2a49212b236 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:59:54.701382   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c144d08f7c4c76a755d2a49212b236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.705221   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c144d08f7c4c76a755d2a49212b236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.706974   550 tablet_service.cc:1511] Processing CreateTablet for tablet 39ced31be3de4b4581ad5c3e4121b2f4 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:59:54.712248   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ced31be3de4b4581ad5c3e4121b2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.717485   625 tablet_service.cc:1511] Processing CreateTablet for tablet 39ced31be3de4b4581ad5c3e4121b2f4 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:59:54.719590   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ced31be3de4b4581ad5c3e4121b2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.732626   705 tablet_service.cc:1511] Processing CreateTablet for tablet 38901755dc724684ab5f9407ea15a220 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:59:54.744586   702 tablet_service.cc:1511] Processing CreateTablet for tablet f1e626c407ad476d85798e0e811d7d12 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:59:54.749892   701 tablet_service.cc:1511] Processing CreateTablet for tablet e77caab168104bf9ac8a9e74fe110ec5 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:59:54.738601   704 tablet_service.cc:1511] Processing CreateTablet for tablet 20ece978b14a4aee944391b0dfc79c0e (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:59:54.755102   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38901755dc724684ab5f9407ea15a220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.760322   700 tablet_service.cc:1511] Processing CreateTablet for tablet 55c144d08f7c4c76a755d2a49212b236 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:59:54.766490   699 tablet_service.cc:1511] Processing CreateTablet for tablet 39ced31be3de4b4581ad5c3e4121b2f4 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:59:54.761282   703 tablet_service.cc:1511] Processing CreateTablet for tablet c4121df3cce444999297d205d8dedeeb (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:59:54.775820   549 tablet_service.cc:1511] Processing CreateTablet for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:59:54.777596   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.770447   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1e626c407ad476d85798e0e811d7d12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:54.790005   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:54.793094   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ece978b14a4aee944391b0dfc79c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.796273   758 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:54.803438   758 tablet_bootstrap.cc:654] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:54.806672   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4121df3cce444999297d205d8dedeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.808806   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77caab168104bf9ac8a9e74fe110ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.817312   624 tablet_service.cc:1511] Processing CreateTablet for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:59:54.819258   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c144d08f7c4c76a755d2a49212b236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.831123   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ced31be3de4b4581ad5c3e4121b2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.840365   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.849371   698 tablet_service.cc:1511] Processing CreateTablet for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:59:54.852270   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a360ceb58a4bbaa26a2d4b4a4941fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.861639   556 tablet_service.cc:1511] Processing CreateTablet for tablet 329fabbc2bc94f8784c0aaeb6990ac09 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:59:54.863021   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 329fabbc2bc94f8784c0aaeb6990ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.873871   759 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:54.877096   758 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:54.877591   758 ts_tablet_manager.cc:1403] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.007s
I20260709 12:59:54.884200   628 tablet_service.cc:1511] Processing CreateTablet for tablet 329fabbc2bc94f8784c0aaeb6990ac09 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:59:54.885361   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 329fabbc2bc94f8784c0aaeb6990ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.882663   758 raft_consensus.cc:359] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:54.896714   758 raft_consensus.cc:385] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:54.904975   758 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:54.925158   761 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:54.929512   705 tablet_service.cc:1511] Processing CreateTablet for tablet 329fabbc2bc94f8784c0aaeb6990ac09 (DEFAULT_TABLE table=test-workload [id=cedf2fb80b894178be9fa1b1d1dfb686]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:59:54.931707   758 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:54.949067   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 329fabbc2bc94f8784c0aaeb6990ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:54.966143   758 ts_tablet_manager.cc:1434] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.087s	user 0.001s	sys 0.016s
I20260709 12:59:54.966295   591 heartbeater.cc:499] Master 127.0.107.254:38733 was elected leader, sending a full tablet report...
I20260709 12:59:54.967120   758 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:54.976833   761 tablet_bootstrap.cc:654] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:54.980557   758 tablet_bootstrap.cc:654] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:54.989501   759 tablet_bootstrap.cc:654] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:54.997779   761 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:54.998382   761 ts_tablet_manager.cc:1403] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.006s
I20260709 12:59:55.001756   759 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.001554   761 raft_consensus.cc:359] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.002202   759 ts_tablet_manager.cc:1403] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.155s	user 0.008s	sys 0.010s
I20260709 12:59:55.002324   761 raft_consensus.cc:385] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.002627   761 raft_consensus.cc:740] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.003424   761 consensus_queue.cc:260] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.005374   759 raft_consensus.cc:359] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.006187   759 raft_consensus.cc:385] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.006556   759 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.007119   758 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.007623   758 ts_tablet_manager.cc:1403] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.012s
I20260709 12:59:55.007575   759 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.010730   758 raft_consensus.cc:359] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.011477   758 raft_consensus.cc:385] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.011832   758 raft_consensus.cc:740] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.014933   761 ts_tablet_manager.cc:1434] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.008s
I20260709 12:59:55.015834   740 heartbeater.cc:499] Master 127.0.107.254:38733 was elected leader, sending a full tablet report...
I20260709 12:59:55.013065   758 consensus_queue.cc:260] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.028267   758 ts_tablet_manager.cc:1434] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:59:55.034439   761 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.038887   766 raft_consensus.cc:493] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.039657   766 raft_consensus.cc:515] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.047263   762 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.034543   758 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.047812   762 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.049211   766 leader_election.cc:290] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.053195   762 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.057744   759 ts_tablet_manager.cc:1434] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.055s	user 0.026s	sys 0.021s
I20260709 12:59:55.058705   666 heartbeater.cc:499] Master 127.0.107.254:38733 was elected leader, sending a full tablet report...
I20260709 12:59:55.061544   759 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.075798   761 tablet_bootstrap.cc:654] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:55.078668   592 tablet.cc:2406] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:59:55.104149   758 tablet_bootstrap.cc:654] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.111718   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.114212   761 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
W20260709 12:59:55.115446   678 leader_election.cc:343] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.116967   761 ts_tablet_manager.cc:1403] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.083s	user 0.005s	sys 0.022s
I20260709 12:59:55.116910   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.118799   759 tablet_bootstrap.cc:654] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.118875   566 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 0.
I20260709 12:59:55.122431   679 leader_election.cc:304] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:55.124137   758 tablet_bootstrap.cc:492] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.124722   768 raft_consensus.cc:2804] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.125154   768 raft_consensus.cc:493] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.125263   758 ts_tablet_manager.cc:1403] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.091s	user 0.010s	sys 0.009s
I20260709 12:59:55.125494   768 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.134086   761 raft_consensus.cc:359] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.135557   761 raft_consensus.cc:385] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.136466   761 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.139216   761 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.133126   758 raft_consensus.cc:359] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.140403   758 raft_consensus.cc:385] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.140707   758 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.143354   759 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.143863   759 ts_tablet_manager.cc:1403] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.015s
I20260709 12:59:55.146628   759 raft_consensus.cc:359] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.147085   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.147346   759 raft_consensus.cc:385] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:55.148646   529 leader_election.cc:343] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.149894   761 ts_tablet_manager.cc:1434] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:59:55.150847   761 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.150668   758 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.147653   759 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.153259   758 ts_tablet_manager.cc:1434] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:59:55.158077   759 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.159365   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.161268   758 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
W20260709 12:59:55.161527   529 leader_election.cc:343] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.162009   529 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad1b341cd64494a810f03572475100a; no voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:55.164463   761 tablet_bootstrap.cc:654] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.164685   759 ts_tablet_manager.cc:1434] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:59:55.164530   768 raft_consensus.cc:515] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.165228   762 raft_consensus.cc:2749] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:55.166504   759 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.174134   759 tablet_bootstrap.cc:654] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.179113   758 tablet_bootstrap.cc:654] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.189972   759 tablet_bootstrap.cc:492] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.190462   759 ts_tablet_manager.cc:1403] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 12:59:55.191633   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.192938   566 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.195822   761 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.195441   759 raft_consensus.cc:359] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.196197   759 raft_consensus.cc:385] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.196318   761 ts_tablet_manager.cc:1403] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.010s
I20260709 12:59:55.196508   759 raft_consensus.cc:740] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.197268   759 consensus_queue.cc:260] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.200187   759 ts_tablet_manager.cc:1434] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:55.201115   759 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.200708   761 raft_consensus.cc:359] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.201926   761 raft_consensus.cc:385] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.204644   761 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.205546   761 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.206868   758 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.207439   768 leader_election.cc:290] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.208515   761 ts_tablet_manager.cc:1434] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
I20260709 12:59:55.208513   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:55.207300   566 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.209357   641 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.207315   758 ts_tablet_manager.cc:1403] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.049s	user 0.006s	sys 0.006s
I20260709 12:59:55.209790   761 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.213105   679 leader_election.cc:304] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.214391   768 raft_consensus.cc:2804] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.215459   768 raft_consensus.cc:697] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:55.216367   768 consensus_queue.cc:237] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.217949   758 raft_consensus.cc:359] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.218629   758 raft_consensus.cc:385] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.218907   758 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.223505   641 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.223649   759 tablet_bootstrap.cc:654] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.224131   761 tablet_bootstrap.cc:654] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.226235   758 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.234181   758 ts_tablet_manager.cc:1434] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 12:59:55.235146   758 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.245528   462 catalog_manager.cc:5697] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.253633   758 tablet_bootstrap.cc:654] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.258576   761 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.259117   761 ts_tablet_manager.cc:1403] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.004s
I20260709 12:59:55.263046   759 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.263573   759 ts_tablet_manager.cc:1403] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.063s	user 0.009s	sys 0.005s
I20260709 12:59:55.267218   759 raft_consensus.cc:359] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.268013   759 raft_consensus.cc:385] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.268491   759 raft_consensus.cc:740] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.268860   761 raft_consensus.cc:359] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.270201   761 raft_consensus.cc:385] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.269817   759 consensus_queue.cc:260] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.270498   761 raft_consensus.cc:740] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.271306   761 consensus_queue.cc:260] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.272512   758 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.272629   759 ts_tablet_manager.cc:1434] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:59:55.273160   758 ts_tablet_manager.cc:1403] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.009s
I20260709 12:59:55.273660   759 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.274098   761 ts_tablet_manager.cc:1434] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.006s
I20260709 12:59:55.274998   761 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.280782   758 raft_consensus.cc:359] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.281539   758 raft_consensus.cc:385] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.281819   758 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.282289   761 tablet_bootstrap.cc:654] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.282526   758 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.286123   759 tablet_bootstrap.cc:654] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.291411   767 raft_consensus.cc:493] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.292490   767 raft_consensus.cc:515] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.295116   767 leader_election.cc:290] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.308161   761 tablet_bootstrap.cc:492] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.308674   761 ts_tablet_manager.cc:1403] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.003s
I20260709 12:59:55.311630   761 raft_consensus.cc:359] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.312356   761 raft_consensus.cc:385] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.287947   758 ts_tablet_manager.cc:1434] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:59:55.312656   761 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.314177   758 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.313884   761 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.322443   758 tablet_bootstrap.cc:654] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.329118   761 ts_tablet_manager.cc:1434] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:59:55.333304   761 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.336330   767 raft_consensus.cc:493] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.336822   767 raft_consensus.cc:515] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.337335   759 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.337831   759 ts_tablet_manager.cc:1403] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.016s
I20260709 12:59:55.339493   767 leader_election.cc:290] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.345084   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.345961   566 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.346104   759 raft_consensus.cc:359] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.348085   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.347396   759 raft_consensus.cc:385] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.348830   759 raft_consensus.cc:740] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.350006   605 leader_election.cc:304] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.351511   761 tablet_bootstrap.cc:654] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.352012   758 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
W20260709 12:59:55.351897   605 leader_election.cc:343] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.351711   767 raft_consensus.cc:2804] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.352723   758 ts_tablet_manager.cc:1403] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.006s
I20260709 12:59:55.352783   767 raft_consensus.cc:493] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.353291   767 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.351907   759 consensus_queue.cc:260] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.358053   758 raft_consensus.cc:359] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.358795   758 raft_consensus.cc:385] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.360780   758 raft_consensus.cc:740] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.362301   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.362221   758 consensus_queue.cc:260] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.362931   715 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.364718   767 raft_consensus.cc:515] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.370177   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.371361   714 raft_consensus.cc:2468] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.373104   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.373636   565 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.373889   604 leader_election.cc:304] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 6ad1b341cd64494a810f03572475100a
I20260709 12:59:55.374177   767 leader_election.cc:290] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.381098   758 ts_tablet_manager.cc:1434] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.004s
I20260709 12:59:55.382423   758 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.389737   792 raft_consensus.cc:2804] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.390215   792 raft_consensus.cc:493] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.390530   792 raft_consensus.cc:3060] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.393774   565 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.398849   605 leader_election.cc:304] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.399784   793 raft_consensus.cc:2804] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.400588   793 raft_consensus.cc:697] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:55.402729   762 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.402043   793 consensus_queue.cc:237] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.403318   762 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.404093   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.404626   714 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.399380   759 ts_tablet_manager.cc:1434] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.061s	user 0.001s	sys 0.031s
I20260709 12:59:55.405572   762 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.407084   759 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.407366   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.408978   715 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 0.
I20260709 12:59:55.410470   529 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.411365   762 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.411726   762 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.412343   762 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.414500   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.415674   758 tablet_bootstrap.cc:654] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.411651   714 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
W20260709 12:59:55.417495   529 leader_election.cc:343] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.425429   768 raft_consensus.cc:493] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.422569   762 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.425928   768 raft_consensus.cc:515] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.428273   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:55.429929   762 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
W20260709 12:59:55.429831   529 leader_election.cc:343] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.430833   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.431550   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.431710   565 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 0.
I20260709 12:59:55.422546   792 raft_consensus.cc:515] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.432273   714 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.433153   768 leader_election.cc:290] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.433983   679 leader_election.cc:304] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.434393   761 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.434427   792 leader_election.cc:290] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.436425   761 ts_tablet_manager.cc:1403] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.004s
I20260709 12:59:55.436440   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.436797   768 raft_consensus.cc:2804] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.437268   768 raft_consensus.cc:493] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.437674   768 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:55.438298   605 leader_election.cc:343] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.441628   761 raft_consensus.cc:359] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.442718   761 raft_consensus.cc:385] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.443110   761 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.444191   761 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.445776   463 catalog_manager.cc:5697] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.447656   761 ts_tablet_manager.cc:1434] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:59:55.452091   761 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.453766   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.454339   715 raft_consensus.cc:3060] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.454470   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.455118   641 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 0.
I20260709 12:59:55.462013   715 raft_consensus.cc:2468] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.468919   759 tablet_bootstrap.cc:654] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.463366   768 raft_consensus.cc:515] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.474637   758 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.474828   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.475085   758 ts_tablet_manager.cc:1403] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.093s	user 0.015s	sys 0.013s
I20260709 12:59:55.474493   714 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 1.
I20260709 12:59:55.475458   565 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.476897   529 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:55.477939   762 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.478479   758 raft_consensus.cc:359] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.479532   758 raft_consensus.cc:385] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.479912   758 raft_consensus.cc:740] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.478458   761 tablet_bootstrap.cc:654] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.478852   768 leader_election.cc:290] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.481571   758 consensus_queue.cc:260] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.478856   604 leader_election.cc:304] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 6ad1b341cd64494a810f03572475100a
I20260709 12:59:55.486930   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:55.487636   793 raft_consensus.cc:2804] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.489920   641 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.490669   565 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.491102   793 raft_consensus.cc:697] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:55.493953   762 raft_consensus.cc:697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:55.493976   679 leader_election.cc:304] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.495559   768 raft_consensus.cc:2804] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.496742   768 raft_consensus.cc:697] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:55.498806   768 consensus_queue.cc:237] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.499454   793 consensus_queue.cc:237] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.501571   762 consensus_queue.cc:237] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.505584   641 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.509924   759 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.511333   759 ts_tablet_manager.cc:1403] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.104s	user 0.011s	sys 0.007s
I20260709 12:59:55.513725   759 raft_consensus.cc:359] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.514233   759 raft_consensus.cc:385] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.514432   759 raft_consensus.cc:740] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.514040   761 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.515456   759 consensus_queue.cc:260] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.515794   761 ts_tablet_manager.cc:1403] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.013s
I20260709 12:59:55.524196   759 ts_tablet_manager.cc:1434] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:59:55.526196   759 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.532527   758 ts_tablet_manager.cc:1434] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.057s	user 0.016s	sys 0.015s
I20260709 12:59:55.533517   758 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.534893   759 tablet_bootstrap.cc:654] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.535619   761 raft_consensus.cc:359] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.536473   761 raft_consensus.cc:385] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.536792   761 raft_consensus.cc:740] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.539397   463 catalog_manager.cc:5697] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.547394   761 consensus_queue.cc:260] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.547320   461 catalog_manager.cc:5697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.549750   462 catalog_manager.cc:5697] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.551712   761 ts_tablet_manager.cc:1434] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:59:55.552767   761 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.561492   758 tablet_bootstrap.cc:654] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.561450   761 tablet_bootstrap.cc:654] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.581104   799 raft_consensus.cc:493] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.581658   799 raft_consensus.cc:515] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.585153   799 leader_election.cc:290] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.586678   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.588485   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
W20260709 12:59:55.589172   529 leader_election.cc:343] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:55.590137   529 leader_election.cc:343] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:55.590691   529 leader_election.cc:304] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad1b341cd64494a810f03572475100a; no voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:55.591971   799 raft_consensus.cc:2749] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:55.597702   758 tablet_bootstrap.cc:492] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.598134   758 ts_tablet_manager.cc:1403] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.015s
I20260709 12:59:55.606949   759 tablet_bootstrap.cc:492] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.607851   759 ts_tablet_manager.cc:1403] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.082s	user 0.033s	sys 0.017s
I20260709 12:59:55.610312   761 tablet_bootstrap.cc:492] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.610756   761 ts_tablet_manager.cc:1403] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.008s
I20260709 12:59:55.610698   759 raft_consensus.cc:359] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.611526   759 raft_consensus.cc:385] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.611845   759 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.613310   761 raft_consensus.cc:359] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.614003   761 raft_consensus.cc:385] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.614128   758 raft_consensus.cc:359] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.614368   761 raft_consensus.cc:740] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.614794   758 raft_consensus.cc:385] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.614271   759 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.615103   758 raft_consensus.cc:740] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.615206   761 consensus_queue.cc:260] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.615981   758 consensus_queue.cc:260] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.617143   761 ts_tablet_manager.cc:1434] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:55.617743   758 ts_tablet_manager.cc:1434] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.008s
I20260709 12:59:55.617939   761 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: Bootstrap starting.
I20260709 12:59:55.618552   758 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: Bootstrap starting.
I20260709 12:59:55.617444   759 ts_tablet_manager.cc:1434] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:59:55.624563   759 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.626402   761 tablet_bootstrap.cc:654] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.627722   793 raft_consensus.cc:493] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.630990   793 raft_consensus.cc:515] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.632989   793 leader_election.cc:290] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.633576   758 tablet_bootstrap.cc:654] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.636071   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.637269   565 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.635295   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.639528   605 leader_election.cc:304] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.640712   793 raft_consensus.cc:2804] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.644006   793 raft_consensus.cc:493] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.645265   793 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.644651   714 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.646087   759 tablet_bootstrap.cc:654] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.662431   758 tablet_bootstrap.cc:492] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: No bootstrap required, opened a new log
I20260709 12:59:55.662914   758 ts_tablet_manager.cc:1403] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.001s
I20260709 12:59:55.663758   793 raft_consensus.cc:515] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.665548   758 raft_consensus.cc:359] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.666311   758 raft_consensus.cc:385] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.666570   758 raft_consensus.cc:740] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.667315   758 consensus_queue.cc:260] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.668504   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.669246   565 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.669399   758 ts_tablet_manager.cc:1434] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:55.673142   759 tablet_bootstrap.cc:492] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.673418   799 raft_consensus.cc:493] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.673839   759 ts_tablet_manager.cc:1403] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.010s
I20260709 12:59:55.673969   799 raft_consensus.cc:515] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.676463   799 leader_election.cc:290] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.676760   759 raft_consensus.cc:359] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.677378   759 raft_consensus.cc:385] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.677611   759 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.678233   759 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.678622   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.680080   759 ts_tablet_manager.cc:1434] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:55.680083   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ced31be3de4b4581ad5c3e4121b2f4" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.681051   714 raft_consensus.cc:2393] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad1b341cd64494a810f03572475100a in current term 1: Already voted for candidate d595fd28c9804e208dc6fbe9448860a3 in this term.
I20260709 12:59:55.681313   759 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: Bootstrap starting.
I20260709 12:59:55.682838   529 leader_election.cc:304] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad1b341cd64494a810f03572475100a; no voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:55.689950   768 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:55.693138   799 raft_consensus.cc:3060] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.694816   759 tablet_bootstrap.cc:654] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:55.698221   762 raft_consensus.cc:493] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.698715   762 raft_consensus.cc:515] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.699929   793 leader_election.cc:290] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.700488   565 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.701804   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.702385   605 leader_election.cc:304] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.703378   793 raft_consensus.cc:2804] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.703673   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.702355   714 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.704783   640 raft_consensus.cc:2468] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 0.
I20260709 12:59:55.704820   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.705500   715 raft_consensus.cc:2468] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 0.
I20260709 12:59:55.703971   793 raft_consensus.cc:697] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:55.707849   793 consensus_queue.cc:237] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.708385   529 leader_election.cc:304] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.703476   762 leader_election.cc:290] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.709370   798 raft_consensus.cc:2804] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.709898   798 raft_consensus.cc:493] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.710343   798 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.714200   714 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.723562   761 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: No bootstrap required, opened a new log
I20260709 12:59:55.724222   761 ts_tablet_manager.cc:1403] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent bootstrapping tablet: real 0.106s	user 0.032s	sys 0.015s
I20260709 12:59:55.727895   798 raft_consensus.cc:515] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.731526   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:55.732650   640 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.733142   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.733688   715 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.739782   640 raft_consensus.cc:2468] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 1.
I20260709 12:59:55.740391   715 raft_consensus.cc:2468] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 1.
I20260709 12:59:55.741847   529 leader_election.cc:304] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.742874   762 raft_consensus.cc:2804] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.747772   798 leader_election.cc:290] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 1 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.731392   799 raft_consensus.cc:2749] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:55.753244   762 raft_consensus.cc:697] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:55.754321   762 consensus_queue.cc:237] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.767252   766 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:55.727746   761 raft_consensus.cc:359] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.771070   761 raft_consensus.cc:385] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.771337   761 raft_consensus.cc:740] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.772305   761 consensus_queue.cc:260] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.777125   759 tablet_bootstrap.cc:492] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: No bootstrap required, opened a new log
I20260709 12:59:55.777570   759 ts_tablet_manager.cc:1403] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.040s
I20260709 12:59:55.779007   462 catalog_manager.cc:5697] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.780567   759 raft_consensus.cc:359] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.781179   759 raft_consensus.cc:385] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:55.781423   759 raft_consensus.cc:740] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Initialized, Role: FOLLOWER
I20260709 12:59:55.782037   759 consensus_queue.cc:260] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.783975   759 ts_tablet_manager.cc:1434] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 12:59:55.786854   461 catalog_manager.cc:5697] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.790463   761 ts_tablet_manager.cc:1434] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
W20260709 12:59:55.794158   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:55.802585   793 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:55.818969   768 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.819654   768 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.821864   768 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.822983   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3" is_pre_election: true
I20260709 12:59:55.823599   640 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 0.
I20260709 12:59:55.824366   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.825201   678 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.825881   565 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 0.
I20260709 12:59:55.826472   768 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.826828   768 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.827157   768 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.841912   793 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:55.844252   792 raft_consensus.cc:493] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.844863   792 raft_consensus.cc:515] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.847127   792 leader_election.cc:290] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.847915   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.848515   566 raft_consensus.cc:2468] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 0.
I20260709 12:59:55.848914   768 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.850090   605 leader_election.cc:304] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.850915   792 raft_consensus.cc:2804] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:55.851272   792 raft_consensus.cc:493] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:55.851590   792 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.851689   768 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:55.852785   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.853160   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:55.853366   566 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.853705   640 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.858260   792 raft_consensus.cc:515] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.860354   566 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.860996   792 leader_election.cc:290] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.861514   792 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:55.861518   679 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:55.862056   792 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.862933   768 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.863080   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:55.860311   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.862982   640 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 1.
I20260709 12:59:55.863907   714 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.865504   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:55.866119   566 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.866343   768 raft_consensus.cc:697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:55.867215   768 consensus_queue.cc:237] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.869030   792 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:55.870040   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" is_pre_election: true
I20260709 12:59:55.870208   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1b341cd64494a810f03572475100a" is_pre_election: true
I20260709 12:59:55.870818   715 raft_consensus.cc:2393] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d595fd28c9804e208dc6fbe9448860a3 in current term 1: Already voted for candidate 6ad1b341cd64494a810f03572475100a in this term.
I20260709 12:59:55.872217   605 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d595fd28c9804e208dc6fbe9448860a3; no voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a
I20260709 12:59:55.873149   792 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:55.873916   566 raft_consensus.cc:2468] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.875407   605 leader_election.cc:304] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:55.876579   793 raft_consensus.cc:2804] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:55.877158   793 raft_consensus.cc:697] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:55.878456   714 raft_consensus.cc:2468] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 1.
I20260709 12:59:55.878135   793 consensus_queue.cc:237] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:55.887109   459 catalog_manager.cc:5697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.898451   792 raft_consensus.cc:2749] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:55.901121   461 catalog_manager.cc:5697] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.977839   793 consensus_queue.cc:1048] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:55.983268   766 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:55.990921   792 consensus_queue.cc:1048] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.013298   766 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.024932   762 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.043344   762 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.087183   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:40990
I20260709 12:59:56.087890   640 raft_consensus.cc:606] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 12:59:56.102914   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:46682
I20260709 12:59:56.103482   715 raft_consensus.cc:606] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 12:59:56.108842   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:46682
I20260709 12:59:56.109488   715 raft_consensus.cc:606] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 12:59:56.114918   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:46682
I20260709 12:59:56.115414   715 raft_consensus.cc:606] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 12:59:56.128767   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:50294
I20260709 12:59:56.129348   565 raft_consensus.cc:606] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:56.140000   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:50294
I20260709 12:59:56.140983   565 raft_consensus.cc:606] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 12:59:56.146009   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:40990
I20260709 12:59:56.146646   640 raft_consensus.cc:606] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Received request to transfer leadership to TS 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 12:59:56.211076   793 consensus_queue.cc:1048] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.223414   835 consensus_queue.cc:1048] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.275569   826 raft_consensus.cc:993] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.276117   826 raft_consensus.cc:1081] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.278017   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50278
I20260709 12:59:56.278621   565 raft_consensus.cc:493] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.279028   565 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.283617   565 raft_consensus.cc:515] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.285522   565 leader_election.cc:290] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.286365   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:56.286720   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.286988   640 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.287292   714 raft_consensus.cc:3055] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.287606   714 raft_consensus.cc:740] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:56.288445   714 consensus_queue.cc:260] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.289776   714 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.294636   640 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.296023   529 leader_election.cc:304] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:56.296756   714 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.296808   762 raft_consensus.cc:2804] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.297364   762 raft_consensus.cc:697] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.298156   762 consensus_queue.cc:237] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.305126   461 catalog_manager.cc:5697] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 1 to 2, leader changed from 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.337481   762 consensus_queue.cc:1048] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.344631   799 raft_consensus.cc:993] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 12:59:56.345098   798 raft_consensus.cc:1081] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 12:59:56.348052   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40986
I20260709 12:59:56.348668   640 raft_consensus.cc:493] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.349087   640 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.353056   762 consensus_queue.cc:1048] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.356611   640 raft_consensus.cc:515] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.358014   641 raft_consensus.cc:1240] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad1b341cd64494a810f03572475100a for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:56.362484   792 raft_consensus.cc:993] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.362906   792 raft_consensus.cc:1081] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.365357   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.365547   640 leader_election.cc:290] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 2 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.366252   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77caab168104bf9ac8a9e74fe110ec5" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:56.366793   566 raft_consensus.cc:3055] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.367138   566 raft_consensus.cc:740] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.367568   762 consensus_queue.cc:1059] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:56.368139   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50292
I20260709 12:59:56.368678   565 raft_consensus.cc:493] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.371405   565 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.371583   566 consensus_queue.cc:260] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.372753   566 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.380143   566 raft_consensus.cc:2468] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 2.
I20260709 12:59:56.381382   605 leader_election.cc:304] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 12:59:56.382067   835 raft_consensus.cc:2804] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.380088   565 raft_consensus.cc:515] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.382535   835 raft_consensus.cc:697] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:56.383353   835 consensus_queue.cc:237] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.384804   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:56.385360   640 raft_consensus.cc:3055] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.385675   640 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:56.386409   640 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.386893   565 leader_election.cc:290] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.388669   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.387428   640 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.389190   714 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.391956   461 catalog_manager.cc:5697] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 1 to 2, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.396330   714 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.397393   529 leader_election.cc:304] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:56.398286   813 raft_consensus.cc:2804] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.397874   640 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.398758   813 raft_consensus.cc:697] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.400036   813 consensus_queue.cc:237] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.407351   461 catalog_manager.cc:5697] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 1 to 2, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.438195   826 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.438378   792 consensus_queue.cc:1048] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.449424   767 raft_consensus.cc:993] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:56.449887   767 raft_consensus.cc:1081] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Signalling peer 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:56.452293   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:46680
I20260709 12:59:56.453029   713 raft_consensus.cc:493] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.453389   713 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.455796   767 consensus_queue.cc:1048] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.460796   713 raft_consensus.cc:515] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.462278   714 raft_consensus.cc:1240] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Rejecting Update request from peer d595fd28c9804e208dc6fbe9448860a3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:56.462486   768 raft_consensus.cc:993] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.462922   768 raft_consensus.cc:1081] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.465610   713 leader_election.cc:290] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 2 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:56.469170   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:56.469842   641 raft_consensus.cc:3055] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.470152   641 raft_consensus.cc:740] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:56.470854   641 consensus_queue.cc:260] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.472617   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "329fabbc2bc94f8784c0aaeb6990ac09" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:56.474131   826 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:56.475389   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50278
I20260709 12:59:56.475939   565 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.476303   565 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.478946   762 raft_consensus.cc:993] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 12:59:56.479355   762 raft_consensus.cc:1081] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 12:59:56.471791   641 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.488607   565 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.490217   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40986
I20260709 12:59:56.490788   639 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.491163   565 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.491351   639 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.499869   638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:56.500550   638 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.506652   641 raft_consensus.cc:2468] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 2.
I20260709 12:59:56.506680   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.507680   678 leader_election.cc:304] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 6ad1b341cd64494a810f03572475100a
I20260709 12:59:56.508189   713 raft_consensus.cc:3055] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.508540   713 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:56.508605   639 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.508947   826 raft_consensus.cc:2804] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.509457   713 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.509838   826 raft_consensus.cc:697] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:56.510677   713 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.510918   826 consensus_queue.cc:237] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.512725   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:56.513449   565 raft_consensus.cc:3055] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.513792   565 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.514689   565 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.515811   565 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.515301   638 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.518373   713 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.519879   639 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 2 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.520598   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.521299   714 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.524797   565 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 2.
I20260709 12:59:56.525785   605 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:56.527151   767 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.527747   767 raft_consensus.cc:697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:56.527163   461 catalog_manager.cc:5697] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 1 to 2, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.528623   767 consensus_queue.cc:237] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.532644   714 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 2.
I20260709 12:59:56.527815   529 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:56.537132   813 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.537603   813 raft_consensus.cc:697] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.538326   813 consensus_queue.cc:237] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.542954   459 catalog_manager.cc:5697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 1 to 2, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.545656   461 catalog_manager.cc:5697] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 1 to 2, leader changed from 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.622385   766 raft_consensus.cc:993] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.622864   826 raft_consensus.cc:1081] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 12:59:56.624437   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50278
I20260709 12:59:56.625012   565 raft_consensus.cc:493] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.625382   565 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.632741   565 raft_consensus.cc:515] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.634841   565 leader_election.cc:290] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 12:59:56.635713   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:56.635983   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 12:59:56.636356   639 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.636574   714 raft_consensus.cc:3055] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.636924   714 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:56.637719   714 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.638931   714 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.641186   639 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.642187   529 leader_election.cc:304] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 12:59:56.642911   813 raft_consensus.cc:2804] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.643340   813 raft_consensus.cc:697] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:56.644084   813 consensus_queue.cc:237] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:56.646979   714 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 12:59:56.650630   461 catalog_manager.cc:5697] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 1 to 2, leader changed from 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.765537   714 raft_consensus.cc:1275] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.766870   813 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.777527   639 raft_consensus.cc:1275] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.780020   799 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.808687   714 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.810248   813 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:56.827528   639 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.829551   799 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:56.889704   565 raft_consensus.cc:1275] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.890872   767 consensus_queue.cc:1048] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.897915   714 raft_consensus.cc:3060] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.906324   714 raft_consensus.cc:1275] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.907618   767 consensus_queue.cc:1048] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.930419   565 raft_consensus.cc:3060] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.935001   565 raft_consensus.cc:1275] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.936213   826 consensus_queue.cc:1048] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.943864   639 raft_consensus.cc:1275] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.945923   826 consensus_queue.cc:1048] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:56.968055   714 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.969262   798 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.976922   639 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.978327   762 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.076159   639 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.077455   762 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.085702   714 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.087270   762 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.099385   714 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.100689   792 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.110165   565 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.111562   835 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:59.175347   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 12:59:59.176333   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.191978   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:38364
I20260709 12:59:59.192476   565 raft_consensus.cc:606] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Received request to transfer leadership to TS 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 12:59:59.201953   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 12:59:59.202481   639 raft_consensus.cc:606] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Received request to transfer leadership to TS 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 12:59:59.203558   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:59.204115   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:59.358901   854 raft_consensus.cc:993] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:59.359426   767 raft_consensus.cc:1081] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Signalling peer 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:59.361019   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:46680
I20260709 12:59:59.361408   714 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.361629   714 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.365619   714 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.366927   714 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 3 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:59.367790   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:59.368039   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:59.368325   565 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.368649   639 raft_consensus.cc:3055] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:59.368938   639 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 12:59:59.369558   639 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.370426   639 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.372596   565 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 3.
I20260709 12:59:59.373526   679 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:59.374171   826 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:59.374961   639 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 3.
I20260709 12:59:59.375272   826 raft_consensus.cc:697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:59.376135   826 consensus_queue.cc:237] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.383020   463 catalog_manager.cc:5697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 2 to 3, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:59.776784   861 raft_consensus.cc:993] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: : Instructing follower 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:59.777223   813 raft_consensus.cc:1081] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Signalling peer 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 12:59:59.778520   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:46668
I20260709 12:59:59.778906   714 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.779129   714 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.782974   714 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.784245   714 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 3 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 12:59:59.785080   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 12:59:59.785564   565 raft_consensus.cc:3055] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:59.785501   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 12:59:59.785846   565 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 12:59:59.786100   639 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.786558   565 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.787371   565 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.790700   639 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 3.
I20260709 12:59:59.790985   565 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 3.
I20260709 12:59:59.791950   679 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 12:59:59.792639   826 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:59.793105   826 raft_consensus.cc:697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 12:59:59.793855   826 consensus_queue.cc:237] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 12:59:59.802208   461 catalog_manager.cc:5697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 2 to 3, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:59.821020   565 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:59.822250   826 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:59.835196   639 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:59.836671   826 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.204936   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:00.212990   639 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.214368   826 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.216267   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:00:00.216728   565 raft_consensus.cc:606] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 13:00:00.223052   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:46682
I20260709 13:00:00.223400   565 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.224063   714 raft_consensus.cc:606] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 13:00:00.225725   826 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.228556   855 raft_consensus.cc:993] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.228971   855 raft_consensus.cc:1081] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.232383   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:00:00.233016   566 raft_consensus.cc:606] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 13:00:00.234241   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40976
I20260709 13:00:00.235013   639 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.235505   639 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.238785   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:00:00.239377   566 raft_consensus.cc:606] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 13:00:00.244239   639 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.246344   639 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:00.247685   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:00.248256   714 raft_consensus.cc:3055] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:00.248512   714 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 13:00:00.249164   714 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.250118   714 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.253165   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
mode: GRACEFUL
new_leader_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:46682
I20260709 13:00:00.253669   713 raft_consensus.cc:606] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Received request to transfer leadership to TS d595fd28c9804e208dc6fbe9448860a3
I20260709 13:00:00.255625   714 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 4.
I20260709 13:00:00.256443   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:00.256834   604 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:00.257139   565 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.257594   767 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:00.258831   767 raft_consensus.cc:697] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:00.259598   767 consensus_queue.cc:237] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.261767   565 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 4.
I20260709 13:00:00.265592   461 catalog_manager.cc:5697] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 3 to 4, leader changed from 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.287115   813 raft_consensus.cc:993] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.287560   861 raft_consensus.cc:1081] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.288992   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40986
I20260709 13:00:00.289454   639 raft_consensus.cc:493] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.289784   639 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.294171   639 raft_consensus.cc:515] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.295706   639 leader_election.cc:290] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:00.296566   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:00.296707   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:00.297134   565 raft_consensus.cc:3055] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.297235   714 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.297437   565 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:00.298136   565 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.298933   565 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.301335   714 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.302217   604 leader_election.cc:304] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:00.302868   767 raft_consensus.cc:2804] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.302989   565 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.303336   767 raft_consensus.cc:697] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:00.304193   767 consensus_queue.cc:237] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.311712   461 catalog_manager.cc:5697] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 2 to 3, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.323454   813 raft_consensus.cc:993] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.323868   813 raft_consensus.cc:1081] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.325173   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40986
I20260709 13:00:00.325579   639 raft_consensus.cc:493] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.325816   639 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.330294   639 raft_consensus.cc:515] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.331952   639 leader_election.cc:290] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:00.332739   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:00.332890   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1e626c407ad476d85798e0e811d7d12" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:00.333271   565 raft_consensus.cc:3055] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.333313   714 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.333683   565 raft_consensus.cc:740] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:00.334367   565 consensus_queue.cc:260] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.335403   565 raft_consensus.cc:3060] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.339143   714 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.340194   604 leader_election.cc:304] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:00.340858   767 raft_consensus.cc:2804] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.340857   565 raft_consensus.cc:2468] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.341467   767 raft_consensus.cc:697] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:00.342237   767 consensus_queue.cc:237] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.348717   461 catalog_manager.cc:5697] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 2 to 3, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.416370   855 raft_consensus.cc:993] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.416821   826 raft_consensus.cc:1081] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.418421   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40976
I20260709 13:00:00.418952   639 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.419399   639 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.423840   639 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.425420   639 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:00.426304   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:00.426367   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:00.427369   565 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.427451   714 raft_consensus.cc:3055] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:00.427860   714 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 13:00:00.428470   714 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.429332   714 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.431845   565 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 4.
I20260709 13:00:00.432848   605 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:00.433538   767 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:00.433490   714 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 4.
I20260709 13:00:00.434109   767 raft_consensus.cc:697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:00.434870   767 consensus_queue.cc:237] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.439947   461 catalog_manager.cc:5697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 3 to 4, leader changed from 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.530507   861 raft_consensus.cc:993] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: : Instructing follower d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.531013   813 raft_consensus.cc:1081] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Signalling peer d595fd28c9804e208dc6fbe9448860a3 to start an election
I20260709 13:00:00.532574   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
 from {username='slave'} at 127.0.0.1:40986
I20260709 13:00:00.533097   639 raft_consensus.cc:493] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.533412   639 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.538106   639 raft_consensus.cc:515] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.539529   639 leader_election.cc:290] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:00.540472   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:00.540616   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "d595fd28c9804e208dc6fbe9448860a3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:00.541038   565 raft_consensus.cc:3055] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.541105   714 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.541385   565 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:00.542074   565 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.543123   565 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.547006   714 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.548115   604 leader_election.cc:304] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:00.548688   565 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595fd28c9804e208dc6fbe9448860a3 in term 3.
I20260709 13:00:00.548834   767 raft_consensus.cc:2804] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.549353   767 raft_consensus.cc:697] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Becoming Leader. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:00.550127   767 consensus_queue.cc:237] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:00.556973   461 catalog_manager.cc:5697] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 reported cstate change: term changed from 2 to 3, leader changed from 6ad1b341cd64494a810f03572475100a (127.0.107.193) to d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "d595fd28c9804e208dc6fbe9448860a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.684819   565 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.686053   767 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:00.693401   714 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.694932   875 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:00.706700   714 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.707811   767 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.716037   565 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.718127   871 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.847659   565 raft_consensus.cc:1275] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.848435   714 raft_consensus.cc:1275] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.848963   767 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.849720   871 consensus_queue.cc:1048] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:00.909926   714 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.911048   875 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:00.920037   565 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.921743   871 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:00.994720   714 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.995945   871 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.004700   565 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Refusing update from remote peer d595fd28c9804e208dc6fbe9448860a3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:01.006211   881 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:00:02.419106   662 debug-util.cc:398] Leaking SignalData structure 0x7b080012d060 after lost signal to thread 434
W20260709 13:00:02.420092   662 debug-util.cc:398] Leaking SignalData structure 0x7b080012d740 after lost signal to thread 497
W20260709 13:00:02.420998   662 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6bc0 after lost signal to thread 590
W20260709 13:00:02.422461   662 debug-util.cc:398] Leaking SignalData structure 0x7b0800108ac0 after lost signal to thread 665
W20260709 13:00:02.423272   662 debug-util.cc:398] Leaking SignalData structure 0x7b08001d78c0 after lost signal to thread 739
I20260709 13:00:03.437100   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:03.445091   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 13:00:03.445989   639 raft_consensus.cc:606] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Received request to transfer leadership to TS 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 13:00:03.448922   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 13:00:03.449344   639 raft_consensus.cc:606] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Received request to transfer leadership to TS 3404525ae60a41ebb736a4d79bf3cd8a
I20260709 13:00:03.451546   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 13:00:03.452067   639 raft_consensus.cc:606] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 13:00:03.454362   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 13:00:03.454798   639 raft_consensus.cc:606] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 13:00:03.457067   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
mode: GRACEFUL
new_leader_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:52298
I20260709 13:00:03.457485   639 raft_consensus.cc:606] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Received request to transfer leadership to TS 6ad1b341cd64494a810f03572475100a
I20260709 13:00:03.458508   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:03.458966   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:03.561525   871 raft_consensus.cc:993] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.562240   891 raft_consensus.cc:1081] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.564329   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50292
I20260709 13:00:03.564960   564 raft_consensus.cc:493] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:03.565394   564 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.573966   564 raft_consensus.cc:515] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.576858   564 leader_election.cc:290] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 4 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:03.578192   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:03.579015   712 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.579226   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4121df3cce444999297d205d8dedeeb" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 13:00:03.579800   639 raft_consensus.cc:3055] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:03.580098   639 raft_consensus.cc:740] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:03.580968   639 consensus_queue.cc:260] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.582337   639 raft_consensus.cc:3060] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.586544   712 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 4.
I20260709 13:00:03.588125   529 leader_election.cc:304] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 13:00:03.595052   639 raft_consensus.cc:2468] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 4.
I20260709 13:00:03.597347   892 raft_consensus.cc:2804] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:03.598637   892 raft_consensus.cc:697] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 4 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:03.599848   892 consensus_queue.cc:237] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.614418   461 catalog_manager.cc:5697] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 3 to 4, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.647106   871 raft_consensus.cc:993] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.647611   871 raft_consensus.cc:1081] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.649255   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50292
I20260709 13:00:03.649860   564 raft_consensus.cc:493] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:03.650261   564 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.658103   564 raft_consensus.cc:515] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.660384   564 leader_election.cc:290] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:03.661015   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 13:00:03.661427   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c144d08f7c4c76a755d2a49212b236" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:03.661577   639 raft_consensus.cc:3055] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.661974   712 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.661976   639 raft_consensus.cc:740] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:03.662966   639 consensus_queue.cc:260] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.664238   639 raft_consensus.cc:3060] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.668906   712 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 13:00:03.670111   529 leader_election.cc:304] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 13:00:03.670933   892 raft_consensus.cc:2804] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:03.671434   892 raft_consensus.cc:697] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:03.670583   639 raft_consensus.cc:2468] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 2.
I20260709 13:00:03.672300   892 consensus_queue.cc:237] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.680817   461 catalog_manager.cc:5697] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 1 to 2, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.905800   891 raft_consensus.cc:993] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 13:00:03.906313   891 raft_consensus.cc:1081] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Signalling peer 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 13:00:03.907873   712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38901755dc724684ab5f9407ea15a220"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:00:03.908437   712 raft_consensus.cc:493] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:03.908799   712 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.916002   712 raft_consensus.cc:515] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.918413   712 leader_election.cc:290] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 4 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163)
I20260709 13:00:03.919109   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 13:00:03.919553   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38901755dc724684ab5f9407ea15a220" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:03.919663   639 raft_consensus.cc:3055] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:03.920070   639 raft_consensus.cc:740] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:03.920141   564 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.920944   639 consensus_queue.cc:260] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.922135   639 raft_consensus.cc:3060] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.926620   564 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 4.
I20260709 13:00:03.927934   679 leader_election.cc:304] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 13:00:03.928181   639 raft_consensus.cc:2468] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 4.
I20260709 13:00:03.929026   890 raft_consensus.cc:2804] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:03.930351   890 raft_consensus.cc:697] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 13:00:03.931162   890 consensus_queue.cc:237] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.939374   461 catalog_manager.cc:5697] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 3 to 4, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 4 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:03.965528   871 raft_consensus.cc:993] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.965997   891 raft_consensus.cc:1081] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Signalling peer 6ad1b341cd64494a810f03572475100a to start an election
I20260709 13:00:03.967720   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe"
dest_uuid: "6ad1b341cd64494a810f03572475100a"
 from {username='slave'} at 127.0.0.1:50292
I20260709 13:00:03.968351   564 raft_consensus.cc:493] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:03.968772   564 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.976420   564 raft_consensus.cc:515] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.978752   564 leader_election.cc:290] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 5 election: Requested vote from peers d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575), 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195:42801)
I20260709 13:00:03.979382   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 13:00:03.979959   639 raft_consensus.cc:3055] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:03.979827   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a360ceb58a4bbaa26a2d4b4a4941fe" candidate_uuid: "6ad1b341cd64494a810f03572475100a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
I20260709 13:00:03.980319   639 raft_consensus.cc:740] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:03.980532   712 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.981160   639 consensus_queue.cc:260] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.981993   639 raft_consensus.cc:3060] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.986055   639 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 5.
I20260709 13:00:03.987165   529 leader_election.cc:304] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad1b341cd64494a810f03572475100a, d595fd28c9804e208dc6fbe9448860a3; no voters: 
I20260709 13:00:03.987892   712 raft_consensus.cc:2468] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1b341cd64494a810f03572475100a in term 5.
I20260709 13:00:03.988031   892 raft_consensus.cc:2804] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:03.988519   892 raft_consensus.cc:697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 5 LEADER]: Becoming Leader. State: Replica: 6ad1b341cd64494a810f03572475100a, State: Running, Role: LEADER
I20260709 13:00:03.989337   892 consensus_queue.cc:237] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:03.997283   461 catalog_manager.cc:5697] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a reported cstate change: term changed from 4 to 5, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 6ad1b341cd64494a810f03572475100a (127.0.107.193). New cstate: current_term: 5 leader_uuid: "6ad1b341cd64494a810f03572475100a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:04.013654   871 raft_consensus.cc:993] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: : Instructing follower 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 13:00:04.014173   871 raft_consensus.cc:1081] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Signalling peer 3404525ae60a41ebb736a4d79bf3cd8a to start an election
I20260709 13:00:04.015712   712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e"
dest_uuid: "3404525ae60a41ebb736a4d79bf3cd8a"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:00:04.016346   712 raft_consensus.cc:493] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:04.016731   712 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.024217   712 raft_consensus.cc:515] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:04.026437   712 leader_election.cc:290] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 5 election: Requested vote from peers 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163), d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575)
I20260709 13:00:04.027204   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6ad1b341cd64494a810f03572475100a"
I20260709 13:00:04.027647   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ece978b14a4aee944391b0dfc79c0e" candidate_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d595fd28c9804e208dc6fbe9448860a3"
I20260709 13:00:04.027823   564 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.028223   639 raft_consensus.cc:3055] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:04.028519   639 raft_consensus.cc:740] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d595fd28c9804e208dc6fbe9448860a3, State: Running, Role: LEADER
I20260709 13:00:04.029264   639 consensus_queue.cc:260] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:04.030428   639 raft_consensus.cc:3060] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.034521   564 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 5.
I20260709 13:00:04.035604   679 leader_election.cc:304] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3404525ae60a41ebb736a4d79bf3cd8a, 6ad1b341cd64494a810f03572475100a; no voters: 
I20260709 13:00:04.036422   890 raft_consensus.cc:2804] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:04.036870   890 raft_consensus.cc:697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 LEADER]: Becoming Leader. State: Replica: 3404525ae60a41ebb736a4d79bf3cd8a, State: Running, Role: LEADER
I20260709 13:00:04.036991   639 raft_consensus.cc:2468] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3404525ae60a41ebb736a4d79bf3cd8a in term 5.
I20260709 13:00:04.037739   890 consensus_queue.cc:237] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } }
I20260709 13:00:04.044971   461 catalog_manager.cc:5697] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a reported cstate change: term changed from 4 to 5, leader changed from d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194) to 3404525ae60a41ebb736a4d79bf3cd8a (127.0.107.195). New cstate: current_term: 5 leader_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:04.099627   639 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:04.099676   712 raft_consensus.cc:1275] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:04.101424   893 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:04.102226   892 consensus_queue.cc:1048] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.134665   715 raft_consensus.cc:1275] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:04.137084   893 consensus_queue.cc:1048] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:04.149842   639 raft_consensus.cc:1275] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:04.151975   910 consensus_queue.cc:1048] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:04.445164   564 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:04.446447   890 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1b341cd64494a810f03572475100a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.458647   639 raft_consensus.cc:1275] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:04.460148   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:04.465401   890 consensus_queue.cc:1048] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.469273   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:00:04.507061   605 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:00:04.513190   715 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:04.517289   910 consensus_queue.cc:1048] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404525ae60a41ebb736a4d79bf3cd8a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:04.522063   605 consensus_peers.cc:597] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 -> Peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Couldn't send request to peer 6ad1b341cd64494a810f03572475100a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:04.523393   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:04.524561   431 tablet_replica.cc:333] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.525828   431 raft_consensus.cc:2243] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.526659   431 raft_consensus.cc:2272] T 39ced31be3de4b4581ad5c3e4121b2f4 P 6ad1b341cd64494a810f03572475100a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.531529   431 tablet_replica.cc:333] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.532568   431 raft_consensus.cc:2243] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.533171   431 raft_consensus.cc:2272] T 329fabbc2bc94f8784c0aaeb6990ac09 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.535354   431 tablet_replica.cc:333] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.536118   431 raft_consensus.cc:2243] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.537017   431 raft_consensus.cc:2272] T 55c144d08f7c4c76a755d2a49212b236 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.541075   431 tablet_replica.cc:333] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.541926   431 raft_consensus.cc:2243] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.543053   431 raft_consensus.cc:2272] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.543280   639 raft_consensus.cc:1275] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Refusing update from remote peer 6ad1b341cd64494a810f03572475100a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:04.546141   431 tablet_replica.cc:333] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.546777   431 raft_consensus.cc:2243] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.547333   431 raft_consensus.cc:2272] T 20ece978b14a4aee944391b0dfc79c0e P 6ad1b341cd64494a810f03572475100a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.550123   431 tablet_replica.cc:333] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.550787   431 raft_consensus.cc:2243] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.551419   431 raft_consensus.cc:2272] T 38901755dc724684ab5f9407ea15a220 P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.554446   431 tablet_replica.cc:333] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.555271   431 raft_consensus.cc:2243] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.555943   431 raft_consensus.cc:2272] T e77caab168104bf9ac8a9e74fe110ec5 P 6ad1b341cd64494a810f03572475100a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.558410   431 tablet_replica.cc:333] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.559122   431 raft_consensus.cc:2243] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.559705   431 raft_consensus.cc:2272] T f1e626c407ad476d85798e0e811d7d12 P 6ad1b341cd64494a810f03572475100a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.562197   431 tablet_replica.cc:333] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a: stopping tablet replica
I20260709 13:00:04.562811   431 raft_consensus.cc:2243] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.563802   431 raft_consensus.cc:2272] T c4121df3cce444999297d205d8dedeeb P 6ad1b341cd64494a810f03572475100a [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:04.581916   529 connection.cc:466] server connection from 127.0.0.1:50278 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50278 (request call id 50) could send its response
W20260709 13:00:04.590149   679 consensus_peers.cc:597] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a -> Peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Couldn't send request to peer 6ad1b341cd64494a810f03572475100a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:04.598588   639 raft_consensus.cc:1275] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Refusing update from remote peer 3404525ae60a41ebb736a4d79bf3cd8a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:04.600023   890 consensus_queue.cc:1048] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d595fd28c9804e208dc6fbe9448860a3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:04.661060   679 consensus_peers.cc:597] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a -> Peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Couldn't send request to peer 6ad1b341cd64494a810f03572475100a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:04.747146   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:00:04.751149   605 consensus_peers.cc:597] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 -> Peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Couldn't send request to peer 6ad1b341cd64494a810f03572475100a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:04.769601   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:00:04.805398   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:04.806303   431 tablet_replica.cc:333] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.807252   431 raft_consensus.cc:2243] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.807946   431 raft_consensus.cc:2272] T 329fabbc2bc94f8784c0aaeb6990ac09 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.810724   431 tablet_replica.cc:333] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.811510   431 raft_consensus.cc:2243] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.812022   431 raft_consensus.cc:2272] T b5a360ceb58a4bbaa26a2d4b4a4941fe P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.814445   431 tablet_replica.cc:333] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.815146   431 raft_consensus.cc:2243] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.815654   431 raft_consensus.cc:2272] T 55c144d08f7c4c76a755d2a49212b236 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.818313   431 tablet_replica.cc:333] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.819047   431 raft_consensus.cc:2243] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.820056   431 raft_consensus.cc:2272] T 39ced31be3de4b4581ad5c3e4121b2f4 P d595fd28c9804e208dc6fbe9448860a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.822553   431 tablet_replica.cc:333] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.823328   431 raft_consensus.cc:2243] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.824396   431 raft_consensus.cc:2272] T e77caab168104bf9ac8a9e74fe110ec5 P d595fd28c9804e208dc6fbe9448860a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.826700   431 tablet_replica.cc:333] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.827383   431 raft_consensus.cc:2243] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.828390   431 raft_consensus.cc:2272] T f1e626c407ad476d85798e0e811d7d12 P d595fd28c9804e208dc6fbe9448860a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.830751   431 tablet_replica.cc:333] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.831432   431 raft_consensus.cc:2243] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.831946   431 raft_consensus.cc:2272] T c4121df3cce444999297d205d8dedeeb P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.834533   431 tablet_replica.cc:333] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.835248   431 raft_consensus.cc:2243] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.835736   431 raft_consensus.cc:2272] T 20ece978b14a4aee944391b0dfc79c0e P d595fd28c9804e208dc6fbe9448860a3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.838351   431 tablet_replica.cc:333] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3: stopping tablet replica
I20260709 13:00:04.838963   431 raft_consensus.cc:2243] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.839448   431 raft_consensus.cc:2272] T 38901755dc724684ab5f9407ea15a220 P d595fd28c9804e208dc6fbe9448860a3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:04.852854   678 consensus_peers.cc:597] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a -> Peer d595fd28c9804e208dc6fbe9448860a3 (127.0.107.194:41575): Couldn't send request to peer d595fd28c9804e208dc6fbe9448860a3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:04.880205   679 consensus_peers.cc:597] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a -> Peer 6ad1b341cd64494a810f03572475100a (127.0.107.193:42163): Couldn't send request to peer 6ad1b341cd64494a810f03572475100a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:04.898865   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:00:04.917057   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:00:04.943835   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:04.944612   431 tablet_replica.cc:333] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.945371   431 raft_consensus.cc:2243] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.946148   431 raft_consensus.cc:2272] T 329fabbc2bc94f8784c0aaeb6990ac09 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.948115   431 tablet_replica.cc:333] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.948676   431 raft_consensus.cc:2243] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.949115   431 raft_consensus.cc:2272] T 55c144d08f7c4c76a755d2a49212b236 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.951875   431 tablet_replica.cc:333] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.952371   431 raft_consensus.cc:2243] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.952812   431 raft_consensus.cc:2272] T c4121df3cce444999297d205d8dedeeb P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.955202   431 tablet_replica.cc:333] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.955716   431 raft_consensus.cc:2243] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.956215   431 raft_consensus.cc:2272] T b5a360ceb58a4bbaa26a2d4b4a4941fe P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.958006   431 tablet_replica.cc:333] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.958498   431 raft_consensus.cc:2243] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.959200   431 raft_consensus.cc:2272] T 20ece978b14a4aee944391b0dfc79c0e P 3404525ae60a41ebb736a4d79bf3cd8a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.961035   431 tablet_replica.cc:333] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.961508   431 raft_consensus.cc:2243] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:04.962188   431 raft_consensus.cc:2272] T 38901755dc724684ab5f9407ea15a220 P 3404525ae60a41ebb736a4d79bf3cd8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.964095   431 tablet_replica.cc:333] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.964548   431 raft_consensus.cc:2243] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.964928   431 raft_consensus.cc:2272] T 39ced31be3de4b4581ad5c3e4121b2f4 P 3404525ae60a41ebb736a4d79bf3cd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.966434   431 tablet_replica.cc:333] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.966914   431 raft_consensus.cc:2243] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.967334   431 raft_consensus.cc:2272] T e77caab168104bf9ac8a9e74fe110ec5 P 3404525ae60a41ebb736a4d79bf3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.969085   431 tablet_replica.cc:333] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a: stopping tablet replica
I20260709 13:00:04.969525   431 raft_consensus.cc:2243] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.969949   431 raft_consensus.cc:2272] T f1e626c407ad476d85798e0e811d7d12 P 3404525ae60a41ebb736a4d79bf3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.996061   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:05.012426   431 master.cc:562] Master@127.0.107.254:38733 shutting down...
I20260709 13:00:05.118937   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:05.119447   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:05.119728   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2cff963597994b30a7d083542e68ccb8: stopping tablet replica
I20260709 13:00:05.139180   431 master.cc:584] Master@127.0.107.254:38733 shutdown complete.
3404525ae60a41ebb736a4d79bf3cd8a  1
6ad1b341cd64494a810f03572475100a  4
d595fd28c9804e208dc6fbe9448860a3  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12939 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:00:05.177948   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39037
I20260709 13:00:05.179050   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.184836   923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.185105   922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.186410   925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.188282   431 server_base.cc:1061] running on GCE node
I20260709 13:00:05.189240   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.189456   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.189622   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005189605 us; error 0 us; skew 500 ppm
I20260709 13:00:05.190263   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.192991   431 webserver.cc:533] Webserver started at http://127.0.107.254:35975/ using document root <none> and password file <none>
I20260709 13:00:05.193553   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.193773   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.194080   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.195263   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.200269   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:00:05.203428   930 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.204313   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:00:05.204553   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.204788   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.221443   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.222707   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.264940   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39037
I20260709 13:00:05.265035   981 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39037 every 8 connection(s)
I20260709 13:00:05.268803   982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.278551   982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee: Bootstrap starting.
I20260709 13:00:05.283519   982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:05.288082   982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee: No bootstrap required, opened a new log
I20260709 13:00:05.290308   982 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER }
I20260709 13:00:05.290726   982 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:05.290964   982 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcdc85c6d6349ec95c02f3fdf3e8cee, State: Initialized, Role: FOLLOWER
I20260709 13:00:05.291625   982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER }
I20260709 13:00:05.292140   982 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:05.292361   982 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:05.292685   982 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:05.296835   982 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER }
I20260709 13:00:05.297412   982 leader_election.cc:304] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fcdc85c6d6349ec95c02f3fdf3e8cee; no voters: 
I20260709 13:00:05.298697   982 leader_election.cc:290] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:05.299085   985 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:05.300657   985 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 1 LEADER]: Becoming Leader. State: Replica: 3fcdc85c6d6349ec95c02f3fdf3e8cee, State: Running, Role: LEADER
I20260709 13:00:05.301445   985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER }
I20260709 13:00:05.301888   982 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:05.307066   987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fcdc85c6d6349ec95c02f3fdf3e8cee. Latest consensus state: current_term: 1 leader_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER } }
I20260709 13:00:05.307848   987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:05.310490   986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcdc85c6d6349ec95c02f3fdf3e8cee" member_type: VOTER } }
I20260709 13:00:05.311100   986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:05.315083   994 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:05.320358   994 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:05.321367   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:05.329206   994 catalog_manager.cc:1383] Generated new cluster ID: ca723b18e81e4ee6825b275615c43519
I20260709 13:00:05.329438   994 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:05.348942   994 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:05.350179   994 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:05.363969   994 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee: Generated new TSK 0
I20260709 13:00:05.364526   994 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:05.388200   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.394341  1003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.395861  1004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.396739  1006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.397188   431 server_base.cc:1061] running on GCE node
I20260709 13:00:05.398195   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.398423   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.398599   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005398575 us; error 0 us; skew 500 ppm
I20260709 13:00:05.399101   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.401597   431 webserver.cc:533] Webserver started at http://127.0.107.193:34693/ using document root <none> and password file <none>
I20260709 13:00:05.402123   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.402343   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.402619   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.403945   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "13184db88a7442c5aafcdda1dc3763c7"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.408444   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:00:05.411819  1011 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.412719   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:00:05.412986   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "13184db88a7442c5aafcdda1dc3763c7"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.413230   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.421352   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.422494   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.423931   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:05.426239   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:05.426407   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.426591   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:05.426736   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.466204   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:35785
I20260709 13:00:05.466310  1073 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:35785 every 8 connection(s)
I20260709 13:00:05.471052   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.479282  1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.480134  1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.482941  1081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.483667   431 server_base.cc:1061] running on GCE node
I20260709 13:00:05.484824   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.485034   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.485215   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005485176 us; error 0 us; skew 500 ppm
I20260709 13:00:05.485738   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.488337   431 webserver.cc:533] Webserver started at http://127.0.107.194:46367/ using document root <none> and password file <none>
I20260709 13:00:05.488986   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.489276   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.489594   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.489823  1074 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39037
I20260709 13:00:05.490196  1074 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.490787   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "dd7150ccd47d47809dc827e8d4cff65d"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.490907  1074 heartbeater.cc:507] Master 127.0.107.254:39037 requested a full tablet report, sending...
I20260709 13:00:05.493059   947 ts_manager.cc:194] Registered new tserver with Master: 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785)
I20260709 13:00:05.494941   947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59282
I20260709 13:00:05.495867   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.005s
I20260709 13:00:05.499330  1086 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.500150   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:00:05.500453   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "dd7150ccd47d47809dc827e8d4cff65d"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.500731   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.517645   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.518850   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.520342   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:05.522536   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:05.522716   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.523010   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:05.523154   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.562009   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43027
I20260709 13:00:05.562121  1148 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43027 every 8 connection(s)
I20260709 13:00:05.566779   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.573724  1152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.577440   431 server_base.cc:1061] running on GCE node
W20260709 13:00:05.577677  1153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.577415  1155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.578656  1149 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39037
I20260709 13:00:05.579001  1149 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.579007   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.579387   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.579557   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005579542 us; error 0 us; skew 500 ppm
I20260709 13:00:05.579918  1149 heartbeater.cc:507] Master 127.0.107.254:39037 requested a full tablet report, sending...
I20260709 13:00:05.580214   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.582136   947 ts_manager.cc:194] Registered new tserver with Master: dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:05.583055   431 webserver.cc:533] Webserver started at http://127.0.107.195:45587/ using document root <none> and password file <none>
I20260709 13:00:05.583727   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.583987   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.584048   947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59286
I20260709 13:00:05.584375   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.585778   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "be113e9528624ac9ae0cc95fa32fde88"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.589984   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:00:05.593031  1160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.593765   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:00:05.594079   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "be113e9528624ac9ae0cc95fa32fde88"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-kgkf"
I20260709 13:00:05.594372   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.605252   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.606433   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.607954   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:05.610083   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:05.610312   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.610549   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:05.610721   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.652479   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:35701
I20260709 13:00:05.652575  1222 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:35701 every 8 connection(s)
I20260709 13:00:05.665854  1223 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39037
I20260709 13:00:05.666180  1223 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.666836  1223 heartbeater.cc:507] Master 127.0.107.254:39037 requested a full tablet report, sending...
I20260709 13:00:05.668648   947 ts_manager.cc:194] Registered new tserver with Master: be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:05.669097   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013564422s
I20260709 13:00:05.670425   947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59296
I20260709 13:00:05.695953   947 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59308:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:05.698786   947 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:05.766456  1188 tablet_service.cc:1511] Processing CreateTablet for tablet 61e88870331a4a07baee7c415fab351a (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:05.767980  1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e88870331a4a07baee7c415fab351a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.771921  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 2a89d6b110f14e3e9de25256390a8822 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:05.773370  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a89d6b110f14e3e9de25256390a8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.778023  1186 tablet_service.cc:1511] Processing CreateTablet for tablet 7f05567e9791422d90ae15c5b7f0cf3e (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:05.779493  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f05567e9791422d90ae15c5b7f0cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.780468  1037 tablet_service.cc:1511] Processing CreateTablet for tablet 7f05567e9791422d90ae15c5b7f0cf3e (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:05.780887  1038 tablet_service.cc:1511] Processing CreateTablet for tablet 2a89d6b110f14e3e9de25256390a8822 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:05.781903  1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f05567e9791422d90ae15c5b7f0cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.780915  1036 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fb55609fd47a79e6d31f937d9f8a1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:05.783324  1039 tablet_service.cc:1511] Processing CreateTablet for tablet 61e88870331a4a07baee7c415fab351a (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:05.783792  1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fb55609fd47a79e6d31f937d9f8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.789050  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 61e88870331a4a07baee7c415fab351a (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:05.790427  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e88870331a4a07baee7c415fab351a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.791460  1111 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fb55609fd47a79e6d31f937d9f8a1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:05.792847  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fb55609fd47a79e6d31f937d9f8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.792930  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 2a89d6b110f14e3e9de25256390a8822 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:05.794232  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a89d6b110f14e3e9de25256390a8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.795178  1035 tablet_service.cc:1511] Processing CreateTablet for tablet 643d374a27244afa95eef29c9ab45f34 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:05.796556  1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643d374a27244afa95eef29c9ab45f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.796860  1112 tablet_service.cc:1511] Processing CreateTablet for tablet 7f05567e9791422d90ae15c5b7f0cf3e (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:05.798137  1112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f05567e9791422d90ae15c5b7f0cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.800997  1110 tablet_service.cc:1511] Processing CreateTablet for tablet 643d374a27244afa95eef29c9ab45f34 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:05.802388  1110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643d374a27244afa95eef29c9ab45f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.802672  1185 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fb55609fd47a79e6d31f937d9f8a1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:05.805006  1109 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:05.805004  1184 tablet_service.cc:1511] Processing CreateTablet for tablet 643d374a27244afa95eef29c9ab45f34 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:05.806456  1109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.806569  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fb55609fd47a79e6d31f937d9f8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.807857  1184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643d374a27244afa95eef29c9ab45f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.821534  1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a89d6b110f14e3e9de25256390a8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.822000  1034 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:05.823011  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e88870331a4a07baee7c415fab351a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.822089  1183 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:05.825536  1033 tablet_service.cc:1511] Processing CreateTablet for tablet 29c5f249ae664bdf97cee148696c59c1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:05.827342  1183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.828395  1108 tablet_service.cc:1511] Processing CreateTablet for tablet 29c5f249ae664bdf97cee148696c59c1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:05.832234  1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2f6c729dbd4ed8a42a9013b24dbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.836310  1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c5f249ae664bdf97cee148696c59c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.840096  1108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c5f249ae664bdf97cee148696c59c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.852901  1186 tablet_service.cc:1511] Processing CreateTablet for tablet 29c5f249ae664bdf97cee148696c59c1 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:05.854326  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c5f249ae664bdf97cee148696c59c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.883246  1036 tablet_service.cc:1511] Processing CreateTablet for tablet fc370965a6dd433fb17e4b1ce8ad7e26 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:05.884038  1107 tablet_service.cc:1511] Processing CreateTablet for tablet fc370965a6dd433fb17e4b1ce8ad7e26 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:05.884588  1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc370965a6dd433fb17e4b1ce8ad7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.885284  1107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc370965a6dd433fb17e4b1ce8ad7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.890728  1106 tablet_service.cc:1511] Processing CreateTablet for tablet d712a4087df946b7a58e04edcf4c398d (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:05.892056  1106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d712a4087df946b7a58e04edcf4c398d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.892776  1034 tablet_service.cc:1511] Processing CreateTablet for tablet d712a4087df946b7a58e04edcf4c398d (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:05.895015  1032 tablet_service.cc:1511] Processing CreateTablet for tablet 22b64d12ed044e0d9b9ea2a461f476dd (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:05.896351  1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b64d12ed044e0d9b9ea2a461f476dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.896854  1105 tablet_service.cc:1511] Processing CreateTablet for tablet 22b64d12ed044e0d9b9ea2a461f476dd (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:05.898087  1105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b64d12ed044e0d9b9ea2a461f476dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.899016  1182 tablet_service.cc:1511] Processing CreateTablet for tablet 22b64d12ed044e0d9b9ea2a461f476dd (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:05.900300  1182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b64d12ed044e0d9b9ea2a461f476dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.915112  1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d712a4087df946b7a58e04edcf4c398d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.910328  1183 tablet_service.cc:1511] Processing CreateTablet for tablet d712a4087df946b7a58e04edcf4c398d (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:05.920847  1183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d712a4087df946b7a58e04edcf4c398d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.929167  1186 tablet_service.cc:1511] Processing CreateTablet for tablet fc370965a6dd433fb17e4b1ce8ad7e26 (DEFAULT_TABLE table=test-workload [id=6500265f7671495fb6afce55c5a4b270]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:05.930595  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc370965a6dd433fb17e4b1ce8ad7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.954458  1239 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:05.958748  1240 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:05.960737  1238 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:05.962126  1239 tablet_bootstrap.cc:654] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:05.965957  1240 tablet_bootstrap.cc:654] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:05.969712  1238 tablet_bootstrap.cc:654] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:05.973094  1239 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:05.973414  1240 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:05.973577  1239 ts_tablet_manager.cc:1403] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.009s
I20260709 13:00:05.973841  1240 ts_tablet_manager.cc:1403] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:00:05.974656  1238 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:05.975044  1238 ts_tablet_manager.cc:1403] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:00:05.976481  1239 raft_consensus.cc:359] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.976651  1240 raft_consensus.cc:359] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.977057  1238 raft_consensus.cc:359] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.977289  1240 raft_consensus.cc:385] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:05.977778  1238 raft_consensus.cc:385] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:05.977852  1240 raft_consensus.cc:740] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:05.978127  1238 raft_consensus.cc:740] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:05.978719  1240 consensus_queue.cc:260] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.978822  1238 consensus_queue.cc:260] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.977259  1239 raft_consensus.cc:385] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:05.979948  1239 raft_consensus.cc:740] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:05.980908  1239 consensus_queue.cc:260] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:05.981266  1149 heartbeater.cc:499] Master 127.0.107.254:39037 was elected leader, sending a full tablet report...
I20260709 13:00:05.981509  1240 ts_tablet_manager.cc:1434] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 13:00:05.982733  1240 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:05.992203  1240 tablet_bootstrap.cc:654] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:05.995092  1074 heartbeater.cc:499] Master 127.0.107.254:39037 was elected leader, sending a full tablet report...
I20260709 13:00:05.995563  1238 ts_tablet_manager.cc:1434] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.020s	user 0.000s	sys 0.020s
I20260709 13:00:05.996693  1238 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:05.999303  1223 heartbeater.cc:499] Master 127.0.107.254:39037 was elected leader, sending a full tablet report...
I20260709 13:00:06.004904  1238 tablet_bootstrap.cc:654] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.009986  1238 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.010219  1240 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.010603  1238 ts_tablet_manager.cc:1403] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 13:00:06.010643  1240 ts_tablet_manager.cc:1403] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.008s
I20260709 13:00:06.013180  1239 ts_tablet_manager.cc:1434] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.017s
I20260709 13:00:06.014220  1239 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.013927  1238 raft_consensus.cc:359] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.014029  1240 raft_consensus.cc:359] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.015007  1238 raft_consensus.cc:385] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.015090  1240 raft_consensus.cc:385] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.015403  1238 raft_consensus.cc:740] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.015547  1240 raft_consensus.cc:740] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.016183  1238 consensus_queue.cc:260] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.016469  1240 consensus_queue.cc:260] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.018276  1238 ts_tablet_manager.cc:1434] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:06.019178  1238 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.021591  1239 tablet_bootstrap.cc:654] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.022392  1240 ts_tablet_manager.cc:1434] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.006s
I20260709 13:00:06.023447  1240 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.025332  1238 tablet_bootstrap.cc:654] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.032594  1240 tablet_bootstrap.cc:654] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.077515  1240 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.077515  1239 tablet_bootstrap.cc:492] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.077672  1238 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.078199  1240 ts_tablet_manager.cc:1403] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.028s
I20260709 13:00:06.078199  1239 ts_tablet_manager.cc:1403] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.064s	user 0.034s	sys 0.029s
I20260709 13:00:06.078317  1238 ts_tablet_manager.cc:1403] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.059s	user 0.022s	sys 0.035s
I20260709 13:00:06.079825  1244 raft_consensus.cc:493] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.080283  1244 raft_consensus.cc:515] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.081192  1240 raft_consensus.cc:359] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.081934  1240 raft_consensus.cc:385] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.081604  1239 raft_consensus.cc:359] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.081801  1238 raft_consensus.cc:359] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.082314  1240 raft_consensus.cc:740] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.082585  1239 raft_consensus.cc:385] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.082623  1238 raft_consensus.cc:385] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.082903  1244 leader_election.cc:290] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.083271  1238 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.083227  1239 raft_consensus.cc:740] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.084235  1238 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.085783  1239 consensus_queue.cc:260] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.086525  1238 ts_tablet_manager.cc:1434] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:06.087715  1238 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.088888  1239 ts_tablet_manager.cc:1434] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:00:06.090742  1240 consensus_queue.cc:260] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.091663  1244 raft_consensus.cc:493] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.092131  1244 raft_consensus.cc:515] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.094352  1239 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.096237  1244 leader_election.cc:290] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.101361  1239 tablet_bootstrap.cc:654] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.102633  1245 raft_consensus.cc:493] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.102783  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.103232  1245 raft_consensus.cc:515] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.104264  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
W20260709 13:00:06.107244  1090 leader_election.cc:343] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.107475  1238 tablet_bootstrap.cc:654] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.103497  1198 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 0.
I20260709 13:00:06.109373  1245 leader_election.cc:290] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.110194  1090 leader_election.cc:304] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.113066  1244 raft_consensus.cc:2804] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.113530  1244 raft_consensus.cc:493] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.113967  1244 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.128552  1245 raft_consensus.cc:493] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.129103  1245 raft_consensus.cc:515] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.131157  1245 leader_election.cc:290] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.136422  1244 raft_consensus.cc:515] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.139516  1244 leader_election.cc:290] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.140682  1240 ts_tablet_manager.cc:1434] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.062s	user 0.016s	sys 0.011s
I20260709 13:00:06.140548  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:06.141366  1198 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.149631  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.149984  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f05567e9791422d90ae15c5b7f0cf3e" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.150717  1196 raft_consensus.cc:2468] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 0.
I20260709 13:00:06.150797  1198 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 1.
W20260709 13:00:06.150806  1015 leader_election.cc:343] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.152014  1015 leader_election.cc:304] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:06.152253  1090 leader_election.cc:304] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.152829  1245 raft_consensus.cc:2804] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.153185  1245 raft_consensus.cc:493] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.153317  1244 raft_consensus.cc:2804] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.153538  1245 raft_consensus.cc:3060] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.153800  1244 raft_consensus.cc:697] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:06.154271  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.154668  1244 consensus_queue.cc:237] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.155726  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.156373  1048 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 0.
I20260709 13:00:06.157428  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
W20260709 13:00:06.159180  1087 leader_election.cc:343] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.160377  1087 leader_election.cc:304] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:06.161162  1245 raft_consensus.cc:515] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.161870  1239 tablet_bootstrap.cc:492] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.162088  1254 raft_consensus.cc:2804] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.162352  1239 ts_tablet_manager.cc:1403] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.009s
I20260709 13:00:06.162621  1254 raft_consensus.cc:493] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.163084  1254 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.165096  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f05567e9791422d90ae15c5b7f0cf3e" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:06.165732  1198 raft_consensus.cc:3060] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.165441  1239 raft_consensus.cc:359] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.166170  1239 raft_consensus.cc:385] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.166472  1239 raft_consensus.cc:740] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
W20260709 13:00:06.161890  1087 leader_election.cc:343] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.170332  1245 leader_election.cc:290] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.159312  1240 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.172811  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.173997  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f05567e9791422d90ae15c5b7f0cf3e" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.174664  1123 raft_consensus.cc:2468] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 0.
I20260709 13:00:06.174899  1238 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.175441  1238 ts_tablet_manager.cc:1403] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.088s	user 0.031s	sys 0.029s
I20260709 13:00:06.175877  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f05567e9791422d90ae15c5b7f0cf3e" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.176085  1198 raft_consensus.cc:2468] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:06.176497  1122 raft_consensus.cc:3060] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.177433  1015 leader_election.cc:304] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:06.178329  1245 raft_consensus.cc:2804] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.167217  1239 consensus_queue.cc:260] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.180940  1239 ts_tablet_manager.cc:1434] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:00:06.180688   946 catalog_manager.cc:5697] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.184537  1122 raft_consensus.cc:2468] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:06.185165  1239 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.193257  1239 tablet_bootstrap.cc:654] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.193260  1240 tablet_bootstrap.cc:654] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.195122  1254 raft_consensus.cc:515] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.198483  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.199045  1047 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.198995  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:06.200011  1198 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.178282  1238 raft_consensus.cc:359] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.200879  1238 raft_consensus.cc:385] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.201215  1238 raft_consensus.cc:740] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.201921  1238 consensus_queue.cc:260] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.206344  1124 raft_consensus.cc:2393] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13184db88a7442c5aafcdda1dc3763c7 in current term 1: Already voted for candidate dd7150ccd47d47809dc827e8d4cff65d in this term.
I20260709 13:00:06.203825  1254 leader_election.cc:290] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.210346  1012 leader_election.cc:304] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7; no voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.219741  1264 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.220399  1264 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.224085  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
W20260709 13:00:06.225936  1012 leader_election.cc:343] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.226389  1047 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 1.
I20260709 13:00:06.228063  1087 leader_election.cc:304] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.230130  1254 raft_consensus.cc:2804] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.217705  1245 raft_consensus.cc:697] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:06.234290  1263 raft_consensus.cc:2749] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.232681  1254 raft_consensus.cc:697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:06.235033  1238 ts_tablet_manager.cc:1434] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.020s
I20260709 13:00:06.235216  1264 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.235242  1254 consensus_queue.cc:237] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.236598  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.237190  1238 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
W20260709 13:00:06.237936  1015 leader_election.cc:343] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.238389  1015 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7; no voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.239753  1264 raft_consensus.cc:2749] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.241299  1245 consensus_queue.cc:237] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.241969  1198 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 1.
I20260709 13:00:06.245538  1239 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.245986  1239 ts_tablet_manager.cc:1403] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.061s	user 0.001s	sys 0.024s
I20260709 13:00:06.246634  1238 tablet_bootstrap.cc:654] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.248392  1239 raft_consensus.cc:359] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.249117  1239 raft_consensus.cc:385] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.249435  1239 raft_consensus.cc:740] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.250515  1239 consensus_queue.cc:260] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.253542  1239 ts_tablet_manager.cc:1434] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:06.254892  1239 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.256742   947 catalog_manager.cc:5697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.260917  1240 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.264842  1240 ts_tablet_manager.cc:1403] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.007s
I20260709 13:00:06.268675  1240 raft_consensus.cc:359] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.269645  1240 raft_consensus.cc:385] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.263260  1239 tablet_bootstrap.cc:654] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.270051  1240 raft_consensus.cc:740] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.275117  1240 consensus_queue.cc:260] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.279894  1239 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.280692  1239 ts_tablet_manager.cc:1403] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.009s
I20260709 13:00:06.285241  1239 raft_consensus.cc:359] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.286391  1239 raft_consensus.cc:385] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.286890  1239 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.288290  1239 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.290930  1240 ts_tablet_manager.cc:1434] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:00:06.291363  1239 ts_tablet_manager.cc:1434] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.008s
I20260709 13:00:06.292133  1240 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.292388  1239 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.298691  1240 tablet_bootstrap.cc:654] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.299795  1239 tablet_bootstrap.cc:654] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.300354  1238 tablet_bootstrap.cc:492] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.300920  1238 ts_tablet_manager.cc:1403] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.030s
I20260709 13:00:06.303369  1246 raft_consensus.cc:493] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.304071  1246 raft_consensus.cc:515] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.304754  1238 raft_consensus.cc:359] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.305867  1238 raft_consensus.cc:385] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.306309  1238 raft_consensus.cc:740] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.307509  1238 consensus_queue.cc:260] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.311347  1238 ts_tablet_manager.cc:1434] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
W20260709 13:00:06.311964   995 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:06.312893  1238 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.317345  1239 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.317867  1239 ts_tablet_manager.cc:1403] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.012s
I20260709 13:00:06.311148  1246 leader_election.cc:290] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.321436  1239 raft_consensus.cc:359] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.322338  1239 raft_consensus.cc:385] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.322682  1239 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.323446  1239 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.324103  1238 tablet_bootstrap.cc:654] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.326100  1239 ts_tablet_manager.cc:1434] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:06.327256  1239 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.332427   947 catalog_manager.cc:5697] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.336002  1239 tablet_bootstrap.cc:654] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.337009  1238 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.337582  1238 ts_tablet_manager.cc:1403] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.005s
I20260709 13:00:06.341403  1238 raft_consensus.cc:359] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.342515  1238 raft_consensus.cc:385] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.342934  1238 raft_consensus.cc:740] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.344280  1238 consensus_queue.cc:260] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.347335  1238 ts_tablet_manager.cc:1434] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 13:00:06.348497  1238 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.348475  1239 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.349314  1239 ts_tablet_manager.cc:1403] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.011s
I20260709 13:00:06.352550  1239 raft_consensus.cc:359] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.353451  1239 raft_consensus.cc:385] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.353816  1239 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.354751  1239 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.356370  1238 tablet_bootstrap.cc:654] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.357337  1239 ts_tablet_manager.cc:1434] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:06.358539  1239 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.367506  1239 tablet_bootstrap.cc:654] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.374689  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.376264  1238 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.376758  1238 ts_tablet_manager.cc:1403] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.013s
W20260709 13:00:06.376794  1161 leader_election.cc:343] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.379232  1239 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.379695  1239 ts_tablet_manager.cc:1403] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 13:00:06.379724  1238 raft_consensus.cc:359] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.380578  1238 raft_consensus.cc:385] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.380870  1238 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.381703  1238 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.382385  1239 raft_consensus.cc:359] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.383039  1239 raft_consensus.cc:385] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.383302  1239 raft_consensus.cc:740] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.384120  1238 ts_tablet_manager.cc:1434] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:06.384155  1239 consensus_queue.cc:260] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.385196  1238 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.386418  1239 ts_tablet_manager.cc:1434] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:00:06.387338  1239 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.392002  1238 tablet_bootstrap.cc:654] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.392792  1239 tablet_bootstrap.cc:654] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.404683  1239 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.404683  1238 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.405210  1238 ts_tablet_manager.cc:1403] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:00:06.405210  1239 ts_tablet_manager.cc:1403] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:00:06.407749  1238 raft_consensus.cc:359] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.407909  1239 raft_consensus.cc:359] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.408474  1238 raft_consensus.cc:385] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.408510  1239 raft_consensus.cc:385] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.408893  1239 raft_consensus.cc:740] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.408864  1238 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.409749  1238 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.409749  1239 consensus_queue.cc:260] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.411700  1238 ts_tablet_manager.cc:1434] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:00:06.412499  1240 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.412632  1238 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.412971  1240 ts_tablet_manager.cc:1403] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.121s	user 0.041s	sys 0.053s
I20260709 13:00:06.413221  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.413714  1239 ts_tablet_manager.cc:1434] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:06.413857  1047 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.414669  1239 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: Bootstrap starting.
I20260709 13:00:06.415264  1161 leader_election.cc:304] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.416116  1246 raft_consensus.cc:2804] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.415906  1240 raft_consensus.cc:359] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.416486  1246 raft_consensus.cc:493] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.416667  1240 raft_consensus.cc:385] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.416879  1246 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.416996  1240 raft_consensus.cc:740] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.417776  1240 consensus_queue.cc:260] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.418648  1238 tablet_bootstrap.cc:654] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.419986  1240 ts_tablet_manager.cc:1434] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:06.420881  1240 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.421166  1239 tablet_bootstrap.cc:654] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.428651  1240 tablet_bootstrap.cc:654] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.436739  1282 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.437386  1282 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.445318  1284 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.445828  1284 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.447212  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.447899  1047 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.449857  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.450502  1246 raft_consensus.cc:515] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.454955  1246 leader_election.cc:290] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.455670  1284 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.458321  1240 tablet_bootstrap.cc:492] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.458801  1240 ts_tablet_manager.cc:1403] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 13:00:06.439653  1282 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.462001  1240 raft_consensus.cc:359] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.462723  1240 raft_consensus.cc:385] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.462981  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.463646  1047 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.464123  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.463017  1240 raft_consensus.cc:740] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.464680  1048 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.465705  1240 consensus_queue.cc:260] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.471887  1048 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.474406  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.474972  1124 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.475714  1240 ts_tablet_manager.cc:1434] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.006s
I20260709 13:00:06.477294  1240 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.482107  1124 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.483738  1240 tablet_bootstrap.cc:654] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.490032  1245 raft_consensus.cc:493] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.490693  1245 raft_consensus.cc:515] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.493613  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
W20260709 13:00:06.495391  1161 leader_election.cc:343] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.496225  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.496300  1161 leader_election.cc:304] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.497756  1282 raft_consensus.cc:2804] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.498153  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.498394  1282 raft_consensus.cc:697] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:06.499277  1198 raft_consensus.cc:2393] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13184db88a7442c5aafcdda1dc3763c7 in current term 1: Already voted for candidate dd7150ccd47d47809dc827e8d4cff65d in this term.
I20260709 13:00:06.499594  1282 consensus_queue.cc:237] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.502017  1015 leader_election.cc:304] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7; no voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.503499  1260 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:06.505548  1161 leader_election.cc:343] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.506896  1245 leader_election.cc:290] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.506469  1161 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.513087  1161 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.513841  1284 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.514196  1284 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.514525  1284 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.515558  1260 raft_consensus.cc:2749] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.519054  1240 tablet_bootstrap.cc:492] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.519675  1240 ts_tablet_manager.cc:1403] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.008s
I20260709 13:00:06.513271   947 catalog_manager.cc:5697] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 0 to 1, leader changed from <none> to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.520745  1284 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.522719  1284 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.520836  1282 raft_consensus.cc:2804] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.523291  1282 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.523662  1282 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.524240  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.524819  1048 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.523448  1240 raft_consensus.cc:359] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.525257  1240 raft_consensus.cc:385] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.525640  1240 raft_consensus.cc:740] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.526543  1240 consensus_queue.cc:260] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.529501  1240 ts_tablet_manager.cc:1434] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:00:06.530819  1240 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.531953  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
W20260709 13:00:06.533461  1161 leader_election.cc:343] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.537451  1254 raft_consensus.cc:493] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.537983  1254 raft_consensus.cc:515] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.539981  1254 leader_election.cc:290] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701), 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785)
I20260709 13:00:06.541560  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
W20260709 13:00:06.542943  1087 leader_election.cc:343] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.544523  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
W20260709 13:00:06.545977  1090 leader_election.cc:343] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.546398  1090 leader_election.cc:304] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7150ccd47d47809dc827e8d4cff65d; no voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:06.547488  1254 raft_consensus.cc:2749] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.549741  1048 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.550254  1238 tablet_bootstrap.cc:492] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.550714  1238 ts_tablet_manager.cc:1403] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.138s	user 0.064s	sys 0.070s
I20260709 13:00:06.551188  1239 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: No bootstrap required, opened a new log
I20260709 13:00:06.551096  1161 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.551672  1239 ts_tablet_manager.cc:1403] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: Time spent bootstrapping tablet: real 0.137s	user 0.039s	sys 0.042s
I20260709 13:00:06.552110  1284 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.547451  1240 tablet_bootstrap.cc:654] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.553321  1284 raft_consensus.cc:697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:06.553594  1282 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.554149  1239 raft_consensus.cc:359] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.554194  1284 consensus_queue.cc:237] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.554908  1239 raft_consensus.cc:385] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.555186  1239 raft_consensus.cc:740] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.555998  1239 consensus_queue.cc:260] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.557204  1282 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.557865  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.558233  1239 ts_tablet_manager.cc:1434] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:00:06.558451  1048 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.559049  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.553292  1238 raft_consensus.cc:359] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.559911  1238 raft_consensus.cc:385] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.560191  1238 raft_consensus.cc:740] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
W20260709 13:00:06.560616  1161 leader_election.cc:343] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.560902  1238 consensus_queue.cc:260] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.562805  1238 ts_tablet_manager.cc:1434] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:00:06.564568  1240 tablet_bootstrap.cc:492] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.565032  1240 ts_tablet_manager.cc:1403] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.003s
I20260709 13:00:06.567718  1048 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.567878  1240 raft_consensus.cc:359] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.568691  1240 raft_consensus.cc:385] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.568990  1240 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.569401  1161 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.569669  1240 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.570950  1282 raft_consensus.cc:2804] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.571597  1282 raft_consensus.cc:697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:06.571838  1240 ts_tablet_manager.cc:1434] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:06.572781  1240 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.572736  1282 consensus_queue.cc:237] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.576531  1246 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.577272  1246 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.579684  1260 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.579741  1246 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.585446  1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.580174  1260 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.586917  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
W20260709 13:00:06.588330  1161 leader_election.cc:343] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.580257  1240 tablet_bootstrap.cc:654] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.581652  1238 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: Bootstrap starting.
I20260709 13:00:06.589165  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
W20260709 13:00:06.592037  1012 leader_election.cc:343] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.587253  1047 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.590256  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.592916  1198 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 0.
I20260709 13:00:06.588519  1260 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.594060  1161 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.595909  1015 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.597000  1260 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.597361  1260 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.597729  1260 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.597842  1238 tablet_bootstrap.cc:654] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.600855   946 catalog_manager.cc:5697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 0 to 1, leader changed from <none> to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.602445  1282 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.603193  1282 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.603644  1282 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.610769  1260 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.622653  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
W20260709 13:00:06.624320  1012 leader_election.cc:343] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.624135   947 catalog_manager.cc:5697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 0 to 1, leader changed from <none> to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.625164  1240 tablet_bootstrap.cc:492] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.625689  1240 ts_tablet_manager.cc:1403] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.019s
I20260709 13:00:06.628069  1240 raft_consensus.cc:359] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.628695  1240 raft_consensus.cc:385] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.628942  1240 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.629752  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:06.629649  1240 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.630532  1260 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.633692  1240 ts_tablet_manager.cc:1434] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:06.633782  1245 consensus_queue.cc:1048] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.634495  1240 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: Bootstrap starting.
I20260709 13:00:06.635030  1254 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.640903  1240 tablet_bootstrap.cc:654] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.643231  1254 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.645354  1244 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.644500  1245 raft_consensus.cc:493] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.646574  1282 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.646497  1245 raft_consensus.cc:515] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.651602  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.652230  1124 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 0.
I20260709 13:00:06.653692  1012 leader_election.cc:304] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.654510  1264 raft_consensus.cc:2804] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.654841  1264 raft_consensus.cc:493] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.655241  1264 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.657620  1245 leader_election.cc:290] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.657985  1198 raft_consensus.cc:2393] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13184db88a7442c5aafcdda1dc3763c7 in current term 1: Already voted for candidate be113e9528624ac9ae0cc95fa32fde88 in this term.
I20260709 13:00:06.660290  1015 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7; no voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.660370  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.661439  1197 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 0.
I20260709 13:00:06.663174  1245 raft_consensus.cc:2749] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.665210  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.666326  1049 raft_consensus.cc:2393] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be113e9528624ac9ae0cc95fa32fde88 in current term 1: Already voted for candidate 13184db88a7442c5aafcdda1dc3763c7 in this term.
I20260709 13:00:06.666115  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.667374  1282 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
W20260709 13:00:06.667547  1161 leader_election.cc:343] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.673391  1161 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88; no voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:06.674135  1282 raft_consensus.cc:2749] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.677969  1264 raft_consensus.cc:515] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.716286  1282 raft_consensus.cc:493] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.718336  1282 raft_consensus.cc:515] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.721541  1282 leader_election.cc:290] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.721812  1238 tablet_bootstrap.cc:492] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: No bootstrap required, opened a new log
I20260709 13:00:06.722065  1240 tablet_bootstrap.cc:492] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: No bootstrap required, opened a new log
I20260709 13:00:06.722282  1238 ts_tablet_manager.cc:1403] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: Time spent bootstrapping tablet: real 0.141s	user 0.020s	sys 0.035s
I20260709 13:00:06.722513  1240 ts_tablet_manager.cc:1403] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.005s
I20260709 13:00:06.723553  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b64d12ed044e0d9b9ea2a461f476dd" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.724198  1124 raft_consensus.cc:2468] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.724885  1238 raft_consensus.cc:359] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.725564  1238 raft_consensus.cc:385] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.725302  1240 raft_consensus.cc:359] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.725852  1238 raft_consensus.cc:740] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.725973  1240 raft_consensus.cc:385] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.725929  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b64d12ed044e0d9b9ea2a461f476dd" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.726392  1240 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.726718  1049 raft_consensus.cc:2468] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.727082  1240 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.728081  1161 leader_election.cc:304] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.729828  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.730489  1124 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.731406  1264 leader_election.cc:290] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.732821  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:06.733349  1197 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.736914  1282 raft_consensus.cc:2804] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.737314  1282 raft_consensus.cc:493] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.737668  1282 raft_consensus.cc:3060] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.738277  1245 consensus_queue.cc:1048] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.735298  1240 ts_tablet_manager.cc:1434] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:00:06.731891  1238 consensus_queue.cc:260] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.758520  1124 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:06.759822  1012 leader_election.cc:304] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:06.760362  1197 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:06.763782  1289 raft_consensus.cc:493] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.764266  1289 raft_consensus.cc:515] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.765725  1264 raft_consensus.cc:2804] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.766436  1289 leader_election.cc:290] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.766594  1264 raft_consensus.cc:697] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:06.767541  1264 consensus_queue.cc:237] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.770376  1244 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:06.774144  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.774787  1048 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.759843  1238 ts_tablet_manager.cc:1434] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:00:06.777310  1161 leader_election.cc:304] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:06.779366  1289 raft_consensus.cc:2804] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.779878  1289 raft_consensus.cc:493] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.780303  1289 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.793090  1282 raft_consensus.cc:515] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.796120  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b64d12ed044e0d9b9ea2a461f476dd" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.796753  1048 raft_consensus.cc:3060] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.798607   947 catalog_manager.cc:5697] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.808835  1048 raft_consensus.cc:2468] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.810034  1161 leader_election.cc:304] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:06.809948  1289 raft_consensus.cc:515] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.810920  1284 raft_consensus.cc:2804] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.811396  1284 raft_consensus.cc:697] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:06.812354  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:06.812252  1284 consensus_queue.cc:237] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.812943  1124 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 0.
I20260709 13:00:06.814040  1282 leader_election.cc:290] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.815824  1289 leader_election.cc:290] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:06.817050  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:06.817627  1049 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.818187  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b64d12ed044e0d9b9ea2a461f476dd" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.818789  1124 raft_consensus.cc:3060] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.824585  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:06.825258  1122 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.826277  1124 raft_consensus.cc:2468] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.826507  1049 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.829301  1161 leader_election.cc:304] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:06.830157  1284 raft_consensus.cc:2804] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.830616  1284 raft_consensus.cc:697] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:06.831951  1122 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 1.
I20260709 13:00:06.831462  1284 consensus_queue.cc:237] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:06.833675   947 catalog_manager.cc:5697] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 0 to 1, leader changed from <none> to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.848206   947 catalog_manager.cc:5697] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 0 to 1, leader changed from <none> to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.861958  1244 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.862445  1244 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.864149  1244 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.864916  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.865146  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.865708  1049 raft_consensus.cc:2393] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7150ccd47d47809dc827e8d4cff65d in current term 1: Already voted for candidate be113e9528624ac9ae0cc95fa32fde88 in this term.
I20260709 13:00:06.866670  1087 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7150ccd47d47809dc827e8d4cff65d; no voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:06.867357  1244 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.871976  1244 raft_consensus.cc:2749] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.918942  1244 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.919397  1244 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.920185  1284 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.921775  1244 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.922385  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.922515  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.923270  1049 raft_consensus.cc:2393] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7150ccd47d47809dc827e8d4cff65d in current term 1: Already voted for candidate be113e9528624ac9ae0cc95fa32fde88 in this term.
I20260709 13:00:06.924922  1087 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7150ccd47d47809dc827e8d4cff65d; no voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:06.925652  1244 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.932528  1244 raft_consensus.cc:2749] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.932870  1282 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.981459  1244 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.981945  1244 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:06.982649  1282 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.984107  1244 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:06.984778  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13184db88a7442c5aafcdda1dc3763c7" is_pre_election: true
I20260709 13:00:06.985059  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:06.985653  1048 raft_consensus.cc:2393] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7150ccd47d47809dc827e8d4cff65d in current term 1: Already voted for candidate 13184db88a7442c5aafcdda1dc3763c7 in this term.
I20260709 13:00:06.985738  1198 raft_consensus.cc:2393] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7150ccd47d47809dc827e8d4cff65d in current term 1: Already voted for candidate be113e9528624ac9ae0cc95fa32fde88 in this term.
I20260709 13:00:06.986936  1090 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7150ccd47d47809dc827e8d4cff65d; no voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:06.987933  1244 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.991410  1246 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:06.995908  1244 raft_consensus.cc:2749] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.997709  1284 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.014093  1282 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.204739  1264 consensus_queue.cc:1048] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.215952  1263 consensus_queue.cc:1048] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.267243  1284 consensus_queue.cc:1048] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.277647  1246 consensus_queue.cc:1048] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.305325  1246 consensus_queue.cc:1048] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:07.319605  1284 consensus_queue.cc:1048] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:07.320987   995 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:08.316438  1263 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.316898  1263 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.318723  1263 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.319406  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d" is_pre_election: true
I20260709 13:00:08.319662  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88" is_pre_election: true
I20260709 13:00:08.319972  1122 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:08.320237  1198 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 1.
I20260709 13:00:08.321066  1012 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.321734  1263 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:08.322047  1263 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:08.322332  1263 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.328271  1263 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
W20260709 13:00:08.328528   995 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:08.329879  1263 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.330717  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:08.330893  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.331202  1122 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.331459  1198 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.336164  1122 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.336270  1198 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.337359  1012 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.338227  1263 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.338713  1263 raft_consensus.cc:697] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:08.339463  1263 consensus_queue.cc:237] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.346391   946 catalog_manager.cc:5697] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 0 to 2, leader changed from <none> to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.519564  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61e88870331a4a07baee7c415fab351a"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:08.520131  1198 raft_consensus.cc:606] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:08.533676  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:39550
I20260709 13:00:08.534217  1122 raft_consensus.cc:606] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:08.541628  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:08.542092  1198 raft_consensus.cc:606] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:08.546531  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:08.547014  1198 raft_consensus.cc:606] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:08.553920  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:08.554482  1198 raft_consensus.cc:606] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Received request to transfer leadership to TS dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:08.567623  1049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:34262
I20260709 13:00:08.568171  1049 raft_consensus.cc:606] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Received request to transfer leadership to TS dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:08.572781  1049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:34262
I20260709 13:00:08.573233  1049 raft_consensus.cc:606] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Received request to transfer leadership to TS dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:08.670764  1282 raft_consensus.cc:993] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:08.671201  1282 raft_consensus.cc:1081] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:08.672667  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34256
I20260709 13:00:08.673141  1049 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.673445  1049 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.677960  1049 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.679565  1049 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.680457  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:08.680586  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.680950  1122 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.681180  1198 raft_consensus.cc:3055] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.681572  1198 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:08.682225  1198 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.683106  1198 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.686466  1122 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.687458  1012 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.688211  1263 raft_consensus.cc:2804] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.688593  1263 raft_consensus.cc:697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:08.688597  1198 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.689433  1263 consensus_queue.cc:237] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.696488   946 catalog_manager.cc:5697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 1 to 2, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.803884  1122 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:08.805104  1263 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:08.812896  1336 raft_consensus.cc:993] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.813814  1323 raft_consensus.cc:1081] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Signalling peer dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.816068  1124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:00:08.816778  1198 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:08.817760  1124 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.818202  1124 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.820739  1336 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:08.826231  1124 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.831183  1124 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 3 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.831899  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:08.832495  1049 raft_consensus.cc:3055] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:08.832844  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.832942  1049 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:08.833462  1198 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.833873  1049 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.835093  1049 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.839273  1333 raft_consensus.cc:993] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:08.839756  1282 raft_consensus.cc:1081] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:08.840554  1198 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 3.
I20260709 13:00:08.841300  1048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34256
I20260709 13:00:08.841856  1048 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.841877  1090 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.842080  1049 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 3.
I20260709 13:00:08.842321  1048 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.843008  1244 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:08.847996  1244 raft_consensus.cc:697] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:08.848711  1244 consensus_queue.cc:237] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.849321  1048 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.852159  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:08.852305  1048 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.852743  1122 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.853787  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.854382  1198 raft_consensus.cc:3055] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.854724  1198 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:08.855512  1198 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.856740  1198 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.856891   947 catalog_manager.cc:5697] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 2 to 3, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.859663  1122 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.859917  1336 raft_consensus.cc:993] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.860356  1336 raft_consensus.cc:1081] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Signalling peer dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.861346  1012 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.862666  1336 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.862969  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:00:08.863361  1336 raft_consensus.cc:697] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:08.863494  1198 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:08.863545  1122 raft_consensus.cc:493] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.864068  1122 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.864270  1336 consensus_queue.cc:237] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.872427  1122 raft_consensus.cc:515] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.874329  1122 leader_election.cc:290] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 2 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.875056  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:08.874656   947 catalog_manager.cc:5697] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 1 to 2, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.875363  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d712a4087df946b7a58e04edcf4c398d" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.875586  1048 raft_consensus.cc:3055] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.875905  1198 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.876022  1048 raft_consensus.cc:740] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:08.876822  1048 consensus_queue.cc:260] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.877710  1048 raft_consensus.cc:3060] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.881973  1198 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 2.
I20260709 13:00:08.883473  1048 raft_consensus.cc:2468] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 2.
I20260709 13:00:08.883397  1090 leader_election.cc:304] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.884652  1342 raft_consensus.cc:2804] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.885043  1342 raft_consensus.cc:697] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:08.885674  1342 consensus_queue.cc:237] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.893201   947 catalog_manager.cc:5697] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 1 to 2, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.919596  1333 raft_consensus.cc:993] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.920082  1282 raft_consensus.cc:1081] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Signalling peer dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:08.921519  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:39544
I20260709 13:00:08.922012  1122 raft_consensus.cc:493] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.922333  1122 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.926859  1122 raft_consensus.cc:515] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.928511  1122 leader_election.cc:290] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 2 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:08.929524  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:08.929605  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:08.930090  1048 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.930172  1198 raft_consensus.cc:3055] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.930534  1198 raft_consensus.cc:740] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:08.931210  1198 consensus_queue.cc:260] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.932093  1198 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.934875  1048 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 2.
I20260709 13:00:08.935923  1087 leader_election.cc:304] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:08.936733  1342 raft_consensus.cc:2804] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.937036  1198 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 2.
I20260709 13:00:08.937320  1342 raft_consensus.cc:697] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:08.938277  1342 consensus_queue.cc:237] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:08.945218   947 catalog_manager.cc:5697] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 1 to 2, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.046485  1244 raft_consensus.cc:993] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:09.047001  1342 raft_consensus.cc:1081] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:09.048609  1048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34248
I20260709 13:00:09.049114  1048 raft_consensus.cc:493] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:09.049423  1048 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.054167  1048 raft_consensus.cc:515] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:09.055846  1048 leader_election.cc:290] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:09.056738  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:09.056907  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:09.057363  1122 raft_consensus.cc:3055] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:09.057463  1198 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.057782  1122 raft_consensus.cc:740] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:09.058511  1122 consensus_queue.cc:260] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:09.059739  1122 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.060102  1333 raft_consensus.cc:993] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:09.060557  1282 raft_consensus.cc:1081] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:09.062196  1048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61e88870331a4a07baee7c415fab351a"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34256
I20260709 13:00:09.062743  1048 raft_consensus.cc:493] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:09.063131  1048 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.063851  1198 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:09.064971  1015 leader_election.cc:304] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:09.065445  1122 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:09.065760  1336 raft_consensus.cc:2804] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:09.066310  1336 raft_consensus.cc:697] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:09.067152  1336 consensus_queue.cc:237] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:09.069710  1048 raft_consensus.cc:515] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:09.072247  1048 leader_election.cc:290] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:09.072757  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:09.073378  1122 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.073925  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e88870331a4a07baee7c415fab351a" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:09.074472  1198 raft_consensus.cc:3055] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:09.074796  1198 raft_consensus.cc:740] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:09.075551  1198 consensus_queue.cc:260] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:09.075431   947 catalog_manager.cc:5697] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 1 to 2, leader changed from dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.076766  1198 raft_consensus.cc:3060] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.078455  1122 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:09.079604  1012 leader_election.cc:304] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:09.080396  1336 raft_consensus.cc:2804] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:09.080873  1336 raft_consensus.cc:697] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:09.081681  1336 consensus_queue.cc:237] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } }
I20260709 13:00:09.082828  1198 raft_consensus.cc:2468] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:09.089464   947 catalog_manager.cc:5697] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 1 to 2, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.149335  1198 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.150725  1336 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.158809  1122 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.160673  1323 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.354141  1198 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.355232  1323 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.358466  1048 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:09.359730  1342 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.363654  1122 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.365105  1263 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.373742  1198 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:09.375226  1342 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.437309  1198 raft_consensus.cc:1275] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.438669  1342 consensus_queue.cc:1048] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.447140  1048 raft_consensus.cc:1275] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.449082  1342 consensus_queue.cc:1048] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.461829  1122 raft_consensus.cc:1275] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.462962  1336 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.473781  1198 raft_consensus.cc:1275] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.474421  1196 raft_consensus.cc:1275] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.475306  1263 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.475939  1342 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.487246  1048 raft_consensus.cc:1275] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.488564  1354 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.553068  1198 raft_consensus.cc:1275] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.554222  1263 consensus_queue.cc:1048] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.562444  1122 raft_consensus.cc:1275] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.563989  1337 consensus_queue.cc:1048] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:11.607717   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:00:11.608446   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:11.629091  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:11.629527  1048 raft_consensus.cc:606] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Received request to transfer leadership to TS be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:11.631676  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:11.632113  1048 raft_consensus.cc:606] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Received request to transfer leadership to TS be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:11.632930   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:11.633325   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:11.899251  1336 raft_consensus.cc:993] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:11.899693  1364 raft_consensus.cc:1081] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Signalling peer be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:11.900959  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:33570
I20260709 13:00:11.901413  1198 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:11.901741  1198 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.906229  1198 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:11.907721  1198 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:11.908775  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:11.909140  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:11.909241  1122 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.909603  1048 raft_consensus.cc:3055] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.909857  1048 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:11.910406  1048 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:11.911201  1048 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.913552  1122 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:11.914552  1161 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:11.915288  1282 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.915535  1048 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:11.916330  1282 raft_consensus.cc:697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:11.917137  1282 consensus_queue.cc:237] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:11.923882   944 catalog_manager.cc:5697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 2 to 3, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:12.086974  1336 raft_consensus.cc:993] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:12.087437  1364 raft_consensus.cc:1081] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Signalling peer be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:12.088920  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:33570
I20260709 13:00:12.089344  1198 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.089571  1198 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.093561  1198 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.094856  1198 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:12.095743  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:12.096024  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:12.096297  1048 raft_consensus.cc:3055] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.096544  1122 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.096652  1048 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:12.097366  1048 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.098284  1048 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.100759  1122 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:12.101691  1161 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:12.102381  1282 raft_consensus.cc:2804] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.102344  1048 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:12.102986  1282 raft_consensus.cc:697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:12.103869  1282 consensus_queue.cc:237] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.111893   944 catalog_manager.cc:5697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 2 to 3, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:12.377435  1049 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.379382  1365 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:12.388617  1122 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.391043  1365 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:12.556870  1049 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.558207  1365 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.567571  1122 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.569288  1282 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.634290   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:12.652354  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:12.653023  1198 raft_consensus.cc:606] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:12.657727  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:33584
I20260709 13:00:12.658181  1198 raft_consensus.cc:606] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:12.662369  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:39550
I20260709 13:00:12.662828  1122 raft_consensus.cc:606] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:12.668041  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:39550
I20260709 13:00:12.668615  1122 raft_consensus.cc:606] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:12.670650  1372 raft_consensus.cc:993] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.671228  1354 raft_consensus.cc:1081] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.672763  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34248
I20260709 13:00:12.673339  1049 raft_consensus.cc:493] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.673772  1049 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.674144  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
mode: GRACEFUL
new_leader_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:39550
I20260709 13:00:12.674708  1122 raft_consensus.cc:606] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Received request to transfer leadership to TS 13184db88a7442c5aafcdda1dc3763c7
I20260709 13:00:12.680735  1049 raft_consensus.cc:515] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.682403  1049 leader_election.cc:290] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:12.683283  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:12.683532  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:12.684010  1122 raft_consensus.cc:3055] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.684090  1198 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.684295  1122 raft_consensus.cc:740] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:12.685138  1122 consensus_queue.cc:260] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.686002  1122 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.688699  1198 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:12.689831  1015 leader_election.cc:304] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:12.690903  1364 raft_consensus.cc:2804] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.692040  1364 raft_consensus.cc:697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:12.692339  1122 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 2.
I20260709 13:00:12.693132  1364 consensus_queue.cc:237] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.699717   944 catalog_manager.cc:5697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 1 to 2, leader changed from dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:12.753647  1372 raft_consensus.cc:993] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.754175  1354 raft_consensus.cc:1081] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.755523  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34248
I20260709 13:00:12.755947  1049 raft_consensus.cc:493] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.756196  1049 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.760722  1049 raft_consensus.cc:515] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.762106  1049 leader_election.cc:290] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 3 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:12.762914  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:12.763264  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29c5f249ae664bdf97cee148696c59c1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:12.763551  1122 raft_consensus.cc:3055] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.763852  1198 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.763943  1122 raft_consensus.cc:740] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:12.764807  1122 consensus_queue.cc:260] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.766119  1122 raft_consensus.cc:3060] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.779507  1198 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 3.
I20260709 13:00:12.779507  1122 raft_consensus.cc:2468] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 3.
I20260709 13:00:12.781013  1012 leader_election.cc:304] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:12.781800  1364 raft_consensus.cc:2804] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.782240  1364 raft_consensus.cc:697] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:12.782979  1364 consensus_queue.cc:237] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.791230   944 catalog_manager.cc:5697] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 2 to 3, leader changed from dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:12.891274  1372 raft_consensus.cc:993] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.891711  1354 raft_consensus.cc:1081] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.893162  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34248
I20260709 13:00:12.893579  1049 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.893817  1049 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.900241  1380 raft_consensus.cc:993] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.900684  1282 raft_consensus.cc:1081] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:12.902673  1048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34256
I20260709 13:00:12.903172  1048 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.903513  1048 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.920460  1049 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.922590  1049 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:12.923382  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:12.923698  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:12.923974  1122 raft_consensus.cc:3055] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.924280  1198 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.924290  1122 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:12.925228  1122 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.926373  1122 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.944227  1122 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:12.944278  1048 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.944941  1198 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:12.946024  1012 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:12.946553  1048 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:12.946801  1364 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.947324  1364 raft_consensus.cc:697] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:12.947981  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:12.947955  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:12.948678  1198 raft_consensus.cc:3055] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.948745  1122 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.949079  1198 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:12.948850  1364 consensus_queue.cc:237] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.949870  1198 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:12.950831  1198 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.956826   944 catalog_manager.cc:5697] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 3 to 4, leader changed from dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.009238  1380 raft_consensus.cc:993] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: : Instructing follower 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:13.009786  1282 raft_consensus.cc:1081] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Signalling peer 13184db88a7442c5aafcdda1dc3763c7 to start an election
I20260709 13:00:13.011209  1048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
 from {username='slave'} at 127.0.0.1:34256
I20260709 13:00:13.011687  1048 raft_consensus.cc:493] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:13.012028  1048 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.039914  1122 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:13.039914  1198 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:13.039955  1048 raft_consensus.cc:515] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:13.041519  1012 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:13.042150  1048 leader_election.cc:290] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Requested vote from peers dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:13.043341  1364 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.044008  1364 raft_consensus.cc:697] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:13.044086  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:13.043813  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fb55609fd47a79e6d31f937d9f8a1" candidate_uuid: "13184db88a7442c5aafcdda1dc3763c7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:13.044902  1198 raft_consensus.cc:3055] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.044937  1122 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.045320  1198 raft_consensus.cc:740] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:13.046082  1198 consensus_queue.cc:260] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:13.044967  1364 consensus_queue.cc:237] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:13.047008  1198 raft_consensus.cc:3060] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.053284   944 catalog_manager.cc:5697] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 3 to 4, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.143276  1198 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:13.143527  1122 raft_consensus.cc:2468] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13184db88a7442c5aafcdda1dc3763c7 in term 4.
I20260709 13:00:13.144675  1015 leader_election.cc:304] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, be113e9528624ac9ae0cc95fa32fde88; no voters: 
I20260709 13:00:13.145516  1364 raft_consensus.cc:2804] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.146080  1364 raft_consensus.cc:697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Becoming Leader. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:13.147010  1364 consensus_queue.cc:237] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:13.153815   944 catalog_manager.cc:5697] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 reported cstate change: term changed from 3 to 4, leader changed from be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195) to 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "13184db88a7442c5aafcdda1dc3763c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.196336  1122 raft_consensus.cc:1275] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.197548  1364 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:13.203249  1124 raft_consensus.cc:1275] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.205390  1364 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.209794  1198 raft_consensus.cc:1275] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.211932  1382 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:13.222795  1196 raft_consensus.cc:1275] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.224815  1391 consensus_queue.cc:1048] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:13.398155  1196 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:13.399281  1391 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.407831  1122 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:13.410043  1391 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.486136  1196 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.487345  1391 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:13.495296  1122 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.496706  1364 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:13.594787  1196 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.596052  1364 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:13.605618  1122 raft_consensus.cc:1275] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Refusing update from remote peer 13184db88a7442c5aafcdda1dc3763c7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.606853  1392 consensus_queue.cc:1048] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:15.681825   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:15.689736  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:15.690230  1048 raft_consensus.cc:606] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Received request to transfer leadership to TS be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:15.692526  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:15.692934  1048 raft_consensus.cc:606] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Received request to transfer leadership to TS dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:15.694940  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:15.695363  1048 raft_consensus.cc:606] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Received request to transfer leadership to TS be113e9528624ac9ae0cc95fa32fde88
I20260709 13:00:15.697468  1048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
mode: GRACEFUL
new_leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:53002
I20260709 13:00:15.697849  1048 raft_consensus.cc:606] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Received request to transfer leadership to TS dd7150ccd47d47809dc827e8d4cff65d
I20260709 13:00:15.698715   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:15.699189   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:15.754884  1391 raft_consensus.cc:993] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:15.755352  1405 raft_consensus.cc:1081] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Signalling peer dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:15.756775  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:00:15.757220  1122 raft_consensus.cc:493] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.757499  1122 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.762049  1122 raft_consensus.cc:515] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.763921  1122 leader_election.cc:290] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 5 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:15.764447  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:15.764940  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc370965a6dd433fb17e4b1ce8ad7e26" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:15.764973  1048 raft_consensus.cc:3055] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:15.765396  1048 raft_consensus.cc:740] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:15.765470  1196 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.766048  1048 consensus_queue.cc:260] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.766839  1048 raft_consensus.cc:3060] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.770895  1048 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 5.
I20260709 13:00:15.771456  1196 raft_consensus.cc:2468] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 5.
I20260709 13:00:15.772019  1087 leader_election.cc:304] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13184db88a7442c5aafcdda1dc3763c7, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:15.772706  1354 raft_consensus.cc:2804] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:15.773509  1354 raft_consensus.cc:697] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 5 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:15.774245  1354 consensus_queue.cc:237] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.780571   946 catalog_manager.cc:5697] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 4 to 5, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 5 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.835903  1391 raft_consensus.cc:993] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:15.836349  1405 raft_consensus.cc:1081] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Signalling peer dd7150ccd47d47809dc827e8d4cff65d to start an election
I20260709 13:00:15.837769  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b"
dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:00:15.838236  1122 raft_consensus.cc:493] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.838522  1122 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.842759  1122 raft_consensus.cc:515] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.844168  1122 leader_election.cc:290] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 3 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195:35701)
I20260709 13:00:15.845010  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:15.845099  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2f6c729dbd4ed8a42a9013b24dbf2b" candidate_uuid: "dd7150ccd47d47809dc827e8d4cff65d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
I20260709 13:00:15.845628  1048 raft_consensus.cc:3055] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:15.845672  1196 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.846112  1048 raft_consensus.cc:740] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:15.846843  1048 consensus_queue.cc:260] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.847991  1048 raft_consensus.cc:3060] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.852577  1196 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 3.
I20260709 13:00:15.853820  1090 leader_election.cc:304] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:15.854349  1048 raft_consensus.cc:2468] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7150ccd47d47809dc827e8d4cff65d in term 3.
I20260709 13:00:15.854641  1354 raft_consensus.cc:2804] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.855206  1354 raft_consensus.cc:697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Becoming Leader. State: Replica: dd7150ccd47d47809dc827e8d4cff65d, State: Running, Role: LEADER
I20260709 13:00:15.855989  1354 consensus_queue.cc:237] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.861910   946 catalog_manager.cc:5697] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d reported cstate change: term changed from 2 to 3, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "dd7150ccd47d47809dc827e8d4cff65d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.956580  1391 raft_consensus.cc:993] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:15.956990  1405 raft_consensus.cc:1081] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Signalling peer be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:15.958305  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:33570
I20260709 13:00:15.958782  1196 raft_consensus.cc:493] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.959069  1196 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.963451  1196 raft_consensus.cc:515] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.965108  1196 leader_election.cc:290] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 5 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:15.965647  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:15.966171  1048 raft_consensus.cc:3055] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:15.966184  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643d374a27244afa95eef29c9ab45f34" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:15.966478  1048 raft_consensus.cc:740] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:15.966764  1122 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.967141  1048 consensus_queue.cc:260] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.967975  1048 raft_consensus.cc:3060] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.970917  1122 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 5.
I20260709 13:00:15.971974  1048 raft_consensus.cc:2468] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 5.
I20260709 13:00:15.972015  1161 leader_election.cc:304] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:15.972796  1282 raft_consensus.cc:2804] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:15.973809  1282 raft_consensus.cc:697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 5 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:15.974551  1282 consensus_queue.cc:237] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:15.981038   946 catalog_manager.cc:5697] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 4 to 5, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:16.072301  1391 raft_consensus.cc:993] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: : Instructing follower be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:16.072808  1405 raft_consensus.cc:1081] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Signalling peer be113e9528624ac9ae0cc95fa32fde88 to start an election
I20260709 13:00:16.074306  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822"
dest_uuid: "be113e9528624ac9ae0cc95fa32fde88"
 from {username='slave'} at 127.0.0.1:33570
I20260709 13:00:16.074817  1196 raft_consensus.cc:493] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:16.075194  1196 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.079732  1196 raft_consensus.cc:515] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:16.081324  1196 leader_election.cc:290] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Requested vote from peers 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785), dd7150ccd47d47809dc827e8d4cff65d (127.0.107.194:43027)
I20260709 13:00:16.081897  1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13184db88a7442c5aafcdda1dc3763c7"
I20260709 13:00:16.082298  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a89d6b110f14e3e9de25256390a8822" candidate_uuid: "be113e9528624ac9ae0cc95fa32fde88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7150ccd47d47809dc827e8d4cff65d"
I20260709 13:00:16.082525  1048 raft_consensus.cc:3055] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.082806  1048 raft_consensus.cc:740] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13184db88a7442c5aafcdda1dc3763c7, State: Running, Role: LEADER
I20260709 13:00:16.082844  1122 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.083580  1048 consensus_queue.cc:260] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:16.084484  1048 raft_consensus.cc:3060] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.086946  1122 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:16.087910  1161 leader_election.cc:304] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be113e9528624ac9ae0cc95fa32fde88, dd7150ccd47d47809dc827e8d4cff65d; no voters: 
I20260709 13:00:16.088645  1282 raft_consensus.cc:2804] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:16.088610  1048 raft_consensus.cc:2468] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be113e9528624ac9ae0cc95fa32fde88 in term 3.
I20260709 13:00:16.089202  1282 raft_consensus.cc:697] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Becoming Leader. State: Replica: be113e9528624ac9ae0cc95fa32fde88, State: Running, Role: LEADER
I20260709 13:00:16.090389  1282 consensus_queue.cc:237] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } }
I20260709 13:00:16.096244   946 catalog_manager.cc:5697] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 reported cstate change: term changed from 2 to 3, leader changed from 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193) to be113e9528624ac9ae0cc95fa32fde88 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "be113e9528624ac9ae0cc95fa32fde88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:16.230311  1048 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:16.231446  1354 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:16.240749  1196 raft_consensus.cc:1275] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:16.244244  1198 raft_consensus.cc:1275] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.247889  1406 consensus_queue.cc:1048] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:16.249642  1406 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be113e9528624ac9ae0cc95fa32fde88" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:16.263553  1048 raft_consensus.cc:1275] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Refusing update from remote peer dd7150ccd47d47809dc827e8d4cff65d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.265264  1354 consensus_queue.cc:1048] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:16.375394  1122 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.376596  1413 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:16.388363  1048 raft_consensus.cc:1275] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.389940  1413 consensus_queue.cc:1048] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:16.557967  1122 raft_consensus.cc:1275] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.559127  1413 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7150ccd47d47809dc827e8d4cff65d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:16.569343  1048 raft_consensus.cc:1275] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Refusing update from remote peer be113e9528624ac9ae0cc95fa32fde88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.570886  1413 consensus_queue.cc:1048] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13184db88a7442c5aafcdda1dc3763c7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:16.700136   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:16.706123   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:00:16.726766   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:16.727511   431 tablet_replica.cc:333] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.728211   431 raft_consensus.cc:2243] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.728605   431 raft_consensus.cc:2272] T 643d374a27244afa95eef29c9ab45f34 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.731267   431 tablet_replica.cc:333] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.731731   431 raft_consensus.cc:2243] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.732161   431 raft_consensus.cc:2272] T 2a89d6b110f14e3e9de25256390a8822 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.734162   431 tablet_replica.cc:333] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.734656   431 raft_consensus.cc:2243] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.735013   431 raft_consensus.cc:2272] T d712a4087df946b7a58e04edcf4c398d P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.736624   431 tablet_replica.cc:333] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.737054   431 raft_consensus.cc:2243] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.737794   431 raft_consensus.cc:2272] T 29c5f249ae664bdf97cee148696c59c1 P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.739295   431 tablet_replica.cc:333] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.739737   431 raft_consensus.cc:2243] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.740166   431 raft_consensus.cc:2272] T fc370965a6dd433fb17e4b1ce8ad7e26 P 13184db88a7442c5aafcdda1dc3763c7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.742072   431 tablet_replica.cc:333] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.742507   431 raft_consensus.cc:2243] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.742847   431 raft_consensus.cc:2272] T 22b64d12ed044e0d9b9ea2a461f476dd P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.744365   431 tablet_replica.cc:333] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.744783   431 raft_consensus.cc:2243] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.745383   431 raft_consensus.cc:2272] T 0c2fb55609fd47a79e6d31f937d9f8a1 P 13184db88a7442c5aafcdda1dc3763c7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.746850   431 tablet_replica.cc:333] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.747247   431 raft_consensus.cc:2243] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.747991   431 raft_consensus.cc:2272] T 61e88870331a4a07baee7c415fab351a P 13184db88a7442c5aafcdda1dc3763c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.749548   431 tablet_replica.cc:333] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.749974   431 raft_consensus.cc:2243] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.750285   431 raft_consensus.cc:2272] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P 13184db88a7442c5aafcdda1dc3763c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.752238   431 tablet_replica.cc:333] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7: stopping tablet replica
I20260709 13:00:16.752746   431 raft_consensus.cc:2243] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.753399   431 raft_consensus.cc:2272] T 7f05567e9791422d90ae15c5b7f0cf3e P 13184db88a7442c5aafcdda1dc3763c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.781970   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:00:16.801784   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:00:16.825105   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:16.825778   431 tablet_replica.cc:333] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.826436   431 raft_consensus.cc:2243] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.826974   431 raft_consensus.cc:2272] T 643d374a27244afa95eef29c9ab45f34 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.829420   431 tablet_replica.cc:333] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.830029   431 raft_consensus.cc:2243] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.830813   431 raft_consensus.cc:2272] T fc370965a6dd433fb17e4b1ce8ad7e26 P dd7150ccd47d47809dc827e8d4cff65d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.832942   431 tablet_replica.cc:333] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.833504   431 raft_consensus.cc:2243] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.833992   431 raft_consensus.cc:2272] T 0c2fb55609fd47a79e6d31f937d9f8a1 P dd7150ccd47d47809dc827e8d4cff65d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.835713   431 tablet_replica.cc:333] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.836306   431 raft_consensus.cc:2243] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.836741   431 raft_consensus.cc:2272] T 22b64d12ed044e0d9b9ea2a461f476dd P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.838491   431 tablet_replica.cc:333] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.839036   431 raft_consensus.cc:2243] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.839468   431 raft_consensus.cc:2272] T 7f05567e9791422d90ae15c5b7f0cf3e P dd7150ccd47d47809dc827e8d4cff65d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.841449   431 tablet_replica.cc:333] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.842008   431 raft_consensus.cc:2243] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.842805   431 raft_consensus.cc:2272] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.844897   431 tablet_replica.cc:333] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.845448   431 raft_consensus.cc:2243] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.845889   431 raft_consensus.cc:2272] T 61e88870331a4a07baee7c415fab351a P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.847643   431 tablet_replica.cc:333] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.848162   431 raft_consensus.cc:2243] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.848593   431 raft_consensus.cc:2272] T 29c5f249ae664bdf97cee148696c59c1 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.850226   431 tablet_replica.cc:333] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.850751   431 raft_consensus.cc:2243] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.851492   431 raft_consensus.cc:2272] T d712a4087df946b7a58e04edcf4c398d P dd7150ccd47d47809dc827e8d4cff65d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.853395   431 tablet_replica.cc:333] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d: stopping tablet replica
I20260709 13:00:16.853866   431 raft_consensus.cc:2243] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.854301   431 raft_consensus.cc:2272] T 2a89d6b110f14e3e9de25256390a8822 P dd7150ccd47d47809dc827e8d4cff65d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:16.865940  1161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35785: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260709 13:00:16.870517  1161 consensus_peers.cc:597] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 -> Peer 13184db88a7442c5aafcdda1dc3763c7 (127.0.107.193:35785): Couldn't send request to peer 13184db88a7442c5aafcdda1dc3763c7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:16.876888   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:00:16.894346   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:00:16.923235   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:16.923945   431 tablet_replica.cc:333] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.924511   431 raft_consensus.cc:2243] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.925318   431 raft_consensus.cc:2272] T 22b64d12ed044e0d9b9ea2a461f476dd P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.927049   431 tablet_replica.cc:333] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.927565   431 raft_consensus.cc:2243] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.927986   431 raft_consensus.cc:2272] T fc370965a6dd433fb17e4b1ce8ad7e26 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.929847   431 tablet_replica.cc:333] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.930337   431 raft_consensus.cc:2243] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.930733   431 raft_consensus.cc:2272] T 0c2fb55609fd47a79e6d31f937d9f8a1 P be113e9528624ac9ae0cc95fa32fde88 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.932377   431 tablet_replica.cc:333] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.932837   431 raft_consensus.cc:2243] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.933208   431 raft_consensus.cc:2272] T d712a4087df946b7a58e04edcf4c398d P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.934762   431 tablet_replica.cc:333] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.935232   431 raft_consensus.cc:2243] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.935631   431 raft_consensus.cc:2272] T 29c5f249ae664bdf97cee148696c59c1 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.937256   431 tablet_replica.cc:333] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.937732   431 raft_consensus.cc:2243] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.938416   431 raft_consensus.cc:2272] T 643d374a27244afa95eef29c9ab45f34 P be113e9528624ac9ae0cc95fa32fde88 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.940302   431 tablet_replica.cc:333] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.940788   431 raft_consensus.cc:2243] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.941191   431 raft_consensus.cc:2272] T 61e88870331a4a07baee7c415fab351a P be113e9528624ac9ae0cc95fa32fde88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.942695   431 tablet_replica.cc:333] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.943178   431 raft_consensus.cc:2243] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.943563   431 raft_consensus.cc:2272] T 5a2f6c729dbd4ed8a42a9013b24dbf2b P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.945509   431 tablet_replica.cc:333] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.945940   431 raft_consensus.cc:2243] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.946288   431 raft_consensus.cc:2272] T 7f05567e9791422d90ae15c5b7f0cf3e P be113e9528624ac9ae0cc95fa32fde88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.947748   431 tablet_replica.cc:333] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88: stopping tablet replica
I20260709 13:00:16.948210   431 raft_consensus.cc:2243] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.948876   431 raft_consensus.cc:2272] T 2a89d6b110f14e3e9de25256390a8822 P be113e9528624ac9ae0cc95fa32fde88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.975749   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:16.993683   431 master.cc:562] Master@127.0.107.254:39037 shutting down...
I20260709 13:00:17.013348   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.013842   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.014122   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3fcdc85c6d6349ec95c02f3fdf3e8cee: stopping tablet replica
I20260709 13:00:17.032585   431 master.cc:584] Master@127.0.107.254:39037 shutdown complete.
13184db88a7442c5aafcdda1dc3763c7  5
be113e9528624ac9ae0cc95fa32fde88  1
dd7150ccd47d47809dc827e8d4cff65d  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (11894 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:00:17.072211   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:37587
I20260709 13:00:17.073280   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.078616  1433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.079980  1434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.080453  1436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.082082   431 server_base.cc:1061] running on GCE node
I20260709 13:00:17.082895   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.083151   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.083313   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017083280 us; error 0 us; skew 500 ppm
I20260709 13:00:17.083922   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.086356   431 webserver.cc:533] Webserver started at http://127.0.107.254:39671/ using document root <none> and password file <none>
I20260709 13:00:17.086889   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.087077   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.087364   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.088541   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "8c0fa710867745f189aafd0e7d84fa64"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.092778   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:00:17.095958  1441 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.096771   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:00:17.097007   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "8c0fa710867745f189aafd0e7d84fa64"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.097267   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.111470   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.112756   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.150000   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:37587
I20260709 13:00:17.150116  1492 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:37587 every 8 connection(s)
I20260709 13:00:17.154277  1493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.164412  1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Bootstrap starting.
I20260709 13:00:17.169306  1493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:17.173837  1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: No bootstrap required, opened a new log
I20260709 13:00:17.176187  1493 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER }
I20260709 13:00:17.176616  1493 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:17.176811  1493 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0fa710867745f189aafd0e7d84fa64, State: Initialized, Role: FOLLOWER
I20260709 13:00:17.177299  1493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER }
I20260709 13:00:17.177716  1493 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:17.177908  1493 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:17.178140  1493 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:17.182418  1493 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER }
I20260709 13:00:17.182901  1493 leader_election.cc:304] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c0fa710867745f189aafd0e7d84fa64; no voters: 
I20260709 13:00:17.183995  1493 leader_election.cc:290] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:17.184275  1496 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:17.185647  1496 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 1 LEADER]: Becoming Leader. State: Replica: 8c0fa710867745f189aafd0e7d84fa64, State: Running, Role: LEADER
I20260709 13:00:17.186293  1496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER }
I20260709 13:00:17.187065  1493 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:17.190682  1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c0fa710867745f189aafd0e7d84fa64. Latest consensus state: current_term: 1 leader_uuid: "8c0fa710867745f189aafd0e7d84fa64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER } }
I20260709 13:00:17.191382  1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:17.191756  1497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c0fa710867745f189aafd0e7d84fa64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0fa710867745f189aafd0e7d84fa64" member_type: VOTER } }
I20260709 13:00:17.192498  1497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:17.195179  1503 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:17.199975  1503 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:17.203833   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:17.208813  1503 catalog_manager.cc:1383] Generated new cluster ID: 8cf68285441248d3ac58c74c4a1cd585
I20260709 13:00:17.209124  1503 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:17.222155  1503 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:17.223589  1503 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:17.243806  1503 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Generated new TSK 0
I20260709 13:00:17.244561  1503 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:17.270188   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.275995  1514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.277092  1515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.278693   431 server_base.cc:1061] running on GCE node
W20260709 13:00:17.278771  1517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.279718   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.279994   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.280124   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017280112 us; error 0 us; skew 500 ppm
I20260709 13:00:17.280570   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.282740   431 webserver.cc:533] Webserver started at http://127.0.107.193:37957/ using document root <none> and password file <none>
I20260709 13:00:17.283154   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.283306   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.283512   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.284595   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "444b494bb53842b59408a7230884398a"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.289472   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:00:17.292622  1522 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.293349   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:00:17.293639   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "444b494bb53842b59408a7230884398a"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.293897   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.311657   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.313002   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.314630   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:17.317013   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:17.317194   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.317504   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:17.317667   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.356822   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38031
I20260709 13:00:17.356920  1584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38031 every 8 connection(s)
I20260709 13:00:17.361781   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.368438  1589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.370330  1590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.372967  1592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.373179   431 server_base.cc:1061] running on GCE node
I20260709 13:00:17.374364   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.374599   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.374776   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017374751 us; error 0 us; skew 500 ppm
I20260709 13:00:17.375283   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.377878   431 webserver.cc:533] Webserver started at http://127.0.107.194:34929/ using document root <none> and password file <none>
I20260709 13:00:17.378358   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.378571   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.378898   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.379948  1585 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37587
I20260709 13:00:17.380055   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "8b2c3934407d422495728f84374c6b53"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.380326  1585 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:17.381048  1585 heartbeater.cc:507] Master 127.0.107.254:37587 requested a full tablet report, sending...
I20260709 13:00:17.383042  1458 ts_manager.cc:194] Registered new tserver with Master: 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:17.384786   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:00:17.384848  1458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45518
I20260709 13:00:17.388351  1597 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.389176   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:00:17.389461   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "8b2c3934407d422495728f84374c6b53"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.389770   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.401134   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.402302   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.403751   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:17.406006   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:17.406239   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.406492   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:17.406668   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.445312   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:39515
I20260709 13:00:17.445421  1659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:39515 every 8 connection(s)
I20260709 13:00:17.450114   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.457377  1663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.458976  1664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.462710  1666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.464164  1660 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37587
I20260709 13:00:17.464599  1660 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:17.464591   431 server_base.cc:1061] running on GCE node
I20260709 13:00:17.465719  1660 heartbeater.cc:507] Master 127.0.107.254:37587 requested a full tablet report, sending...
I20260709 13:00:17.465806   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.466135   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.466346   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017466327 us; error 0 us; skew 500 ppm
I20260709 13:00:17.466986   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.468138  1458 ts_manager.cc:194] Registered new tserver with Master: 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:17.469735   431 webserver.cc:533] Webserver started at http://127.0.107.195:36399/ using document root <none> and password file <none>
I20260709 13:00:17.469983  1458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45532
I20260709 13:00:17.470364   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.470625   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.470937   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.472483   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "00f3fbc60b884078a3c219909b44f64c"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.476717   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:00:17.479871  1671 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.480602   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:00:17.480890   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "00f3fbc60b884078a3c219909b44f64c"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-kgkf"
I20260709 13:00:17.481218   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.500870   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.502084   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.503526   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:17.505733   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:17.505959   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.506204   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:17.506377   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.544795   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:45021
I20260709 13:00:17.544898  1733 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:45021 every 8 connection(s)
I20260709 13:00:17.558463  1734 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37587
I20260709 13:00:17.558833  1734 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:17.559469  1734 heartbeater.cc:507] Master 127.0.107.254:37587 requested a full tablet report, sending...
I20260709 13:00:17.561542  1458 ts_manager.cc:194] Registered new tserver with Master: 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:17.562497   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014717631s
I20260709 13:00:17.563076  1458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45542
I20260709 13:00:17.587857  1458 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45548:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:17.593760  1458 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:17.794376  1550 tablet_service.cc:1511] Processing CreateTablet for tablet c8a9cb8cee6d447e9042f38fe30cb6e0 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:17.795892  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a9cb8cee6d447e9042f38fe30cb6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.796927  1548 tablet_service.cc:1511] Processing CreateTablet for tablet ab336cb98e24470eb828d9e9fdbcb001 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:17.798254  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab336cb98e24470eb828d9e9fdbcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.799607  1547 tablet_service.cc:1511] Processing CreateTablet for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:17.795375  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 9438873b8e1a4f0780fff8b93a9d89b8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:17.801091  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.803357  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 9438873b8e1a4f0780fff8b93a9d89b8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:17.803524  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 2139fc31dcac45ddbcc3f5046bd79ade (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:17.804744  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9438873b8e1a4f0780fff8b93a9d89b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.804766  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2139fc31dcac45ddbcc3f5046bd79ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.808073  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 41b83284fac147339ac4124c2eedee11 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:17.808915  1697 tablet_service.cc:1511] Processing CreateTablet for tablet ab336cb98e24470eb828d9e9fdbcb001 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:17.809451  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b83284fac147339ac4124c2eedee11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.810240  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab336cb98e24470eb828d9e9fdbcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.812397  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 4481ad55e16e48b7a3442111d9636f55 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:17.812620  1699 tablet_service.cc:1511] Processing CreateTablet for tablet c8a9cb8cee6d447e9042f38fe30cb6e0 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:17.813746  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a9cb8cee6d447e9042f38fe30cb6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.813750  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4481ad55e16e48b7a3442111d9636f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.814918  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9438873b8e1a4f0780fff8b93a9d89b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.816279  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:17.817830  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.822154  1695 tablet_service.cc:1511] Processing CreateTablet for tablet 2139fc31dcac45ddbcc3f5046bd79ade (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:17.823979  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2139fc31dcac45ddbcc3f5046bd79ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.829437  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 41b83284fac147339ac4124c2eedee11 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:17.831387  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b83284fac147339ac4124c2eedee11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.833366  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 4481ad55e16e48b7a3442111d9636f55 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:17.840210  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4481ad55e16e48b7a3442111d9636f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.867935  1623 tablet_service.cc:1511] Processing CreateTablet for tablet ab336cb98e24470eb828d9e9fdbcb001 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:17.868836  1625 tablet_service.cc:1511] Processing CreateTablet for tablet c8a9cb8cee6d447e9042f38fe30cb6e0 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:17.869413  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab336cb98e24470eb828d9e9fdbcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.870139  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a9cb8cee6d447e9042f38fe30cb6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.872926  1624 tablet_service.cc:1511] Processing CreateTablet for tablet 9438873b8e1a4f0780fff8b93a9d89b8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:17.874224  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9438873b8e1a4f0780fff8b93a9d89b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.874850  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 41b83284fac147339ac4124c2eedee11 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:17.876187  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b83284fac147339ac4124c2eedee11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.876950  1543 tablet_service.cc:1511] Processing CreateTablet for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:17.878242  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.871404  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:17.883560  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.885332  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:17.887094  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.872931  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 2139fc31dcac45ddbcc3f5046bd79ade (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:17.895011  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2139fc31dcac45ddbcc3f5046bd79ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.894845  1619 tablet_service.cc:1511] Processing CreateTablet for tablet 4481ad55e16e48b7a3442111d9636f55 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:17.897008  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4481ad55e16e48b7a3442111d9636f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.900334  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:17.901641  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.883188  1542 tablet_service.cc:1511] Processing CreateTablet for tablet e629c77df1d84117a78341cd5a55b213 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:17.908630  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e629c77df1d84117a78341cd5a55b213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.945143  1622 tablet_service.cc:1511] Processing CreateTablet for tablet e629c77df1d84117a78341cd5a55b213 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:17.947057  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e629c77df1d84117a78341cd5a55b213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.959971  1750 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:17.961218  1691 tablet_service.cc:1511] Processing CreateTablet for tablet e629c77df1d84117a78341cd5a55b213 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:17.964594  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e629c77df1d84117a78341cd5a55b213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.964825  1749 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:17.976886  1749 tablet_bootstrap.cc:654] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:17.999622  1694 tablet_service.cc:1511] Processing CreateTablet for tablet a986b2c96ad0416f8fc5927d45ebadc4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:18.000989  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a986b2c96ad0416f8fc5927d45ebadc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.001101  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 2292aebe10c64bc99c27e3976075c309 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:18.008867  1751 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:18.012742  1621 tablet_service.cc:1511] Processing CreateTablet for tablet a986b2c96ad0416f8fc5927d45ebadc4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:18.014014  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a986b2c96ad0416f8fc5927d45ebadc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.014402  1750 tablet_bootstrap.cc:654] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.001046  1549 tablet_service.cc:1511] Processing CreateTablet for tablet a986b2c96ad0416f8fc5927d45ebadc4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:18.004503  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 2292aebe10c64bc99c27e3976075c309 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:18.021057  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2292aebe10c64bc99c27e3976075c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.025362  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2292aebe10c64bc99c27e3976075c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.025631  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 2292aebe10c64bc99c27e3976075c309 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:18.026934  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2292aebe10c64bc99c27e3976075c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.030331  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a986b2c96ad0416f8fc5927d45ebadc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.037671  1542 tablet_service.cc:1511] Processing CreateTablet for tablet d6beee9f16eb4c7db2d293d74dfec496 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:18.039100  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6beee9f16eb4c7db2d293d74dfec496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.038226  1751 tablet_bootstrap.cc:654] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.057350  1624 tablet_service.cc:1511] Processing CreateTablet for tablet d6beee9f16eb4c7db2d293d74dfec496 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:18.058182  1692 tablet_service.cc:1511] Processing CreateTablet for tablet d6beee9f16eb4c7db2d293d74dfec496 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:18.058842  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6beee9f16eb4c7db2d293d74dfec496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.059427  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6beee9f16eb4c7db2d293d74dfec496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.063525  1749 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.065645  1543 tablet_service.cc:1511] Processing CreateTablet for tablet 288b67d6ad124d2aa18d641ed4e1f3df (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:18.067313  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 288b67d6ad124d2aa18d641ed4e1f3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.064126  1749 ts_tablet_manager.cc:1403] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.100s	user 0.024s	sys 0.006s
I20260709 13:00:18.069669  1548 tablet_service.cc:1511] Processing CreateTablet for tablet ec944cb8c5da4037a28a6917759780b4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:18.070901  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec944cb8c5da4037a28a6917759780b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.073271  1617 tablet_service.cc:1511] Processing CreateTablet for tablet ec944cb8c5da4037a28a6917759780b4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:18.074973  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec944cb8c5da4037a28a6917759780b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.079229  1749 raft_consensus.cc:359] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.086156  1749 raft_consensus.cc:385] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.087114  1749 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.084925  1750 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:18.088171  1750 ts_tablet_manager.cc:1403] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.013s
I20260709 13:00:18.088604  1695 tablet_service.cc:1511] Processing CreateTablet for tablet ec944cb8c5da4037a28a6917759780b4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:18.088593  1619 tablet_service.cc:1511] Processing CreateTablet for tablet 288b67d6ad124d2aa18d641ed4e1f3df (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:18.090137  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 288b67d6ad124d2aa18d641ed4e1f3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.089718  1749 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.099116  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec944cb8c5da4037a28a6917759780b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.114485  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 288b67d6ad124d2aa18d641ed4e1f3df (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:18.115403  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 9f032b031edf4e5eb9736c18f8ba0c03 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:18.115906  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 288b67d6ad124d2aa18d641ed4e1f3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.116655  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f032b031edf4e5eb9736c18f8ba0c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.116930  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 9f032b031edf4e5eb9736c18f8ba0c03 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:18.117126  1750 raft_consensus.cc:359] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.117931  1750 raft_consensus.cc:385] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.118264  1750 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.118394  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f032b031edf4e5eb9736c18f8ba0c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.128602  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 9f032b031edf4e5eb9736c18f8ba0c03 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:18.119166  1750 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.129983  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f032b031edf4e5eb9736c18f8ba0c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.138839  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 9d254c50f08d431298fe57ef1dde64be (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:18.140270  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d254c50f08d431298fe57ef1dde64be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.148765  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 9d254c50f08d431298fe57ef1dde64be (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:18.151800  1585 heartbeater.cc:499] Master 127.0.107.254:37587 was elected leader, sending a full tablet report...
I20260709 13:00:18.153355  1751 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:18.153858  1751 ts_tablet_manager.cc:1403] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.145s	user 0.010s	sys 0.002s
I20260709 13:00:18.160856  1751 raft_consensus.cc:359] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.164793  1751 raft_consensus.cc:385] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.157233  1749 ts_tablet_manager.cc:1434] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.088s	user 0.011s	sys 0.020s
I20260709 13:00:18.168542  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d254c50f08d431298fe57ef1dde64be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.169243  1624 tablet_service.cc:1511] Processing CreateTablet for tablet c2dc331078c54a6aa71b5eab7cb93250 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:18.170606  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2dc331078c54a6aa71b5eab7cb93250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.180418  1749 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.184891  1750 ts_tablet_manager.cc:1434] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.002s
I20260709 13:00:18.185704  1660 heartbeater.cc:499] Master 127.0.107.254:37587 was elected leader, sending a full tablet report...
I20260709 13:00:18.185957  1750 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:18.192137  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 9d254c50f08d431298fe57ef1dde64be (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:18.193503  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d254c50f08d431298fe57ef1dde64be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.194885  1749 tablet_bootstrap.cc:654] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.165160  1751 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.196326  1751 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:18.200934  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:18.201828  1695 tablet_service.cc:1511] Processing CreateTablet for tablet c2dc331078c54a6aa71b5eab7cb93250 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:18.203882  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2dc331078c54a6aa71b5eab7cb93250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.203609  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 542a10c27e2d4480a7a262c77167ed4b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:18.204825  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a10c27e2d4480a7a262c77167ed4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.205600  1550 tablet_service.cc:1511] Processing CreateTablet for tablet c2dc331078c54a6aa71b5eab7cb93250 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:18.206331  1624 tablet_service.cc:1511] Processing CreateTablet for tablet 542a10c27e2d4480a7a262c77167ed4b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:18.207041  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2dc331078c54a6aa71b5eab7cb93250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.214247  1734 heartbeater.cc:499] Master 127.0.107.254:37587 was elected leader, sending a full tablet report...
I20260709 13:00:18.216133  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 542a10c27e2d4480a7a262c77167ed4b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:18.217516  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a10c27e2d4480a7a262c77167ed4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.223930  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a10c27e2d4480a7a262c77167ed4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.229031  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 5e0479a2b6a6473eac54a01fde6fc25c (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:18.230475  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0479a2b6a6473eac54a01fde6fc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.240761  1750 tablet_bootstrap.cc:654] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.249356  1749 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.252208  1751 ts_tablet_manager.cc:1434] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.098s	user 0.005s	sys 0.006s
I20260709 13:00:18.252843  1749 ts_tablet_manager.cc:1403] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.021s
I20260709 13:00:18.253568  1751 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:18.255918  1749 raft_consensus.cc:359] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.256699  1749 raft_consensus.cc:385] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.257002  1749 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.257733  1749 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.259898  1749 ts_tablet_manager.cc:1434] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:18.260826  1749 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.265857  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 5e0479a2b6a6473eac54a01fde6fc25c (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:18.269910  1751 tablet_bootstrap.cc:654] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.270138  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0479a2b6a6473eac54a01fde6fc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.274993  1754 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.275571  1754 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.279023  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 5e0479a2b6a6473eac54a01fde6fc25c (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:18.281489  1749 tablet_bootstrap.cc:654] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.284353  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0479a2b6a6473eac54a01fde6fc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.285730  1750 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:18.286293  1750 ts_tablet_manager.cc:1403] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.101s	user 0.009s	sys 0.009s
I20260709 13:00:18.290544  1754 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:18.294446  1624 tablet_service.cc:1511] Processing CreateTablet for tablet a165cfdf51b34a5ea1bb6ece57f06bbc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:18.299309  1695 tablet_service.cc:1511] Processing CreateTablet for tablet a165cfdf51b34a5ea1bb6ece57f06bbc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:18.300895  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165cfdf51b34a5ea1bb6ece57f06bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.311794  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165cfdf51b34a5ea1bb6ece57f06bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.317891  1754 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.317793  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 012e7f60d41140d496901ad4a16e6d4d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:18.318436  1754 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.320458  1754 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:18.320657  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012e7f60d41140d496901ad4a16e6d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.324932  1542 tablet_service.cc:1511] Processing CreateTablet for tablet a165cfdf51b34a5ea1bb6ece57f06bbc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:18.326224  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165cfdf51b34a5ea1bb6ece57f06bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.329649  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 012e7f60d41140d496901ad4a16e6d4d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:18.330902  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012e7f60d41140d496901ad4a16e6d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.331717  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 012e7f60d41140d496901ad4a16e6d4d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:18.332998  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012e7f60d41140d496901ad4a16e6d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.331717  1550 tablet_service.cc:1511] Processing CreateTablet for tablet ce44b855d6d442778079778cd0e68c34 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:18.341141  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce44b855d6d442778079778cd0e68c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.344095  1750 raft_consensus.cc:359] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.344828  1691 tablet_service.cc:1511] Processing CreateTablet for tablet ce44b855d6d442778079778cd0e68c34 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:18.346129  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce44b855d6d442778079778cd0e68c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.347330  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:18.348762  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:18.349025  1749 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.349515  1749 ts_tablet_manager.cc:1403] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.089s	user 0.007s	sys 0.015s
I20260709 13:00:18.349779  1751 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:18.350226  1751 ts_tablet_manager.cc:1403] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.003s
W20260709 13:00:18.350787  1525 leader_election.cc:343] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:18.351521  1525 leader_election.cc:343] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.352465  1749 raft_consensus.cc:359] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.353173  1749 raft_consensus.cc:385] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.353462  1749 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.354188  1749 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.357434  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 21235338303a443e8f10b8773c68e2db (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:18.358692  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21235338303a443e8f10b8773c68e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.360548  1751 raft_consensus.cc:359] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.361245  1751 raft_consensus.cc:385] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.361526  1751 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.362239  1751 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.362872  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 21235338303a443e8f10b8773c68e2db (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:18.364102  1624 tablet_service.cc:1511] Processing CreateTablet for tablet ce44b855d6d442778079778cd0e68c34 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:18.365293  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce44b855d6d442778079778cd0e68c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.369232  1543 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:18.370186  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21235338303a443e8f10b8773c68e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.370424  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.377348  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:18.378605  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.380656  1751 ts_tablet_manager.cc:1434] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.004s
I20260709 13:00:18.380673  1548 tablet_service.cc:1511] Processing CreateTablet for tablet e27b39f413f94e9192458b7bef0a86b7 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:18.344853  1750 raft_consensus.cc:385] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.381971  1750 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.382943  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27b39f413f94e9192458b7bef0a86b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.384003  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 21235338303a443e8f10b8773c68e2db (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:18.384649  1617 tablet_service.cc:1511] Processing CreateTablet for tablet e27b39f413f94e9192458b7bef0a86b7 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:18.385244  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21235338303a443e8f10b8773c68e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.386230  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27b39f413f94e9192458b7bef0a86b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.422407  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 7346e6ca66624e10898433387e0bbe68 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:18.422096  1545 tablet_service.cc:1511] Processing CreateTablet for tablet b1f19e2b30024564847382b872fc80e1 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:18.418103  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 35904a4f4e3747dd85997b0dae5ae7bc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:18.437374  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7346e6ca66624e10898433387e0bbe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.437542  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35904a4f4e3747dd85997b0dae5ae7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.444844  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f19e2b30024564847382b872fc80e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.444960  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 334958c5aee349629d63e898f13ba95b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:18.382748  1750 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.448719  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 334958c5aee349629d63e898f13ba95b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:18.420912  1547 tablet_service.cc:1511] Processing CreateTablet for tablet 35904a4f4e3747dd85997b0dae5ae7bc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:18.450008  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334958c5aee349629d63e898f13ba95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.450827  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35904a4f4e3747dd85997b0dae5ae7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.451665  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334958c5aee349629d63e898f13ba95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.388104  1619 tablet_service.cc:1511] Processing CreateTablet for tablet 7346e6ca66624e10898433387e0bbe68 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:18.393510  1620 tablet_service.cc:1511] Processing CreateTablet for tablet b1f19e2b30024564847382b872fc80e1 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:18.391360  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:18.464723  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.391319  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 7346e6ca66624e10898433387e0bbe68 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:18.420430  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:18.466971  1708 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:18.467381  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7346e6ca66624e10898433387e0bbe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.409459  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 35904a4f4e3747dd85997b0dae5ae7bc (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:18.419549  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:18.431921  1749 ts_tablet_manager.cc:1434] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 13:00:18.431699  1756 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.470858  1756 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.473639  1756 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:18.463598  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7346e6ca66624e10898433387e0bbe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.476738  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35904a4f4e3747dd85997b0dae5ae7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.477533  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f19e2b30024564847382b872fc80e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.478677  1749 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.485762  1749 tablet_bootstrap.cc:654] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.433661  1695 tablet_service.cc:1511] Processing CreateTablet for tablet e27b39f413f94e9192458b7bef0a86b7 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:18.392951  1694 tablet_service.cc:1511] Processing CreateTablet for tablet b1f19e2b30024564847382b872fc80e1 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:18.436513  1751 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:18.500483  1750 ts_tablet_manager.cc:1434] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.214s	user 0.012s	sys 0.009s
I20260709 13:00:18.502919  1751 tablet_bootstrap.cc:654] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.505151  1750 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:18.508232  1749 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.508625  1757 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.508785  1749 ts_tablet_manager.cc:1403] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.000s
I20260709 13:00:18.509212  1757 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.512301  1749 raft_consensus.cc:359] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.513157  1749 raft_consensus.cc:385] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.513505  1749 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.514263  1749 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.516744  1749 ts_tablet_manager.cc:1434] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:18.517318  1757 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:18.517815  1749 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.527074  1749 tablet_bootstrap.cc:654] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.506939  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27b39f413f94e9192458b7bef0a86b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.533170  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f19e2b30024564847382b872fc80e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.539155  1749 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.539734  1749 ts_tablet_manager.cc:1403] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.006s
I20260709 13:00:18.542990  1749 raft_consensus.cc:359] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.544102  1749 raft_consensus.cc:385] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.544485  1749 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.545253  1749 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.551136  1749 ts_tablet_manager.cc:1434] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:00:18.552309  1749 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.574546  1750 tablet_bootstrap.cc:654] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.584213  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:18.591718  1675 leader_election.cc:343] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.592216  1749 tablet_bootstrap.cc:654] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.560736  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.600128  1754 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.600517  1754 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.600863  1754 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.598143  1696 tablet_service.cc:1511] Processing CreateTablet for tablet e008100b0d36427c939df1036b5e0b67 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:18.606925  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e008100b0d36427c939df1036b5e0b67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:18.607965  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50982.6us
W20260709 13:00:18.608803  1523 leader_election.cc:343] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.609212  1523 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.601719  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 334958c5aee349629d63e898f13ba95b (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:18.618551  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334958c5aee349629d63e898f13ba95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.620376  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 62c0efbdda23450cb269a39d62c15422 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:18.621630  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c0efbdda23450cb269a39d62c15422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.628069  1764 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.628511  1750 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:18.629199  1750 ts_tablet_manager.cc:1403] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.013s
I20260709 13:00:18.628585  1764 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.632637  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:18.634727  1764 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:18.634864  1674 leader_election.cc:343] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.635577  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.637172  1749 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:18.637238  1757 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.637727  1749 ts_tablet_manager.cc:1403] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.007s
I20260709 13:00:18.642978  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:18.643385  1751 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:18.643899  1751 ts_tablet_manager.cc:1403] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.208s	user 0.014s	sys 0.000s
I20260709 13:00:18.644285  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:18.645008  1559 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:18.645257  1622 tablet_service.cc:1511] Processing CreateTablet for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:18.646735  1601 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:18.647183  1749 raft_consensus.cc:359] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.614997  1694 tablet_service.cc:1511] Processing CreateTablet for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:18.647876  1764 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.648016  1749 raft_consensus.cc:385] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.648394  1764 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.648694  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.648852  1764 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.649142  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 62c0efbdda23450cb269a39d62c15422 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:18.649474  1751 raft_consensus.cc:359] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.650262  1751 raft_consensus.cc:385] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.650554  1751 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.651333  1751 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.646667  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.652177  1757 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.656742  1757 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.648419  1749 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.658840  1757 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:00:18.659468  1601 leader_election.cc:343] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.660383  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb72ffc01144a2880010ee54d6e6c44 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:18.648010  1619 tablet_service.cc:1511] Processing CreateTablet for tablet e008100b0d36427c939df1036b5e0b67 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:18.662595  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb72ffc01144a2880010ee54d6e6c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.663216  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c0efbdda23450cb269a39d62c15422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.666923  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:18.669162  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:18.669866  1560 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:18.671097  1675 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:00:18.672081  1674 leader_election.cc:343] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.677007  1757 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.677428  1757 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.677773  1757 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.679440  1754 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.681994  1754 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:18.682540  1754 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:18.685631  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76822.3us
I20260709 13:00:18.687537  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb72ffc01144a2880010ee54d6e6c44 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:18.688951  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb72ffc01144a2880010ee54d6e6c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.689752  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb76969419149b19b9ea9d40ebd2c24 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:18.694547  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:18.695238  1709 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.695447  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb76969419149b19b9ea9d40ebd2c24 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:18.697747  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb76969419149b19b9ea9d40ebd2c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.700332  1751 ts_tablet_manager.cc:1434] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.000s
I20260709 13:00:18.702131  1751 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:18.699237  1547 tablet_service.cc:1511] Processing CreateTablet for tablet e008100b0d36427c939df1036b5e0b67 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:18.707943  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:18.708882  1751 tablet_bootstrap.cc:654] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.708918  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e008100b0d36427c939df1036b5e0b67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:18.709628  1525 leader_election.cc:343] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.709916  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb76969419149b19b9ea9d40ebd2c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.717528  1545 tablet_service.cc:1511] Processing CreateTablet for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:18.719034  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.720749  1750 raft_consensus.cc:359] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.721648  1750 raft_consensus.cc:385] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.722054  1750 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.722916  1750 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.725396  1750 ts_tablet_manager.cc:1434] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:00:18.740262  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 62c0efbdda23450cb269a39d62c15422 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:18.746299  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:00:18.748143  1598 leader_election.cc:343] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.748628  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:18.749836  1756 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.750828  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c0efbdda23450cb269a39d62c15422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.754534  1757 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.757030  1757 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:18.756073  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 26aae1e2780d42cda794411c9c1c0f32 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:18.761327  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 26aae1e2780d42cda794411c9c1c0f32 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:18.762745  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26aae1e2780d42cda794411c9c1c0f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.764341  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 26aae1e2780d42cda794411c9c1c0f32 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:18.764292  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:18.765175  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:18.762481  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb76969419149b19b9ea9d40ebd2c24 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:18.765743  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e008100b0d36427c939df1036b5e0b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.746404  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:18.768253  1560 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.773805  1764 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.773978  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 9df64a488003414aac04146e56487c81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:18.775411  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df64a488003414aac04146e56487c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.775910  1764 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:18.776381  1674 leader_election.cc:343] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.768486  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26aae1e2780d42cda794411c9c1c0f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.778622  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb76969419149b19b9ea9d40ebd2c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.781106  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:18.781770  1559 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:18.784783  1598 leader_election.cc:343] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.769014  1547 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb72ffc01144a2880010ee54d6e6c44 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:18.771394  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 9df64a488003414aac04146e56487c81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:18.788311  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:00:18.790995  1598 leader_election.cc:343] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.792037  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26aae1e2780d42cda794411c9c1c0f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.792695  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df64a488003414aac04146e56487c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.796136  1750 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:18.796625  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb72ffc01144a2880010ee54d6e6c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.798064  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1d1686090d4b3abcf51f94f1a798d4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:18.800050  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1d1686090d4b3abcf51f94f1a798d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.685055  1749 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.806093  1690 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1d1686090d4b3abcf51f94f1a798d4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:18.808346  1690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1d1686090d4b3abcf51f94f1a798d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.810561  1751 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:18.811136  1751 ts_tablet_manager.cc:1403] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.109s	user 0.006s	sys 0.027s
I20260709 13:00:18.812022  1776 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.810616  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1d1686090d4b3abcf51f94f1a798d4 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:18.811683  1543 tablet_service.cc:1511] Processing CreateTablet for tablet c4b89801fe624f179701cbd4b98f1490 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:18.815165  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b89801fe624f179701cbd4b98f1490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.815815  1622 tablet_service.cc:1511] Processing CreateTablet for tablet c4b89801fe624f179701cbd4b98f1490 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:18.819010  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 61539cc4a0c645ed9fb29927e85f3e20 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:18.812520  1776 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.812827  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 9df64a488003414aac04146e56487c81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:18.820344  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 61539cc4a0c645ed9fb29927e85f3e20 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:18.824590  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61539cc4a0c645ed9fb29927e85f3e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.824837  1550 tablet_service.cc:1511] Processing CreateTablet for tablet af498956d398433a8c293e398bd1255f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:18.826129  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1d1686090d4b3abcf51f94f1a798d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.826745  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df64a488003414aac04146e56487c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.827347  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af498956d398433a8c293e398bd1255f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.829931  1751 raft_consensus.cc:359] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.830648  1751 raft_consensus.cc:385] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.830920  1751 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.818539  1750 tablet_bootstrap.cc:654] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.836941  1776 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:18.837461  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:18.837662  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:00:18.839097  1523 leader_election.cc:343] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:18.839588  1525 leader_election.cc:343] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.840114  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.841507  1776 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.842732  1764 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.843567  1776 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.843240  1764 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.844975  1749 ts_tablet_manager.cc:1434] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.207s	user 0.007s	sys 0.002s
I20260709 13:00:18.846254  1709 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:18.845757  1764 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:18.851758  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.853055  1754 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.852922  1698 tablet_service.cc:1511] Processing CreateTablet for tablet c4b89801fe624f179701cbd4b98f1490 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:18.854260  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b89801fe624f179701cbd4b98f1490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.849088  1776 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.869329  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:18.870606  1707 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:18.872263  1541 tablet_service.cc:1511] Processing CreateTablet for tablet 7f8c2a4b167d431aac3de9d8166bab1f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:18.875553  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:18.875646  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:00:18.876946  1601 leader_election.cc:343] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.877627  1624 tablet_service.cc:1511] Processing CreateTablet for tablet af498956d398433a8c293e398bd1255f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:18.878165  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 61539cc4a0c645ed9fb29927e85f3e20 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:18.879423  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61539cc4a0c645ed9fb29927e85f3e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.876001  1541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f8c2a4b167d431aac3de9d8166bab1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.831650  1751 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.848850  1560 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:18.879735  1764 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.873416  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61539cc4a0c645ed9fb29927e85f3e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.886876  1749 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:18.887214  1777 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.887915  1675 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.885118  1764 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.888949  1757 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.889192  1764 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.889532  1757 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:18.890472  1757 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.891104  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:18.892756  1525 leader_election.cc:343] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.887717  1777 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.894928  1777 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:18.895013  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:18.896456  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af498956d398433a8c293e398bd1255f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:18.899470  1523 leader_election.cc:343] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.899983  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.899818  1693 tablet_service.cc:1511] Processing CreateTablet for tablet af498956d398433a8c293e398bd1255f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:18.901247  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af498956d398433a8c293e398bd1255f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.902505  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:18.904073  1674 leader_election.cc:343] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.907886  1776 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:18.908743  1776 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.886233  1754 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:18.912657  1754 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:18.928337  1750 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:18.929020  1750 ts_tablet_manager.cc:1403] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.000s
W20260709 13:00:18.932499  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44716.8us
I20260709 13:00:18.932960  1750 raft_consensus.cc:359] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.933966  1750 raft_consensus.cc:385] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.934356  1750 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.934396  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:18.936163  1675 leader_election.cc:343] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.935516  1750 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.967743  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 4bcf131126594cae9b8c0f94d39caaf8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:18.969224  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcf131126594cae9b8c0f94d39caaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.936650  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:18.939276  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b89801fe624f179701cbd4b98f1490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.940572  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 7f8c2a4b167d431aac3de9d8166bab1f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:18.942376  1559 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:18.942525  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 7f8c2a4b167d431aac3de9d8166bab1f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:18.948325  1619 tablet_service.cc:1511] Processing CreateTablet for tablet 77308d0ddc334cbfa9170c6422895ca9 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:18.949424  1695 tablet_service.cc:1511] Processing CreateTablet for tablet 77308d0ddc334cbfa9170c6422895ca9 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:18.957475  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 77308d0ddc334cbfa9170c6422895ca9 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:18.958187  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 4bcf131126594cae9b8c0f94d39caaf8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:18.955564  1749 tablet_bootstrap.cc:654] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.982589  1601 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:18.983654  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f8c2a4b167d431aac3de9d8166bab1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.000164  1756 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.986632  1751 ts_tablet_manager.cc:1434] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.175s	user 0.009s	sys 0.000s
I20260709 13:00:18.978824  1764 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:18.996045  1781 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.998061  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77308d0ddc334cbfa9170c6422895ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.997834  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 4bcf131126594cae9b8c0f94d39caaf8 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:18.998335  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5f3bc0667d44e69539243d156dac36 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:19.005590  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5f3bc0667d44e69539243d156dac36 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:19.010021  1751 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.010564  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77308d0ddc334cbfa9170c6422895ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.013037  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5f3bc0667d44e69539243d156dac36 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:19.000797  1458 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.007704  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77308d0ddc334cbfa9170c6422895ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.015661  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcf131126594cae9b8c0f94d39caaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.016166  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5f3bc0667d44e69539243d156dac36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.019682  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5f3bc0667d44e69539243d156dac36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.021951  1764 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.028906  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:19.030745  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:00:19.031111  1601 leader_election.cc:343] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.031351  1707 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.990849  1456 catalog_manager.cc:5697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.031105  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f8c2a4b167d431aac3de9d8166bab1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.040814  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcf131126594cae9b8c0f94d39caaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.047919  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5f3bc0667d44e69539243d156dac36. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:19.017086  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 80966.4us
I20260709 13:00:19.056329  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0ffe513d79430292b928b5fcde37fe (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:19.058393  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0ffe513d79430292b928b5fcde37fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.063141  1750 ts_tablet_manager.cc:1434] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.134s	user 0.010s	sys 0.000s
I20260709 13:00:19.066701  1764 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.067238  1764 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.067724  1690 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0ffe513d79430292b928b5fcde37fe (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:19.069270  1690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0ffe513d79430292b928b5fcde37fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.069437  1764 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.071285  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:19.072008  1783 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:00:19.073016  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.073689  1709 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
W20260709 13:00:19.075719  1601 leader_election.cc:343] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.076578  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:19.077821  1764 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.078215  1764 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.078562  1764 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.049948  1756 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:19.082302  1756 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.086788  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0ffe513d79430292b928b5fcde37fe (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:19.087136  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 0d703c11931d4a2688046a3a48b71290 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:19.088372  1750 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:19.088397  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0ffe513d79430292b928b5fcde37fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.104195  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 0d703c11931d4a2688046a3a48b71290 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:19.105651  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d703c11931d4a2688046a3a48b71290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.100883  1764 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.122684  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:00:19.124174  1601 leader_election.cc:343] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.127630  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:19.128280  1709 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.128883  1751 tablet_bootstrap.cc:654] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.096495  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d703c11931d4a2688046a3a48b71290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.130884  1764 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.149691  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 309f24e44cc94815a1ad3911262a7872 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:19.149674  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 0d703c11931d4a2688046a3a48b71290 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:19.167814  1749 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.168406  1749 ts_tablet_manager.cc:1403] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.294s	user 0.027s	sys 0.007s
I20260709 13:00:19.171257  1621 tablet_service.cc:1511] Processing CreateTablet for tablet bcd4e3d43c7d4c97a3f229082334515f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:19.172320  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd4e3d43c7d4c97a3f229082334515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.172750  1692 tablet_service.cc:1511] Processing CreateTablet for tablet bcd4e3d43c7d4c97a3f229082334515f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:19.174113  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd4e3d43c7d4c97a3f229082334515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.167811  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 309f24e44cc94815a1ad3911262a7872 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:19.177059  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f24e44cc94815a1ad3911262a7872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.181631  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f24e44cc94815a1ad3911262a7872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.183876  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d703c11931d4a2688046a3a48b71290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.185482  1549 tablet_service.cc:1511] Processing CreateTablet for tablet c9744e1e31604277953315481c56b171 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:19.186757  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9744e1e31604277953315481c56b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.190834  1707 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:19.216519  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:19.220213  1785 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.220739  1785 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:19.221905  1785 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.217144  1750 tablet_bootstrap.cc:654] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.224980  1542 tablet_service.cc:1511] Processing CreateTablet for tablet bcd4e3d43c7d4c97a3f229082334515f (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:19.218374  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 309f24e44cc94815a1ad3911262a7872 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:19.226583  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f24e44cc94815a1ad3911262a7872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.171355  1749 raft_consensus.cc:359] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.231491  1749 raft_consensus.cc:385] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.226608  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd4e3d43c7d4c97a3f229082334515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.236547  1619 tablet_service.cc:1511] Processing CreateTablet for tablet c9744e1e31604277953315481c56b171 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:19.238315  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9744e1e31604277953315481c56b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.241084  1751 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:19.239974  1691 tablet_service.cc:1511] Processing CreateTablet for tablet c9744e1e31604277953315481c56b171 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:19.241742  1751 ts_tablet_manager.cc:1403] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.232s	user 0.011s	sys 0.009s
I20260709 13:00:19.242717  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9744e1e31604277953315481c56b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.231878  1749 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.260718  1751 raft_consensus.cc:359] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.272578  1751 raft_consensus.cc:385] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.273177  1751 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.279243  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 78699c8b654944aa9788917e2a177d14 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:19.280715  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78699c8b654944aa9788917e2a177d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.279959  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 78699c8b654944aa9788917e2a177d14 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:19.281752  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78699c8b654944aa9788917e2a177d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.276825  1749 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:19.285640  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:19.285087  1457 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.290321  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 78699c8b654944aa9788917e2a177d14 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:19.291709  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78699c8b654944aa9788917e2a177d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.296010  1709 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:19.303488  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:19.315003  1749 ts_tablet_manager.cc:1434] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.146s	user 0.009s	sys 0.000s
I20260709 13:00:19.284440  1751 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.316143  1749 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:19.317960  1751 ts_tablet_manager.cc:1434] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:00:19.318956  1751 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.323506  1749 tablet_bootstrap.cc:654] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.315529  1785 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.347492  1785 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:19.345220  1750 tablet_bootstrap.cc:492] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:19.348505  1750 ts_tablet_manager.cc:1403] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.260s	user 0.028s	sys 0.040s
I20260709 13:00:19.348392  1785 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.351830  1750 raft_consensus.cc:359] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:19.352573  1750 raft_consensus.cc:385] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.352869  1750 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.355396  1550 tablet_service.cc:1511] Processing CreateTablet for tablet de80dd9c151b41e88e1469985d074b81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:19.356909  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de80dd9c151b41e88e1469985d074b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.362962  1618 tablet_service.cc:1511] Processing CreateTablet for tablet de80dd9c151b41e88e1469985d074b81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:19.368342  1542 tablet_service.cc:1511] Processing CreateTablet for tablet bd868018972045628b3604762f904e05 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:19.373077  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd868018972045628b3604762f904e05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.364253  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de80dd9c151b41e88e1469985d074b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.369880  1697 tablet_service.cc:1511] Processing CreateTablet for tablet de80dd9c151b41e88e1469985d074b81 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:19.375406  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de80dd9c151b41e88e1469985d074b81. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:19.383196  1525 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:19.363724  1751 tablet_bootstrap.cc:654] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:19.388983  1674 consensus_peers.cc:597] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:19.394582  1621 tablet_service.cc:1511] Processing CreateTablet for tablet bd868018972045628b3604762f904e05 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:19.395911  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd868018972045628b3604762f904e05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.396426  1749 tablet_bootstrap.cc:492] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.398052  1749 ts_tablet_manager.cc:1403] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.001s
I20260709 13:00:19.400110  1691 tablet_service.cc:1511] Processing CreateTablet for tablet bd868018972045628b3604762f904e05 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:19.401470  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd868018972045628b3604762f904e05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.396059  1750 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:19.403544  1750 ts_tablet_manager.cc:1434] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:00:19.404577  1750 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:19.400676  1749 raft_consensus.cc:359] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.407397  1749 raft_consensus.cc:385] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.407833  1749 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.417999  1751 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:19.418735  1751 ts_tablet_manager.cc:1403] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.100s	user 0.003s	sys 0.009s
I20260709 13:00:19.443727  1749 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.454051  1749 ts_tablet_manager.cc:1434] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.056s	user 0.014s	sys 0.004s
I20260709 13:00:19.458213  1749 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:19.460150  1692 tablet_service.cc:1511] Processing CreateTablet for tablet dea1928bdd634efaa34550f21ff564f3 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:19.462052  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea1928bdd634efaa34550f21ff564f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.464277  1749 tablet_bootstrap.cc:654] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.465122  1782 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.465950  1751 raft_consensus.cc:359] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.466704  1751 raft_consensus.cc:385] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.466995  1751 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.468587  1542 tablet_service.cc:1511] Processing CreateTablet for tablet dea1928bdd634efaa34550f21ff564f3 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:19.469751  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea1928bdd634efaa34550f21ff564f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.472085  1750 tablet_bootstrap.cc:654] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.488879  1549 tablet_service.cc:1511] Processing CreateTablet for tablet f9d92cdb4b624c3bbc074afe1340b761 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:19.490723  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d92cdb4b624c3bbc074afe1340b761. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:19.500103  1598 consensus_peers.cc:597] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:19.501340  1781 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.506085  1618 tablet_service.cc:1511] Processing CreateTablet for tablet f9d92cdb4b624c3bbc074afe1340b761 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:19.507457  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d92cdb4b624c3bbc074afe1340b761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.509773  1545 tablet_service.cc:1511] Processing CreateTablet for tablet bcf9e0da0f934b968a7e9da19f0e0749 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:19.511078  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf9e0da0f934b968a7e9da19f0e0749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.512148  1621 tablet_service.cc:1511] Processing CreateTablet for tablet dea1928bdd634efaa34550f21ff564f3 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:19.512107  1458 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.512735  1697 tablet_service.cc:1511] Processing CreateTablet for tablet f9d92cdb4b624c3bbc074afe1340b761 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:19.517580  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d92cdb4b624c3bbc074afe1340b761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.517176  1458 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.520867  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea1928bdd634efaa34550f21ff564f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.523582  1619 tablet_service.cc:1511] Processing CreateTablet for tablet bcf9e0da0f934b968a7e9da19f0e0749 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:19.524879  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf9e0da0f934b968a7e9da19f0e0749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.490710  1751 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.533720  1751 ts_tablet_manager.cc:1434] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.115s	user 0.003s	sys 0.007s
I20260709 13:00:19.541522  1751 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.539893  1623 tablet_service.cc:1511] Processing CreateTablet for tablet e43ddec31b504a3b9c8f06a96cd7013d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:19.543939  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ddec31b504a3b9c8f06a96cd7013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.561681  1691 tablet_service.cc:1511] Processing CreateTablet for tablet bcf9e0da0f934b968a7e9da19f0e0749 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:19.563072  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf9e0da0f934b968a7e9da19f0e0749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.566131  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 9c25c5b342d344f3ac325ab5073d1747 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:19.569567  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c25c5b342d344f3ac325ab5073d1747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.572751  1542 tablet_service.cc:1511] Processing CreateTablet for tablet e43ddec31b504a3b9c8f06a96cd7013d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:19.575529  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ddec31b504a3b9c8f06a96cd7013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.582068  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 9c25c5b342d344f3ac325ab5073d1747 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:19.583025  1781 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.583547  1781 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.584098  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c25c5b342d344f3ac325ab5073d1747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.586180  1781 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:19.587375  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:19.588272  1560 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:19.590103  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:19.592110  1674 leader_election.cc:343] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.592718  1785 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.594038  1675 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:19.595847  1781 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.596356  1781 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.596835  1781 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.597954  1749 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.598435  1749 ts_tablet_manager.cc:1403] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.140s	user 0.026s	sys 0.021s
I20260709 13:00:19.601190  1749 raft_consensus.cc:359] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.602320  1749 raft_consensus.cc:385] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.602593  1749 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.603252  1749 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.610661  1692 tablet_service.cc:1511] Processing CreateTablet for tablet e43ddec31b504a3b9c8f06a96cd7013d (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:19.611768  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ddec31b504a3b9c8f06a96cd7013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.604831  1751 tablet_bootstrap.cc:654] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.619402  1750 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:19.619973  1750 ts_tablet_manager.cc:1403] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.216s	user 0.021s	sys 0.011s
I20260709 13:00:19.620946  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 88dfd27965e145dcb810423d7396b403 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:19.622318  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88dfd27965e145dcb810423d7396b403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.623095  1750 raft_consensus.cc:359] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.625908  1750 raft_consensus.cc:385] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.626225  1750 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.626763  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce0393bceaa489d896dd65113130131 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:19.640002  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0393bceaa489d896dd65113130131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.640910  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 88dfd27965e145dcb810423d7396b403 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:19.644567  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 9c25c5b342d344f3ac325ab5073d1747 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:19.626994  1750 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.645869  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c25c5b342d344f3ac325ab5073d1747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.654232  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88dfd27965e145dcb810423d7396b403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.653234  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce0393bceaa489d896dd65113130131 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:19.655917  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0393bceaa489d896dd65113130131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.671322  1749 ts_tablet_manager.cc:1434] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.073s	user 0.011s	sys 0.000s
I20260709 13:00:19.672401  1749 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:19.661681  1802 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.679499  1802 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.679001  1750 ts_tablet_manager.cc:1434] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 13:00:19.707907  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:19.708252  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.708597  1634 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:19.709851  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:19.710824  1808 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.711227  1808 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.711575  1808 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.713299  1781 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.715372  1781 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:19.717931  1559 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.720624  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 88dfd27965e145dcb810423d7396b403 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:00:19.720809  1523 leader_election.cc:343] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.722137  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88dfd27965e145dcb810423d7396b403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.724787  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:19.725392  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce0393bceaa489d896dd65113130131 (DEFAULT_TABLE table=test-workload [id=5089d902ed414408826ea2cd4e7a981f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:19.726781  1802 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.726792  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0393bceaa489d896dd65113130131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:19.739928  1806 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.740649  1806 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.743752  1806 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:00:19.731935  1674 leader_election.cc:343] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.732100  1750 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:19.743944  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:19.761127  1560 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.763967  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:19.768337  1785 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.768810  1751 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:19.769320  1751 ts_tablet_manager.cc:1403] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.228s	user 0.009s	sys 0.018s
I20260709 13:00:19.770965  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:19.772305  1674 leader_election.cc:343] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.764690  1557 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:19.772154  1751 raft_consensus.cc:359] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.772894  1751 raft_consensus.cc:385] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.773228  1751 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.773703  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:19.773939  1751 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.776429  1812 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.777709  1812 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.778132  1812 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.779956  1749 tablet_bootstrap.cc:654] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.783735  1756 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.784910  1751 ts_tablet_manager.cc:1434] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:00:19.784368  1756 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:00:19.786696  1756 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.793648  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.802927  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:19.804586  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:19.804436  1756 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.812600  1560 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:19.813156  1756 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.813529  1808 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.814787  1802 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.818487  1675 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:19.818773  1634 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.825232  1804 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.825716  1804 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.826400  1808 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.831604  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:00:19.833953  1523 leader_election.cc:343] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.822942  1802 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.834501  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:19.835346  1635 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.831545  1807 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.840269  1807 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:19.841559  1751 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.845679  1802 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.846640  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.836015  1804 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.851824  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:19.853595  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:19.854350  1804 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.856907  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.841144  1807 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.858587  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:19.862840  1818 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:00:19.863704  1525 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:19.864975  1804 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.875110  1751 tablet_bootstrap.cc:654] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.875552  1749 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.876051  1749 ts_tablet_manager.cc:1403] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.204s	user 0.011s	sys 0.007s
I20260709 13:00:19.878294  1632 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.878680  1749 raft_consensus.cc:359] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.879362  1749 raft_consensus.cc:385] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.879628  1749 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.880420  1749 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.883813  1749 ts_tablet_manager.cc:1434] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:00:19.884605  1750 tablet_bootstrap.cc:654] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.884924  1749 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:19.892017  1557 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.907898  1751 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:19.908394  1751 ts_tablet_manager.cc:1403] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.007s
I20260709 13:00:19.911599  1749 tablet_bootstrap.cc:654] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.916153  1751 raft_consensus.cc:359] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.916904  1751 raft_consensus.cc:385] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.917207  1751 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.917850  1751 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.918973  1812 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.920828  1751 ts_tablet_manager.cc:1434] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:19.921120  1812 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:19.921988  1751 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.923720  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:19.925236  1674 leader_election.cc:343] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.927208  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:19.927878  1560 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.928857  1751 tablet_bootstrap.cc:654] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.928679  1456 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.935122  1749 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.935587  1749 ts_tablet_manager.cc:1403] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.002s
I20260709 13:00:19.936199  1750 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:19.936678  1750 ts_tablet_manager.cc:1403] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.232s	user 0.014s	sys 0.005s
I20260709 13:00:19.938290  1749 raft_consensus.cc:359] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.939433  1749 raft_consensus.cc:385] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.939703  1749 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.939814  1750 raft_consensus.cc:359] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.940507  1750 raft_consensus.cc:385] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.940893  1750 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.940614  1749 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.942384  1749 ts_tablet_manager.cc:1434] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:00:19.945679  1750 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.948733  1750 ts_tablet_manager.cc:1434] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:19.949148  1818 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.952011  1785 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:00:19.952726  1785 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.957139  1785 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:19.958455  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:19.962582  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:19.962900  1749 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:19.964250  1601 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:19.965024  1785 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.966049  1750 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:19.972298  1751 tablet_bootstrap.cc:492] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:19.972860  1751 ts_tablet_manager.cc:1403] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.018s
I20260709 13:00:19.973666  1749 tablet_bootstrap.cc:654] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.973714  1750 tablet_bootstrap.cc:654] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.976058  1751 raft_consensus.cc:359] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.976876  1751 raft_consensus.cc:385] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.977320  1751 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.978296  1751 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.980865  1751 ts_tablet_manager.cc:1434] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:00:19.982111  1751 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:19.986639  1749 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:19.987164  1749 ts_tablet_manager.cc:1403] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.005s
I20260709 13:00:19.988853  1751 tablet_bootstrap.cc:654] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.989948  1749 raft_consensus.cc:359] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.990682  1749 raft_consensus.cc:385] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.990965  1749 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.992391  1749 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:19.997040  1749 ts_tablet_manager.cc:1434] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:00:19.997987  1749 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.000872  1750 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.001106  1751 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.001852  1750 ts_tablet_manager.cc:1403] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.008s
I20260709 13:00:20.002425  1751 ts_tablet_manager.cc:1403] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:00:20.004475  1751 raft_consensus.cc:359] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.004431  1750 raft_consensus.cc:359] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.005134  1751 raft_consensus.cc:385] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.005246  1750 raft_consensus.cc:385] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.005494  1751 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.005613  1750 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.006312  1750 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.006970  1751 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.008246  1750 ts_tablet_manager.cc:1434] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:00:20.009253  1751 ts_tablet_manager.cc:1434] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:00:20.009418  1750 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.010437  1751 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.015317  1749 tablet_bootstrap.cc:654] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.016883  1750 tablet_bootstrap.cc:654] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.018106  1751 tablet_bootstrap.cc:654] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.027093  1635 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:20.028436  1560 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.029768  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:20.031075  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.031934  1807 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.032390  1807 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:20.033151  1807 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.037185  1804 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.037737  1804 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:20.038537  1804 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.044480  1751 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.044943  1751 ts_tablet_manager.cc:1403] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.000s
I20260709 13:00:20.047358  1751 raft_consensus.cc:359] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.048020  1751 raft_consensus.cc:385] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.048280  1751 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.054430  1751 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.058477  1751 ts_tablet_manager.cc:1434] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:00:20.054325  1455 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.059435  1751 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.068571  1709 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.084115  1804 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.084606  1804 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.086969  1751 tablet_bootstrap.cc:654] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.087116  1804 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.087119  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.087338  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.090821  1750 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.091356  1750 ts_tablet_manager.cc:1403] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.082s	user 0.025s	sys 0.035s
I20260709 13:00:20.091050  1455 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:20.095667  1523 leader_election.cc:343] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.096078  1749 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.096828  1749 ts_tablet_manager.cc:1403] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.099s	user 0.039s	sys 0.021s
I20260709 13:00:20.096284  1750 raft_consensus.cc:359] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.097937  1750 raft_consensus.cc:385] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:20.098874  1525 leader_election.cc:343] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.099318  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.100500  1802 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.098246  1750 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.101670  1750 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.104790  1749 raft_consensus.cc:359] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.105502  1749 raft_consensus.cc:385] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.105772  1749 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.106415  1749 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.107059  1750 ts_tablet_manager.cc:1434] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 13:00:20.108088  1750 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.108438  1749 ts_tablet_manager.cc:1434] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:00:20.109349  1749 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.114502  1751 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.115029  1751 ts_tablet_manager.cc:1403] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.056s	user 0.005s	sys 0.006s
I20260709 13:00:20.115477  1749 tablet_bootstrap.cc:654] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.116464  1750 tablet_bootstrap.cc:654] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.118664  1751 raft_consensus.cc:359] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.119395  1751 raft_consensus.cc:385] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.119648  1751 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.121229  1751 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.128773  1785 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.129289  1785 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.130748  1751 ts_tablet_manager.cc:1434] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:00:20.131335  1785 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.131692  1751 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.133630  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.135887  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.137485  1560 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:20.134024  1750 tablet_bootstrap.cc:492] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.139932  1750 ts_tablet_manager.cc:1403] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.001s
I20260709 13:00:20.140604  1601 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:20.141906  1785 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.143026  1750 raft_consensus.cc:359] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.149816  1750 raft_consensus.cc:385] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.150969  1750 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.151742  1750 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.164811  1750 ts_tablet_manager.cc:1434] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 13:00:20.165957  1750 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.170419  1749 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.170884  1749 ts_tablet_manager.cc:1403] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.007s
I20260709 13:00:20.172889  1751 tablet_bootstrap.cc:654] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.174175  1750 tablet_bootstrap.cc:654] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.178474  1749 raft_consensus.cc:359] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.179206  1749 raft_consensus.cc:385] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.179543  1749 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.180043  1751 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.180485  1751 ts_tablet_manager.cc:1403] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.001s
I20260709 13:00:20.180339  1749 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.183012  1751 raft_consensus.cc:359] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.184873  1751 raft_consensus.cc:385] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.185154  1751 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.186025  1749 ts_tablet_manager.cc:1434] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 13:00:20.185844  1751 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.186933  1749 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.188346  1751 ts_tablet_manager.cc:1434] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:00:20.190122  1750 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.190594  1750 ts_tablet_manager.cc:1403] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 13:00:20.191035  1751 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.193840  1750 raft_consensus.cc:359] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.194540  1750 raft_consensus.cc:385] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.194834  1750 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.195474  1750 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.196264  1807 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.196729  1807 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.197578  1750 ts_tablet_manager.cc:1434] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:20.198678  1750 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.198853  1807 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.199150  1751 tablet_bootstrap.cc:654] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.201020  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.202626  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:20.202754  1674 leader_election.cc:343] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.203272  1749 tablet_bootstrap.cc:654] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.203449  1560 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:20.204831  1675 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.206115  1807 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.206487  1807 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.206851  1807 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.210388  1750 tablet_bootstrap.cc:654] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.218022  1749 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.218525  1749 ts_tablet_manager.cc:1403] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.003s
I20260709 13:00:20.221206  1749 raft_consensus.cc:359] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.221908  1749 raft_consensus.cc:385] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.222203  1749 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.223004  1749 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.224857  1785 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.224711  1807 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.225899  1749 ts_tablet_manager.cc:1434] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:00:20.226799  1807 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.228427  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:20.229847  1674 leader_election.cc:343] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.232765  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:20.232939  1749 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.233353  1560 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.247138  1807 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:20.252482  1807 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.255813  1750 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.256233  1750 ts_tablet_manager.cc:1403] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.000s
I20260709 13:00:20.260118  1812 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.260751  1812 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.253396  1807 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.262351  1632 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.263046  1812 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.270180  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.271175  1635 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:20.271796  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.273841  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.275081  1812 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.276903  1751 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.277381  1751 ts_tablet_manager.cc:1403] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.037s
I20260709 13:00:20.279549  1807 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.281383  1782 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.284266  1749 tablet_bootstrap.cc:654] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.284268  1782 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.287950  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.288641  1559 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:20.289351  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.290025  1782 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.291441  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.291759  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.292526  1675 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:20.292870  1635 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
W20260709 13:00:20.293640  1525 leader_election.cc:343] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.295147  1807 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.295570  1807 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.295972  1807 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.305240  1750 raft_consensus.cc:359] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.305847  1750 raft_consensus.cc:385] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.306146  1750 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.306912  1750 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.308213  1751 raft_consensus.cc:359] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.309037  1750 ts_tablet_manager.cc:1434] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.002s
I20260709 13:00:20.309118  1751 raft_consensus.cc:385] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.309574  1751 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.310565  1751 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:20.305385  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:20.306190  1782 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.313932  1813 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.315061  1751 ts_tablet_manager.cc:1434] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 13:00:20.314898  1813 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.316006  1751 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.317924  1813 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.314265  1709 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:20.319630  1750 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.315677  1782 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.324622  1785 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.324800  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.325400  1707 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:20.325165  1785 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.326819  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.327478  1632 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:20.328177  1812 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.329963  1560 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.330137  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:20.331490  1751 tablet_bootstrap.cc:654] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.332588  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.333237  1804 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.333021  1812 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.333318  1635 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:20.333539  1804 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.333909  1804 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.334792  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.334741  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:20.340404  1785 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.342108  1806 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.342509  1806 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.342865  1806 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.342455  1523 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.344883  1802 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.345248  1802 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.345584  1802 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.346069  1749 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.346261  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.346547  1749 ts_tablet_manager.cc:1403] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.114s	user 0.020s	sys 0.016s
I20260709 13:00:20.348927  1601 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:20.352113  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.352936  1560 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:20.353780  1675 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.329643  1782 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.363195  1813 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.363731  1813 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:20.365283  1812 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.365996  1812 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.368646  1812 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.368685  1813 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.369884  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:20.371616  1674 leader_election.cc:343] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.376391  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.379981  1750 tablet_bootstrap.cc:654] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.382823  1749 raft_consensus.cc:359] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.383697  1749 raft_consensus.cc:385] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.384014  1749 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.385440  1749 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.386345  1560 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:20.387723  1749 ts_tablet_manager.cc:1434] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 13:00:20.389037  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.390934  1749 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.395990  1849 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.396381  1750 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.396353  1849 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.396793  1849 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.396896  1750 ts_tablet_manager.cc:1403] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.008s
I20260709 13:00:20.398160  1785 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.406708  1749 tablet_bootstrap.cc:654] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.408689  1750 raft_consensus.cc:359] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.409701  1750 raft_consensus.cc:385] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.410049  1750 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.411410  1750 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.414232  1750 ts_tablet_manager.cc:1434] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:00:20.416406  1750 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.417963  1785 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.418553  1785 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.421489  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.422477  1560 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:20.423439  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.425395  1785 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.425801  1807 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.426015  1802 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.427621  1804 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.428962  1807 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.429627  1804 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.431473  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:20.432155  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:20.432950  1632 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.429646  1802 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.437414  1458 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.440093  1751 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.440627  1751 ts_tablet_manager.cc:1403] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.125s	user 0.009s	sys 0.014s
I20260709 13:00:20.436811  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:20.442767  1525 leader_election.cc:343] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.437013  1785 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.454577  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:20.455229  1560 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.462378  1849 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.463385  1785 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.438999  1804 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.464082  1804 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.472101  1750 tablet_bootstrap.cc:654] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.486765  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.483673  1804 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:20.490738  1525 leader_election.cc:343] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.492610  1806 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.494076  1560 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.495325  1849 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.495981  1675 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:20.498145  1751 raft_consensus.cc:359] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.498759  1751 raft_consensus.cc:385] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.499013  1751 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.499876  1751 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.503158  1751 ts_tablet_manager.cc:1434] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 13:00:20.503189  1851 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.504134  1851 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:20.504339  1751 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.488618  1632 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
W20260709 13:00:20.498059  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 77739.4us
I20260709 13:00:20.510723  1751 tablet_bootstrap.cc:654] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.502355  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:20.512032  1560 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.514523  1750 tablet_bootstrap.cc:492] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.508158  1806 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.514678  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:20.508878  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:20.514990  1750 ts_tablet_manager.cc:1403] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.099s	user 0.009s	sys 0.002s
I20260709 13:00:20.515712  1634 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.518507  1750 raft_consensus.cc:359] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.519138  1750 raft_consensus.cc:385] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.519389  1750 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.520068  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:20.520040  1750 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.520407  1858 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.511020  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:20.521612  1557 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.512456  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:20.522003  1804 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.522213  1750 ts_tablet_manager.cc:1434] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:20.522316  1632 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.522640  1804 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:20.523608  1804 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.526947  1557 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.521044  1635 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.528199  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:20.510026  1749 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.531447  1632 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.532804  1634 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.532958  1806 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.506075  1851 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.533710  1806 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:20.535069  1635 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.531831  1749 ts_tablet_manager.cc:1403] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.060s
I20260709 13:00:20.543243  1750 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.548182  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:20.548873  1709 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.534902  1806 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.539304  1749 raft_consensus.cc:359] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.553709  1749 raft_consensus.cc:385] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.554220  1749 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.555580  1749 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.556567  1458 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.559166  1709 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:20.559525  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:20.560132  1707 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.560405  1818 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.560812  1818 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.562467  1749 ts_tablet_manager.cc:1434] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:00:20.562930  1818 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.559804  1802 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.563910  1749 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.565761  1560 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:20.566098  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.567020  1559 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:20.568073  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:20.568975  1858 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.569564  1858 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:20.570289  1707 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:20.572232  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.574211  1749 tablet_bootstrap.cc:654] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.582252  1750 tablet_bootstrap.cc:654] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.574163  1523 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.574903  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:20.585421  1802 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:20.585613  1598 leader_election.cc:343] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.585974  1802 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:20.586248  1818 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.586678  1818 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.587172  1818 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.586800  1802 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.587913  1782 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.595216  1855 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.578148  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.578382  1751 tablet_bootstrap.cc:492] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.571030  1858 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.598413  1751 ts_tablet_manager.cc:1403] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.094s	user 0.019s	sys 0.016s
I20260709 13:00:20.604451  1749 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.605026  1749 ts_tablet_manager.cc:1403] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.004s
I20260709 13:00:20.610525  1751 raft_consensus.cc:359] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.611625  1818 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.612202  1751 raft_consensus.cc:385] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.612586  1751 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.613255  1751 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.617954  1749 raft_consensus.cc:359] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.618978  1749 raft_consensus.cc:385] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.619374  1749 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.598372  1707 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:20.621191  1749 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.627712  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.629024  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:20.629810  1559 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.632023  1804 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:20.633409  1751 ts_tablet_manager.cc:1434] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.003s
I20260709 13:00:20.641561  1749 ts_tablet_manager.cc:1434] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.003s
I20260709 13:00:20.633018  1804 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.642597  1804 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.642733  1749 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.659698  1751 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.651365  1804 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.656985  1806 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.668233  1806 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.670521  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:20.676299  1782 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.707115  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:20.713037  1782 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.681033  1818 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.715739  1782 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.797091  1707 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.806802  1804 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:20.808591  1525 leader_election.cc:343] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.809861  1455 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.810829  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.819903  1458 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.817811  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:20.687067  1559 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:20.832737  1751 tablet_bootstrap.cc:654] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.834096  1749 tablet_bootstrap.cc:654] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.834316  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:20.824699  1707 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:20.839670  1750 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.844733  1750 ts_tablet_manager.cc:1403] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.302s	user 0.010s	sys 0.010s
I20260709 13:00:20.837322  1818 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.845856  1818 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:00:20.842756  1525 leader_election.cc:343] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.846876  1818 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.848320  1806 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:20.852665  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.851835  1750 raft_consensus.cc:359] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.860222  1750 raft_consensus.cc:385] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.860483  1750 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.861557  1750 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.863562  1750 ts_tablet_manager.cc:1434] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:00:20.864378  1750 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.869772  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.872613  1806 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.881780  1749 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.882274  1749 ts_tablet_manager.cc:1403] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.240s	user 0.012s	sys 0.010s
I20260709 13:00:20.883890  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:20.884456  1709 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:20.885496  1457 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.890295  1750 tablet_bootstrap.cc:654] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.890791  1871 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.891116  1751 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:20.891562  1751 ts_tablet_manager.cc:1403] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.232s	user 0.020s	sys 0.012s
I20260709 13:00:20.891245  1871 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.893250  1749 raft_consensus.cc:359] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.893923  1749 raft_consensus.cc:385] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.894199  1749 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.895486  1749 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.900053  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.900417  1858 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:00:20.905841  1523 leader_election.cc:343] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.906457  1523 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.907436  1782 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.908300  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:20.909287  1709 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
W20260709 13:00:20.910609  1525 leader_election.cc:343] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.911216  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.912030  1782 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:20.912480  1782 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:20.912838  1871 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:20.913303  1782 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.915405  1751 raft_consensus.cc:359] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.916093  1751 raft_consensus.cc:385] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.916374  1751 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.917655  1749 ts_tablet_manager.cc:1434] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:00:20.917693  1751 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.919507  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:20.920948  1751 ts_tablet_manager.cc:1434] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.002s
I20260709 13:00:20.922060  1751 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:20.925670  1749 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:20.924296  1458 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:20.929808  1523 leader_election.cc:343] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:20.930683  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:20.932534  1458 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.937353  1872 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:20.939990  1750 tablet_bootstrap.cc:492] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:20.940667  1750 ts_tablet_manager.cc:1403] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.076s	user 0.019s	sys 0.003s
I20260709 13:00:20.953133  1750 raft_consensus.cc:359] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.953341  1457 catalog_manager.cc:5697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.956139  1749 tablet_bootstrap.cc:654] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.955919  1750 raft_consensus.cc:385] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.957553  1750 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:20.958653  1858 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.961941  1858 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.970997  1750 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.974635  1750 ts_tablet_manager.cc:1434] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.003s
I20260709 13:00:20.977304  1749 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:20.977744  1749 ts_tablet_manager.cc:1403] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.012s
I20260709 13:00:20.981243  1751 tablet_bootstrap.cc:654] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:20.984967  1782 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.980280  1749 raft_consensus.cc:359] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:20.986953  1749 raft_consensus.cc:385] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:20.987757  1806 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.996124  1750 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:20.999749  1806 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:21.008635  1782 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.987257  1749 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.020282  1750 tablet_bootstrap.cc:654] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.025162  1749 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.026871  1751 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.027434  1751 ts_tablet_manager.cc:1403] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.106s	user 0.008s	sys 0.006s
I20260709 13:00:21.027493  1806 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.028461  1749 ts_tablet_manager.cc:1434] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 13:00:21.029726  1749 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.030589  1751 raft_consensus.cc:359] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.031344  1751 raft_consensus.cc:385] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.031720  1751 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.032814  1751 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.042335  1749 tablet_bootstrap.cc:654] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.042645  1751 ts_tablet_manager.cc:1434] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:00:21.044005  1751 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.051422  1749 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.051884  1749 ts_tablet_manager.cc:1403] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.003s
W20260709 13:00:21.052891  1525 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:21.053090  1751 tablet_bootstrap.cc:654] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.054877  1749 raft_consensus.cc:359] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.055756  1749 raft_consensus.cc:385] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.056049  1749 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.056579  1749 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.059083  1749 ts_tablet_manager.cc:1434] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:21.059985  1749 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.060140  1751 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.060549  1751 ts_tablet_manager.cc:1403] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 13:00:21.063026  1751 raft_consensus.cc:359] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.063714  1751 raft_consensus.cc:385] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.063926  1751 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.064602  1751 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.065923  1749 tablet_bootstrap.cc:654] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.066747  1751 ts_tablet_manager.cc:1434] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:21.067821  1751 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.075721  1751 tablet_bootstrap.cc:654] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.090087  1872 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.102046  1750 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.102572  1750 ts_tablet_manager.cc:1403] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.127s	user 0.006s	sys 0.007s
I20260709 13:00:21.109683  1806 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:21.120064  1750 raft_consensus.cc:359] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.120993  1750 raft_consensus.cc:385] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.121311  1750 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.122359  1750 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.128247  1750 ts_tablet_manager.cc:1434] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 13:00:21.129112  1750 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.143370  1806 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.160053  1750 tablet_bootstrap.cc:654] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.160838  1749 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.161304  1749 ts_tablet_manager.cc:1403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.102s	user 0.015s	sys 0.036s
I20260709 13:00:21.163282  1751 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.163746  1751 ts_tablet_manager.cc:1403] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.096s	user 0.032s	sys 0.021s
I20260709 13:00:21.164574  1749 raft_consensus.cc:359] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.165504  1749 raft_consensus.cc:385] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.166046  1749 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.167675  1749 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.169497  1751 raft_consensus.cc:359] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.170217  1751 raft_consensus.cc:385] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.170516  1751 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.171164  1751 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.173724  1751 ts_tablet_manager.cc:1434] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 13:00:21.174696  1751 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.171594  1749 ts_tablet_manager.cc:1434] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:21.178155  1818 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.178323  1749 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.178642  1818 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.180632  1818 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.183348  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.184118  1709 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:21.184490  1749 tablet_bootstrap.cc:654] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.185524  1751 tablet_bootstrap.cc:654] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.186041  1750 tablet_bootstrap.cc:492] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.188730  1750 ts_tablet_manager.cc:1403] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.001s
I20260709 13:00:21.191248  1750 raft_consensus.cc:359] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.192355  1750 raft_consensus.cc:385] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.192687  1750 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.192956  1749 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.193373  1749 ts_tablet_manager.cc:1403] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:00:21.193544  1750 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.194633  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.196568  1750 ts_tablet_manager.cc:1434] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:00:21.196374  1749 raft_consensus.cc:359] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.197026  1749 raft_consensus.cc:385] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.197290  1749 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.197688  1750 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.199417  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:21.199528  1749 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.201794  1749 ts_tablet_manager.cc:1434] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:21.202092  1818 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.202639  1749 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.204414  1858 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.204854  1858 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.205142  1756 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.205611  1756 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.207340  1858 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.208750  1756 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.208983  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:21.210316  1674 leader_election.cc:343] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.211045  1751 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.211455  1751 ts_tablet_manager.cc:1403] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.012s
I20260709 13:00:21.211508  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.211639  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.212277  1560 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:21.212296  1559 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.213490  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.213586  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.213788  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.214043  1751 raft_consensus.cc:359] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.214313  1756 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.214396  1709 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:21.214614  1858 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.214759  1756 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.215093  1858 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.215168  1756 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.215480  1858 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.216730  1749 tablet_bootstrap.cc:654] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.214670  1751 raft_consensus.cc:385] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.219270  1751 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.218580  1872 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.218755  1818 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.220134  1872 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.220377  1751 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.222692  1751 ts_tablet_manager.cc:1434] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:00:21.224123  1751 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.224717  1872 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.228112  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.229336  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.229177  1756 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.230000  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:21.231281  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:00:21.233307  1525 leader_election.cc:343] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.236232  1858 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.239192  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.239897  1559 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.247460  1559 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:21.248751  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:21.238286  1872 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.249634  1851 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.249884  1872 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.250185  1851 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:21.250316  1872 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.251086  1851 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.254591  1756 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.268853  1858 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.258411  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.275022  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.275499  1560 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.276141  1750 tablet_bootstrap.cc:654] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.272722  1872 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.278043  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.279126  1709 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:21.277307  1674 leader_election.cc:343] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.281344  1749 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.281663  1749 ts_tablet_manager.cc:1403] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.027s
I20260709 13:00:21.282956  1751 tablet_bootstrap.cc:654] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.283071  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.280885  1872 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.284819  1708 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.284936  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:21.286146  1525 leader_election.cc:343] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.287484  1749 raft_consensus.cc:359] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.288160  1749 raft_consensus.cc:385] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.288522  1749 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.289111  1749 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.287679  1457 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.293147  1749 ts_tablet_manager.cc:1434] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:00:21.299237  1756 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:21.306368  1749 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.307474  1708 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:21.314392  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.315534  1802 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.316104  1802 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:21.318648  1802 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.322906  1851 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.315587  1709 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:21.323846  1851 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.326689  1598 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.327268  1871 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.327487  1818 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.328006  1818 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:21.327755  1871 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.328974  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.328845  1818 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.329584  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.329842  1871 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.330320  1560 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:21.330835  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:21.332561  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.333385  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:21.335319  1858 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.335824  1858 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.336263  1858 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.344830  1802 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.345363  1802 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.345955  1751 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.346381  1751 ts_tablet_manager.cc:1403] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.122s	user 0.018s	sys 0.011s
I20260709 13:00:21.348863  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.351251  1871 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.352113  1749 tablet_bootstrap.cc:654] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.351612  1871 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.353355  1851 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.354215  1802 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.355374  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.355477  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.356609  1871 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.356791  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
W20260709 13:00:21.356637  1525 leader_election.cc:343] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.365859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:00:21.367188  1523 leader_election.cc:343] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.367591  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.368752  1802 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.369335  1750 tablet_bootstrap.cc:492] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.369827  1750 ts_tablet_manager.cc:1403] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.172s	user 0.020s	sys 0.006s
I20260709 13:00:21.369598  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.369483  1858 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.370690  1707 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:21.372617  1851 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.378695  1851 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.379714  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.380518  1635 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.382072  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:21.384104  1674 leader_election.cc:343] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.384667  1851 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.375926  1858 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.374965  1785 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:00:21.387658  1457 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.395506  1750 raft_consensus.cc:359] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.396243  1750 raft_consensus.cc:385] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.396518  1750 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.391188  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.397179  1750 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.403302  1750 ts_tablet_manager.cc:1434] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
W20260709 13:00:21.405269  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:21.406126  1750 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.397305  1557 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.387835  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.410816  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.413796  1751 raft_consensus.cc:359] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.414480  1751 raft_consensus.cc:385] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.414785  1751 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.414660  1871 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.417675  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.418313  1851 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.415560  1751 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.421665  1560 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:21.422334  1851 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.422770  1851 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.423094  1871 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.423025  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.431531  1707 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:21.432576  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.435328  1871 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.438817  1751 ts_tablet_manager.cc:1434] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.001s
I20260709 13:00:21.438699  1851 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.440120  1751 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.442577  1750 tablet_bootstrap.cc:654] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.442093  1456 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.444259  1871 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.444833  1871 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.445884  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:21.447672  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:21.450343  1851 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.450866  1851 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.451465  1851 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:21.452265  1851 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.452904  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.453449  1558 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.457332  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.461686  1871 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.462177  1871 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.464200  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.464798  1709 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:21.465516  1802 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.466377  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:21.467944  1525 leader_election.cc:343] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:21.468926  1674 leader_election.cc:343] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.469090  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.470711  1802 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.471107  1802 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.471503  1802 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.471431  1751 tablet_bootstrap.cc:654] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.485133  1785 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.485406  1750 tablet_bootstrap.cc:492] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.486017  1750 ts_tablet_manager.cc:1403] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.002s
I20260709 13:00:21.485880  1785 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.527055  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.528033  1557 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:21.529104  1457 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.513644  1802 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.533839  1802 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.534406  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.535844  1750 raft_consensus.cc:359] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.536489  1750 raft_consensus.cc:385] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.536521  1558 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:21.536844  1750 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.537439  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.537869  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.538192  1707 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.540104  1858 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.537501  1750 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.540624  1858 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:21.541939  1750 ts_tablet_manager.cc:1434] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.004s
I20260709 13:00:21.541520  1858 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.542832  1750 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
W20260709 13:00:21.547626  1525 leader_election.cc:343] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.549461  1750 tablet_bootstrap.cc:654] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.552881  1707 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:21.554108  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.555302  1782 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.555842  1782 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:21.556746  1782 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.559911  1818 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.561112  1818 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.563300  1749 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.563671  1749 ts_tablet_manager.cc:1403] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.258s	user 0.013s	sys 0.031s
I20260709 13:00:21.564431  1751 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.564843  1751 ts_tablet_manager.cc:1403] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.125s	user 0.015s	sys 0.004s
I20260709 13:00:21.565600  1785 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.567557  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.567716  1751 raft_consensus.cc:359] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.568876  1751 raft_consensus.cc:385] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.569128  1751 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.569820  1751 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.563504  1818 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.573015  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.576943  1458 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.576812  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:21.578903  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.579012  1818 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.579779  1749 raft_consensus.cc:359] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.580487  1749 raft_consensus.cc:385] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.580793  1749 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.581578  1749 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.583799  1749 ts_tablet_manager.cc:1434] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:00:21.584743  1749 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.585386  1750 tablet_bootstrap.cc:492] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.585892  1750 ts_tablet_manager.cc:1403] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.004s
I20260709 13:00:21.587941  1818 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.589074  1750 raft_consensus.cc:359] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.589839  1750 raft_consensus.cc:385] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.590126  1750 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.590952  1751 ts_tablet_manager.cc:1434] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:00:21.590812  1750 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.591892  1751 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.593017  1750 ts_tablet_manager.cc:1434] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:21.596326  1750 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.599602  1456 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.600592  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:21.602864  1818 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.603984  1749 tablet_bootstrap.cc:654] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.603999  1751 tablet_bootstrap.cc:654] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.607143  1750 tablet_bootstrap.cc:654] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.613608  1635 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.614694  1749 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.615163  1749 ts_tablet_manager.cc:1403] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.006s
I20260709 13:00:21.617980  1749 raft_consensus.cc:359] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.618686  1749 raft_consensus.cc:385] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.619001  1749 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.619752  1749 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.621847  1750 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.622385  1750 ts_tablet_manager.cc:1403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.009s
I20260709 13:00:21.623226  1749 ts_tablet_manager.cc:1434] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:00:21.623353  1751 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.623895  1751 ts_tablet_manager.cc:1403] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.006s
I20260709 13:00:21.624325  1749 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.625198  1750 raft_consensus.cc:359] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.625900  1750 raft_consensus.cc:385] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.626209  1750 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.626585  1751 raft_consensus.cc:359] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.627292  1751 raft_consensus.cc:385] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.626912  1750 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.627600  1751 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.628346  1751 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.630146  1750 ts_tablet_manager.cc:1434] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:21.630530  1751 ts_tablet_manager.cc:1434] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:21.631078  1750 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.631419  1751 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.638389  1751 tablet_bootstrap.cc:654] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.641686  1750 tablet_bootstrap.cc:654] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.648360  1749 tablet_bootstrap.cc:654] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.651589  1751 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.652253  1751 ts_tablet_manager.cc:1403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:00:21.658929  1750 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.659426  1750 ts_tablet_manager.cc:1403] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.011s
I20260709 13:00:21.662592  1750 raft_consensus.cc:359] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.663322  1750 raft_consensus.cc:385] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.663618  1750 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.664338  1750 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.657174  1751 raft_consensus.cc:359] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.667779  1751 raft_consensus.cc:385] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.668072  1751 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.668779  1751 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.669438  1750 ts_tablet_manager.cc:1434] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:00:21.670423  1750 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.675987  1751 ts_tablet_manager.cc:1434] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:00:21.678212  1751 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.679574  1750 tablet_bootstrap.cc:654] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.684917  1782 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.685956  1782 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.689980  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:21.691741  1525 leader_election.cc:343] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.692051  1858 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.692550  1858 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.694610  1858 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.697196  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.697877  1708 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:21.698431  1751 tablet_bootstrap.cc:654] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.699198  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.700763  1749 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.731252  1858 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.735385  1749 ts_tablet_manager.cc:1403] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.111s	user 0.012s	sys 0.018s
I20260709 13:00:21.735272  1858 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.736927  1750 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.737375  1750 ts_tablet_manager.cc:1403] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.067s	user 0.021s	sys 0.023s
I20260709 13:00:21.738524  1749 raft_consensus.cc:359] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.738839  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.739480  1858 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.704787  1523 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.739673  1558 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.733678  1855 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.740231  1750 raft_consensus.cc:359] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.740970  1750 raft_consensus.cc:385] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.741186  1750 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.741365  1751 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.741828  1751 ts_tablet_manager.cc:1403] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.064s	user 0.007s	sys 0.003s
I20260709 13:00:21.741784  1675 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:21.741879  1750 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.742745  1855 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.743141  1855 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.743486  1855 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.744248  1750 ts_tablet_manager.cc:1434] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:21.704734  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.692577  1782 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.738904  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.745702  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.700712  1560 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.745097  1635 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:21.747576  1872 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.747992  1674 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.748955  1872 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.749351  1872 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.749987  1851 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.750418  1851 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.750787  1851 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.748275  1750 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.739207  1749 raft_consensus.cc:385] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.756526  1751 raft_consensus.cc:359] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.757182  1751 raft_consensus.cc:385] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.757441  1751 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.758026  1751 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.759554  1872 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.760031  1751 ts_tablet_manager.cc:1434] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:00:21.762420  1872 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.763245  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.765337  1632 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.755182  1749 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.768570  1750 tablet_bootstrap.cc:654] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.769286  1751 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.773280  1749 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.775571  1749 ts_tablet_manager.cc:1434] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 13:00:21.777053  1749 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.782913  1749 tablet_bootstrap.cc:654] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.784691  1872 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.785219  1872 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.786525  1855 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.787274  1872 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.789321  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:21.790720  1525 leader_election.cc:343] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.794279  1751 tablet_bootstrap.cc:654] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.796903  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.797240  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:00:21.798431  1523 leader_election.cc:343] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.798875  1523 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.800660  1872 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.801821  1806 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:21.814982  1751 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.815444  1751 ts_tablet_manager.cc:1403] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.000s
I20260709 13:00:21.818091  1855 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.820240  1851 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.818347  1632 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:21.818390  1782 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:21.820317  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.821468  1635 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.821622  1708 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:21.822937  1851 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:21.824445  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.825294  1782 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.825733  1782 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:21.826241  1750 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.826622  1750 ts_tablet_manager.cc:1403] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.079s	user 0.031s	sys 0.017s
I20260709 13:00:21.826483  1782 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.828655  1750 raft_consensus.cc:359] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.829035  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.829293  1750 raft_consensus.cc:385] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.829586  1560 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.829835  1750 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.830482  1750 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.832445  1750 ts_tablet_manager.cc:1434] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:21.833240  1750 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.835141  1635 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:21.836961  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.837666  1632 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:21.837823  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:21.838227  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:21.840296  1872 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.840752  1872 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.841746  1855 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:21.841969  1750 tablet_bootstrap.cc:654] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.842224  1855 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:21.842572  1749 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:21.843549  1749 ts_tablet_manager.cc:1403] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.067s	user 0.005s	sys 0.007s
I20260709 13:00:21.843106  1855 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.848445  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.843278  1872 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.848898  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.850205  1675 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:00:21.850976  1525 leader_election.cc:343] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.851809  1851 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:00:21.852139  1523 leader_election.cc:343] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.852183  1632 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.852681  1523 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.859910  1872 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:21.860844  1749 raft_consensus.cc:359] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.861526  1749 raft_consensus.cc:385] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.861788  1749 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.862136  1560 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:21.859926  1751 raft_consensus.cc:359] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.863595  1751 raft_consensus.cc:385] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.863921  1751 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.863984  1872 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:21.864681  1751 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.866825  1751 ts_tablet_manager.cc:1434] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.000s
I20260709 13:00:21.869235  1750 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:21.869084  1749 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.869699  1750 ts_tablet_manager.cc:1403] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.011s
I20260709 13:00:21.867753  1751 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:21.873926  1750 raft_consensus.cc:359] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.874862  1750 raft_consensus.cc:385] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:21.875213  1750 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:21.876430  1750 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.879555  1750 ts_tablet_manager.cc:1434] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:00:21.882902  1855 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.884686  1750 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:21.886572  1749 ts_tablet_manager.cc:1434] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.001s
I20260709 13:00:21.887315  1749 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:21.893334  1458 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.904153  1751 tablet_bootstrap.cc:654] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.910269  1749 tablet_bootstrap.cc:654] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.912710  1785 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.914973  1456 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.926697  1750 tablet_bootstrap.cc:654] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:21.927480  1785 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.932054  1804 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.931841  1785 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.932545  1804 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.934571  1804 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.936338  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:21.937098  1785 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.936338  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:21.938892  1560 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:00:21.939934  1525 leader_election.cc:343] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.936275  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.941838  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:21.942032  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:21.942642  1709 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:21.943106  1785 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.943861  1523 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:21.953842  1872 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:21.954237  1872 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.954579  1872 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:21.962484  1872 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.970336  1855 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.978899  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:21.978914  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:21.981429  1709 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:21.980638  1525 leader_election.cc:343] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:21.984273  1751 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:21.986552  1751 ts_tablet_manager.cc:1403] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.003s
I20260709 13:00:21.984948  1909 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:21.990394  1751 raft_consensus.cc:359] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.000041  1751 raft_consensus.cc:385] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.000437  1751 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.001379  1751 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.004133  1751 ts_tablet_manager.cc:1434] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:00:21.984527  1872 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:21.998782  1749 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:22.005646  1749 ts_tablet_manager.cc:1403] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.118s	user 0.031s	sys 0.057s
I20260709 13:00:22.009191  1749 raft_consensus.cc:359] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.010126  1749 raft_consensus.cc:385] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.010438  1749 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.011237  1749 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:21.998847  1709 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:22.013947  1523 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.014441  1749 ts_tablet_manager.cc:1434] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:00:22.015033  1871 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.015528  1871 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:21.998504  1785 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:22.016608  1751 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.016389  1871 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.013947  1855 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:00:22.022670  1851 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.024538  1751 tablet_bootstrap.cc:654] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.015671  1749 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.034605  1872 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.037166  1749 tablet_bootstrap.cc:654] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.055879  1872 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.056412  1872 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.057670  1749 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:22.058149  1749 ts_tablet_manager.cc:1403] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.000s
I20260709 13:00:22.058446  1785 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.059135  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.058919  1785 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.060963  1785 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.062054  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.061929  1749 raft_consensus.cc:359] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.062587  1749 raft_consensus.cc:385] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:22.063459  1598 leader_election.cc:343] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.065348  1751 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.065285  1785 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.065917  1751 ts_tablet_manager.cc:1403] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.008s
I20260709 13:00:22.065888  1785 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.069424  1751 raft_consensus.cc:359] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.069895  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.070307  1751 raft_consensus.cc:385] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.070714  1751 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.062842  1749 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.071805  1909 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.072309  1909 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:22.072038  1525 leader_election.cc:343] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.074944  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.075152  1909 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.075049  1751 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.072011  1749 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.076522  1785 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.084892  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.092842  1751 ts_tablet_manager.cc:1434] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:00:22.093786  1782 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.094342  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.093073  1558 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:22.076864  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.094389  1782 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.095129  1708 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:22.095799  1749 ts_tablet_manager.cc:1434] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:00:22.096555  1598 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:22.076272  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.095039  1707 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:22.098169  1785 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.098661  1785 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.098789  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.100046  1802 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.100505  1802 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.100919  1802 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.097190  1749 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.094539  1751 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.108951  1749 tablet_bootstrap.cc:654] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.109338  1802 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.093246  1872 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.115183  1632 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.125788  1802 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.099123  1785 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.120527  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.119858  1751 tablet_bootstrap.cc:654] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.127405  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:22.128126  1707 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.078413  1601 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:00:22.130313  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52028.7us
I20260709 13:00:22.118113  1560 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:22.133601  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.135845  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:22.131276  1782 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.139400  1601 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:22.140616  1818 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.141019  1818 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.141353  1818 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:22.141628  1523 leader_election.cc:343] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:22.151840  1525 leader_election.cc:343] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.152387  1525 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.153982  1750 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:22.154613  1750 ts_tablet_manager.cc:1403] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.270s	user 0.048s	sys 0.062s
I20260709 13:00:22.154986  1785 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.157232  1782 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:22.158514  1525 leader_election.cc:343] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.160243  1750 raft_consensus.cc:359] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.160787  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:22.161887  1750 raft_consensus.cc:385] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.161978  1560 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.162633  1750 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.162669  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:22.170332  1750 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.163251  1785 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.170361  1708 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.179587  1750 ts_tablet_manager.cc:1434] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.003s
I20260709 13:00:22.180572  1750 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:22.163395  1707 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:22.182156  1909 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:22.188428  1456 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:22.196254  1749 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:22.196702  1749 ts_tablet_manager.cc:1403] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.014s
I20260709 13:00:22.199474  1871 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.199615  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.200883  1782 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.201345  1782 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:22.202135  1782 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.207875  1818 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.210891  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:22.212025  1558 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.212462  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:22.213518  1818 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:22.213865  1598 leader_election.cc:343] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.215464  1560 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:22.218221  1750 tablet_bootstrap.cc:654] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.219192  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:22.224059  1818 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.225297  1818 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:22.226125  1818 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.232434  1751 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.232213  1749 raft_consensus.cc:359] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.232861  1751 ts_tablet_manager.cc:1403] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.139s	user 0.010s	sys 0.019s
I20260709 13:00:22.232895  1749 raft_consensus.cc:385] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.233278  1749 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.235803  1749 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.237238  1749 ts_tablet_manager.cc:1434] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.000s
I20260709 13:00:22.238094  1749 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.237870  1751 raft_consensus.cc:359] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.238495  1751 raft_consensus.cc:385] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.238729  1751 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.239296  1751 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.243489  1751 ts_tablet_manager.cc:1434] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:00:22.264566  1751 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.275571  1858 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:22.278034  1456 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:22.283141  1558 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:22.295720  1457 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:22.300824  1708 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:22.300899  1601 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:22.301891  1818 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.302529  1818 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:22.304857  1818 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.307435  1749 tablet_bootstrap.cc:654] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.318146  1750 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:22.318621  1750 ts_tablet_manager.cc:1403] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.138s	user 0.012s	sys 0.030s
I20260709 13:00:22.313561  1782 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.332088  1751 tablet_bootstrap.cc:654] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.335034  1750 raft_consensus.cc:359] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.336146  1750 raft_consensus.cc:385] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.336514  1750 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.337435  1750 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.342489  1750 ts_tablet_manager.cc:1434] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 13:00:22.343899  1750 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:22.359104  1858 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.365322  1804 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:22.381502  1749 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:22.382177  1749 ts_tablet_manager.cc:1403] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.144s	user 0.015s	sys 0.006s
I20260709 13:00:22.391311  1749 raft_consensus.cc:359] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.392313  1749 raft_consensus.cc:385] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.392656  1749 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.393460  1749 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.396395  1749 ts_tablet_manager.cc:1434] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:00:22.402595  1751 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.403128  1751 ts_tablet_manager.cc:1403] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.139s	user 0.023s	sys 0.030s
I20260709 13:00:22.405025  1909 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.411715  1750 tablet_bootstrap.cc:654] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.411530  1909 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.418910  1751 raft_consensus.cc:359] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.419474  1751 raft_consensus.cc:385] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.417474  1909 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.419716  1751 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.420372  1751 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.421958  1751 ts_tablet_manager.cc:1434] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260709 13:00:22.460767  1751 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.462450  1750 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:22.463733  1750 ts_tablet_manager.cc:1403] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.008s
I20260709 13:00:22.467278  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.468261  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.474301  1858 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.478881  1933 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.479730  1933 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.469149  1709 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:22.469411  1557 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:22.475339  1858 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.487082  1858 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.491951  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.498858  1749 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.503634  1750 raft_consensus.cc:359] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.504626  1750 raft_consensus.cc:385] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.504953  1750 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.504344  1457 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:22.505731  1750 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:22.503173  1674 leader_election.cc:343] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.509877  1933 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.512528  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.522104  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:22.524092  1909 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.524641  1909 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.525104  1909 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.528565  1933 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.529321  1933 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:22.547205  1674 leader_election.cc:343] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:22.553908  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72967us
I20260709 13:00:22.560864  1751 tablet_bootstrap.cc:654] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.568888  1933 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.570917  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.572009  1559 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:22.573889  1632 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.574368  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.599692  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.605703  1749 tablet_bootstrap.cc:654] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:22.606775  1674 leader_election.cc:343] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.608160  1750 ts_tablet_manager.cc:1434] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.144s	user 0.008s	sys 0.003s
I20260709 13:00:22.610070  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.611452  1559 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:22.613300  1750 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:22.617645  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:00:22.620548  1675 leader_election.cc:343] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.621025  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.622212  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.623152  1858 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.623546  1858 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.623920  1858 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.624003  1970 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:22.624699  1970 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.625013  1970 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.625336  1970 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.630621  1909 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.632942  1782 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:22.636507  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:22.637006  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:22.637638  1560 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.638047  1909 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.638819  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:22.640777  1751 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.642724  1708 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.642558  1970 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.643267  1751 ts_tablet_manager.cc:1403] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.185s	user 0.013s	sys 0.008s
I20260709 13:00:22.645488  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:22.646142  1751 raft_consensus.cc:359] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.646873  1751 raft_consensus.cc:385] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.647138  1970 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.647142  1751 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.649927  1708 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:22.650156  1858 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.651322  1598 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:22.646138  1559 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.652351  1909 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.653355  1909 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:22.654850  1909 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.657819  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:22.657945  1751 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.660624  1751 ts_tablet_manager.cc:1434] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
W20260709 13:00:22.666265  1674 leader_election.cc:343] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.671103  1818 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.671633  1818 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.674077  1560 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:22.676878  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.678553  1751 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.677950  1818 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.684618  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.686085  1598 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:22.687230  1818 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:22.687990  1818 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.692556  1559 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:22.698040  1709 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.708971  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:22.710223  1909 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.713827  1751 tablet_bootstrap.cc:654] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.714440  1802 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:22.720399  1674 leader_election.cc:343] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.721891  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.728415  1909 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.739874  1858 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.741034  1971 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.741902  1971 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:22.743103  1751 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.742880  1971 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.743638  1751 ts_tablet_manager.cc:1403] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.004s
I20260709 13:00:22.678224  1750 tablet_bootstrap.cc:654] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.752864  1751 raft_consensus.cc:359] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.754252  1751 raft_consensus.cc:385] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.754599  1751 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.755353  1751 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.745867  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:22.759644  1560 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.760202  1751 ts_tablet_manager.cc:1434] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:00:22.775732  1751 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.772943  1782 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:00:22.793680  1749 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:22.794361  1749 ts_tablet_manager.cc:1403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.297s	user 0.051s	sys 0.067s
I20260709 13:00:22.806340  1909 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.806829  1909 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.814795  1909 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.804591  1456 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:22.821270  1560 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:22.828028  1750 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:22.828397  1751 tablet_bootstrap.cc:654] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.828651  1750 ts_tablet_manager.cc:1403] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.216s	user 0.014s	sys 0.003s
I20260709 13:00:22.821501  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.831605  1749 raft_consensus.cc:359] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.832681  1749 raft_consensus.cc:385] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.832935  1749 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.833640  1749 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.834414  1632 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.835822  1749 ts_tablet_manager.cc:1434] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:00:22.837013  1749 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.838660  1750 raft_consensus.cc:359] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.839327  1750 raft_consensus.cc:385] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.839609  1750 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.841868  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.842621  1933 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:22.840281  1750 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.843364  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.843843  1933 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:22.844889  1601 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:22.844754  1933 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.849748  1818 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.850045  1909 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:22.850317  1750 ts_tablet_manager.cc:1434] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:00:22.851507  1750 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:22.861579  1750 tablet_bootstrap.cc:654] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.873461  1751 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.873991  1751 ts_tablet_manager.cc:1403] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.101s	user 0.019s	sys 0.001s
I20260709 13:00:22.878186  1751 raft_consensus.cc:359] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.878883  1751 raft_consensus.cc:385] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.879190  1751 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.879889  1751 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.882293  1751 ts_tablet_manager.cc:1434] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:00:22.883375  1751 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.889557  1751 tablet_bootstrap.cc:654] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.893534  1751 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.893615  1802 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:22.893987  1751 ts_tablet_manager.cc:1403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:00:22.896407  1751 raft_consensus.cc:359] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.914659  1751 raft_consensus.cc:385] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.915010  1751 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.915807  1751 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.917768  1751 ts_tablet_manager.cc:1434] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.000s
I20260709 13:00:22.918702  1751 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:22.921253  1749 tablet_bootstrap.cc:654] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.924484  1751 tablet_bootstrap.cc:654] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.924876  1456 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:22.927345  1456 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:22.931392  1909 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.931870  1909 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.933862  1909 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:22.934331  1750 tablet_bootstrap.cc:492] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:22.934834  1750 ts_tablet_manager.cc:1403] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.084s	user 0.023s	sys 0.027s
I20260709 13:00:22.936340  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:22.936900  1970 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.937348  1970 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.932361  1749 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
W20260709 13:00:22.938035  1598 leader_election.cc:343] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.937862  1750 raft_consensus.cc:359] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.939047  1749 ts_tablet_manager.cc:1403] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.102s	user 0.004s	sys 0.009s
I20260709 13:00:22.939324  1750 raft_consensus.cc:385] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.936576  1751 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:22.959570  1750 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.959849  1751 ts_tablet_manager.cc:1403] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.001s
I20260709 13:00:22.960465  1750 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.946162  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.962687  1750 ts_tablet_manager.cc:1434] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.011s
I20260709 13:00:22.940475  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:22.962809  1751 raft_consensus.cc:359] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.963250  1557 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:22.963670  1750 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:22.940428  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.966145  1749 raft_consensus.cc:359] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.963518  1751 raft_consensus.cc:385] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.966826  1749 raft_consensus.cc:385] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:22.939388  1970 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.967211  1749 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:22.966895  1751 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
W20260709 13:00:22.968304  1674 leader_election.cc:343] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.967928  1749 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.968304  1751 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.969111  1675 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.970188  1750 tablet_bootstrap.cc:654] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.970338  1971 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.970703  1971 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.970999  1751 ts_tablet_manager.cc:1434] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:00:22.971030  1971 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:22.973841  1751 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
W20260709 13:00:22.975574  1601 leader_election.cc:343] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.976037  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:22.978257  1858 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.978770  1858 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:22.981870  1858 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:22.982000  1751 tablet_bootstrap.cc:654] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:22.983480  1909 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:22.984738  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:22.986104  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:22.988400  1674 leader_election.cc:343] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:22.990216  1559 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:22.991672  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:22.993983  1970 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:22.996044  1749 ts_tablet_manager.cc:1434] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.002s
I20260709 13:00:22.997167  1749 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:22.995222  1970 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.999610  1970 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.006428  1971 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.009562  1751 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.010124  1751 ts_tablet_manager.cc:1403] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.009s
I20260709 13:00:23.012403  1749 tablet_bootstrap.cc:654] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.013422  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.014238  1559 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.017086  1750 tablet_bootstrap.cc:492] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.017542  1750 ts_tablet_manager.cc:1403] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.031s
I20260709 13:00:23.018013  1751 raft_consensus.cc:359] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.018743  1751 raft_consensus.cc:385] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.019049  1751 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.019853  1751 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.019974  1970 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.020375  1750 raft_consensus.cc:359] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.021040  1750 raft_consensus.cc:385] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.021313  1750 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.022382  1751 ts_tablet_manager.cc:1434] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:00:23.022716  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.023289  1560 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.023339  1750 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.026152  1750 ts_tablet_manager.cc:1434] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:23.026770  1559 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:23.028009  1675 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.028373  1749 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.028882  1749 ts_tablet_manager.cc:1403] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.002s
I20260709 13:00:23.030120  1933 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.030611  1933 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.031756  1750 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.031420  1933 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.032599  1749 raft_consensus.cc:359] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.033264  1749 raft_consensus.cc:385] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.033545  1749 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.034200  1749 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.036445  1749 ts_tablet_manager.cc:1434] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:00:23.037443  1749 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.038471  1750 tablet_bootstrap.cc:654] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.044158  1749 tablet_bootstrap.cc:654] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.046860  1751 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.047068  1560 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:23.048321  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.049086  1858 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.049546  1858 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.050338  1858 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.056511  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.057773  1751 tablet_bootstrap.cc:654] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:23.057880  1674 leader_election.cc:343] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.059057  1971 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.065042  1749 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.065749  1749 ts_tablet_manager.cc:1403] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.000s
I20260709 13:00:23.069373  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.069281  1749 raft_consensus.cc:359] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.070252  1749 raft_consensus.cc:385] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.070601  1749 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
W20260709 13:00:23.071548  1674 leader_election.cc:343] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.072162  1970 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.071450  1749 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.075975  1751 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.076397  1751 ts_tablet_manager.cc:1403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.000s
I20260709 13:00:23.077149  1750 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.077565  1750 ts_tablet_manager.cc:1403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.024s
I20260709 13:00:23.079615  1749 ts_tablet_manager.cc:1434] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:00:23.079344  1751 raft_consensus.cc:359] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.080030  1751 raft_consensus.cc:385] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.080247  1750 raft_consensus.cc:359] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.080540  1872 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.080926  1749 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.080298  1751 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.086532  1750 raft_consensus.cc:385] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.086994  1750 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.087314  1751 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.087721  1750 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.088516  1872 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.089766  1750 ts_tablet_manager.cc:1434] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 13:00:23.091203  1750 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.091485  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:23.092916  1525 leader_election.cc:343] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.093863  1872 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.094990  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.095358  1751 ts_tablet_manager.cc:1434] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:00:23.096568  1751 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
W20260709 13:00:23.096724  1523 leader_election.cc:343] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.097188  1523 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.097975  1872 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.097198  1750 tablet_bootstrap.cc:654] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.099488  1455 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.102185  1455 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.104406  1749 tablet_bootstrap.cc:654] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.110884  1751 tablet_bootstrap.cc:654] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.116951  1750 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.117452  1749 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.118221  1750 ts_tablet_manager.cc:1403] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.007s
I20260709 13:00:23.118235  1749 ts_tablet_manager.cc:1403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.010s
I20260709 13:00:23.120952  1750 raft_consensus.cc:359] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.121070  1749 raft_consensus.cc:359] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.122278  1749 raft_consensus.cc:385] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.122535  1749 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.123137  1749 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.121595  1750 raft_consensus.cc:385] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.124150  1751 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.125185  1751 ts_tablet_manager.cc:1403] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.002s
I20260709 13:00:23.126928  1749 ts_tablet_manager.cc:1434] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:00:23.124239  1750 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.128136  1749 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.127687  1750 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.130622  1750 ts_tablet_manager.cc:1434] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:00:23.131683  1750 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.133893  1909 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.132591  1751 raft_consensus.cc:359] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.134585  1751 raft_consensus.cc:385] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.139921  1751 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.139638  1750 tablet_bootstrap.cc:654] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.145088  1751 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.146834  1909 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.147324  1909 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.149358  1909 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.151264  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.152019  1560 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:23.152493  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.154765  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:23.155464  1751 ts_tablet_manager.cc:1434] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 13:00:23.156051  1818 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.159734  1751 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.165753  1818 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.170938  1750 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.171407  1750 ts_tablet_manager.cc:1403] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.009s
I20260709 13:00:23.173449  1970 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.173924  1970 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.176189  1750 raft_consensus.cc:359] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.176903  1750 raft_consensus.cc:385] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.177160  1750 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.177979  1750 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.180691  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.180226  1909 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.172088  1818 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260709 13:00:23.182190  1675 leader_election.cc:343] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.181504  1909 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.183529  1751 tablet_bootstrap.cc:654] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.184695  1909 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.185874  1970 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.186568  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.189888  1749 tablet_bootstrap.cc:654] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.190156  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.187883  1909 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.192001  1674 leader_election.cc:343] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.191741  1909 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.192437  1674 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.192740  1750 ts_tablet_manager.cc:1434] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:00:23.193747  1750 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.194614  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.195268  1560 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:23.196898  1909 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.197353  1970 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.187510  1559 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:23.211807  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.214046  1751 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.214605  1751 ts_tablet_manager.cc:1403] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.002s
I20260709 13:00:23.217712  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.218178  1751 raft_consensus.cc:359] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.219111  1751 raft_consensus.cc:385] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.219424  1751 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.220252  1751 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.222764  1751 ts_tablet_manager.cc:1434] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:00:23.223315  1750 tablet_bootstrap.cc:654] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.223877  1751 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.225284  1749 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.225768  1749 ts_tablet_manager.cc:1403] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.009s
I20260709 13:00:23.229223  1749 raft_consensus.cc:359] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.230185  1749 raft_consensus.cc:385] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.230326  1751 tablet_bootstrap.cc:654] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.233021  1749 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.235473  1750 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.234097  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:23.236128  1750 ts_tablet_manager.cc:1403] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.010s
I20260709 13:00:23.236238  1804 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.236596  2005 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.236701  1804 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.238960  1750 raft_consensus.cc:359] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.239706  1750 raft_consensus.cc:385] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.234949  1749 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.240012  1872 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.240648  1872 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.243173  1872 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.246376  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.247215  1632 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:23.248452  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.250293  1749 ts_tablet_manager.cc:1434] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:00:23.251228  1749 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.253221  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.253829  1708 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:23.254422  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:00:23.255631  1525 leader_election.cc:343] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.258339  2005 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.259685  1804 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.260673  1872 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.261176  1872 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.261618  1872 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.263319  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.265115  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.265962  1706 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:23.267714  1804 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.270963  1749 tablet_bootstrap.cc:654] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.271976  1804 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.272390  1804 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.240063  1750 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.273556  1750 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.275508  1858 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.278060  1872 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.236737  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:23.280371  1909 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.280602  1872 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.281010  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.281898  1635 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.285554  1751 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.286043  1751 ts_tablet_manager.cc:1403] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.023s
I20260709 13:00:23.288329  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:23.289005  1706 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.289163  1751 raft_consensus.cc:359] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.289852  1751 raft_consensus.cc:385] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.290140  1751 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.290755  1751 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.292903  1751 ts_tablet_manager.cc:1434] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:00:23.294386  1933 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.294806  1933 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.296741  1933 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.296764  1970 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:23.297559  1751 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.300473  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.300724  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.301079  1560 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:23.299511  2005 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.302949  1675 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.302776  2005 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.303960  1970 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.304332  1970 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.304692  1970 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.306296  1751 tablet_bootstrap.cc:654] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.306437  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.307260  1560 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:23.308929  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:23.309784  1818 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.310148  1818 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.310475  1818 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.315465  1933 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.316171  1933 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.318776  1933 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.320425  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.322165  1751 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.322643  1751 ts_tablet_manager.cc:1403] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.004s
W20260709 13:00:23.322451  1675 leader_election.cc:343] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.327557  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.328227  1634 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
W20260709 13:00:23.330542  1674 leader_election.cc:343] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.332288  2005 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.334216  1706 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:23.334975  1909 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:23.335884  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 99230.9us
I20260709 13:00:23.331658  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:23.338491  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.338910  1933 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.339092  1708 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 0.
I20260709 13:00:23.275906  1750 ts_tablet_manager.cc:1434] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.001s
I20260709 13:00:23.342407  1750 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.342607  1970 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.339493  1933 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.349021  1804 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.352995  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.353502  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.354122  1557 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.351574  1970 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:00:23.355049  1525 leader_election.cc:343] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.357852  1804 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.346833  1933 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.364161  1971 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.365496  1635 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:23.366674  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:23.368422  1802 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.368860  1802 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:23.369112  1557 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:23.370880  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.369234  1751 raft_consensus.cc:359] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.371809  1751 raft_consensus.cc:385] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.372159  1751 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.379253  1933 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.381139  1933 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.369658  1802 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.319576  1818 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.375484  1855 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.381852  1855 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.374590  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:23.383024  1818 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.383291  1709 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.383723  1855 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.386727  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.379127  1751 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.387274  1557 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.389874  1709 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:23.390820  1751 ts_tablet_manager.cc:1434] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.000s
I20260709 13:00:23.393934  1751 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.395395  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:23.396651  1709 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.374063  1750 tablet_bootstrap.cc:654] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.401171  1751 tablet_bootstrap.cc:654] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.402277  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.403807  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.404117  1709 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:23.408209  1598 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:23.408975  1818 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.409655  1818 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:23.410470  1818 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.410930  1557 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:23.414505  1750 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.414968  1750 ts_tablet_manager.cc:1403] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.000s
W20260709 13:00:23.377715  1674 leader_election.cc:343] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.415180  1751 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.415637  1751 ts_tablet_manager.cc:1403] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:00:23.416631  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.417037  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.417210  1635 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.418226  1751 raft_consensus.cc:359] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.418550  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.418864  1751 raft_consensus.cc:385] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.419145  1751 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.419847  1751 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.417644  1750 raft_consensus.cc:359] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.420362  1675 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.420717  1750 raft_consensus.cc:385] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.420992  1750 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.421099  1804 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.421545  1804 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:23.421823  1750 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.422277  1804 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.424429  1750 ts_tablet_manager.cc:1434] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:00:23.425365  1750 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.426928  1749 tablet_bootstrap.cc:492] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.427357  1749 ts_tablet_manager.cc:1403] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.176s	user 0.034s	sys 0.037s
I20260709 13:00:23.429929  1749 raft_consensus.cc:359] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.430568  1749 raft_consensus.cc:385] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.430827  1749 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.431461  1749 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.434479  1749 ts_tablet_manager.cc:1434] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:23.435698  1749 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.438885  1635 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:23.442508  2005 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.442967  2005 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.445032  2005 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.445960  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.446622  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.447477  1751 ts_tablet_manager.cc:1434] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:00:23.446650  1709 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:23.448835  1598 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:23.449194  1455 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:23.450124  1749 tablet_bootstrap.cc:654] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.450243  1785 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.452631  1750 tablet_bootstrap.cc:654] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.452580  1933 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.453195  1751 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.453213  1933 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.455413  1933 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:23.463876  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66983.3us
I20260709 13:00:23.465629  1751 tablet_bootstrap.cc:654] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:23.466212  1675 leader_election.cc:343] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:23.466898  1675 leader_election.cc:343] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.473814  1802 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.474334  1802 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.476325  1802 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.475116  1458 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:23.477649  1749 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.478185  1749 ts_tablet_manager.cc:1403] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.043s	user 0.007s	sys 0.007s
I20260709 13:00:23.481673  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.482439  1635 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:23.482790  1749 raft_consensus.cc:359] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.484726  1749 raft_consensus.cc:385] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.485100  1749 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.485922  1749 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.488178  1749 ts_tablet_manager.cc:1434] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:00:23.489336  1749 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.496009  1855 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.496526  1855 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.498625  1855 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.500120  1971 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:23.501986  1674 leader_election.cc:343] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.502470  1674 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.505379  1971 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.507539  1749 tablet_bootstrap.cc:654] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.503453  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.506693  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:23.510056  1971 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.510625  1971 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.511682  1971 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.516007  1970 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.516680  1970 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.519381  1970 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.521473  1785 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.522056  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:23.525621  1675 leader_election.cc:343] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.532243  1749 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.532833  1749 ts_tablet_manager.cc:1403] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.010s
I20260709 13:00:23.533861  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.535552  1749 raft_consensus.cc:359] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.536326  1749 raft_consensus.cc:385] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.536659  1749 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.537323  1749 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.539511  1749 ts_tablet_manager.cc:1434] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:23.550026  1970 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.550382  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.550568  1970 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.550338  1458 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:23.551703  1557 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:23.552727  1970 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.553261  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.553900  1749 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.565501  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.567636  1749 tablet_bootstrap.cc:654] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.569820  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.571168  1804 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.578052  1970 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:23.579252  1749 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.579702  1749 ts_tablet_manager.cc:1403] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:00:23.579473  1675 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.582224  1971 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.582280  1749 raft_consensus.cc:359] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.583034  1749 raft_consensus.cc:385] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.583307  1749 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.582655  1971 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.584071  1749 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:23.583997  1674 leader_election.cc:343] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.586766  1971 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.588488  1455 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.596108  1749 ts_tablet_manager.cc:1434] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:00:23.596992  2022 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.597236  1749 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:23.597493  2022 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.594586  1971 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.600293  2022 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.603729  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.605451  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.606534  1557 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:23.608103  1601 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:23.609577  2022 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.678179  2021 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.683955  1933 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:00:23.688616  2021 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.691602  2021 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.612047  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:23.692478  1556 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:23.686885  1750 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.694591  1750 ts_tablet_manager.cc:1403] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.269s	user 0.040s	sys 0.020s
I20260709 13:00:23.694887  1675 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.695737  1933 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.696134  1933 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.696491  1933 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.697332  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.697947  1558 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.707716  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.704386  1971 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.751736  1971 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.753680  1971 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:23.762223  1525 leader_election.cc:343] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.586885  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.764663  1855 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.772161  1751 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.772644  1751 ts_tablet_manager.cc:1403] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.320s	user 0.073s	sys 0.117s
I20260709 13:00:23.773566  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:23.774994  1675 leader_election.cc:343] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.775168  1751 raft_consensus.cc:359] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.775907  1751 raft_consensus.cc:385] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:23.776352  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200013us
W20260709 13:00:23.777254  1674 leader_election.cc:343] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.776239  1751 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.778277  1751 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:23.780320  1674 leader_election.cc:343] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.777608  1750 raft_consensus.cc:359] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.781996  1750 raft_consensus.cc:385] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.782063  1804 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.782110  1971 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.783810  1750 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.783064  1751 ts_tablet_manager.cc:1434] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.005s
I20260709 13:00:23.775555  1933 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.785230  1751 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:23.786371  1933 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:23.787211  1455 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.659449  2022 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.798115  1749 tablet_bootstrap.cc:654] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.787853  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:23.799598  1559 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.797071  1750 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.805104  1750 ts_tablet_manager.cc:1434] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.110s	user 0.013s	sys 0.002s
I20260709 13:00:23.806097  1750 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:23.806505  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.818337  1751 tablet_bootstrap.cc:654] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.824467  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:23.825246  1709 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:23.829114  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:23.839170  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.839933  1635 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.839499  1559 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
W20260709 13:00:23.842140  2033 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
I20260709 13:00:23.841364  1675 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:23.848696  1855 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.855823  1855 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.856864  1855 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.857388  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.867383  1558 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:23.860529  1872 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:23.872135  1872 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.872522  1872 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.875672  2022 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:23.881197  1872 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.889921  1750 tablet_bootstrap.cc:654] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:23.892438  1749 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:23.901206  1749 ts_tablet_manager.cc:1403] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.304s	user 0.018s	sys 0.002s
I20260709 13:00:23.906343  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:23.906685  1749 raft_consensus.cc:359] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.907687  1749 raft_consensus.cc:385] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:23.907943  1525 leader_election.cc:343] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.908090  1749 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.908978  1872 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:23.909034  1749 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.911257  1909 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.912129  1749 ts_tablet_manager.cc:1434] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:00:23.913231  1749 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
W20260709 13:00:23.914175  1674 leader_election.cc:343] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.917790  1751 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:23.918463  1751 ts_tablet_manager.cc:1403] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.134s	user 0.030s	sys 0.028s
I20260709 13:00:23.901963  1858 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
W20260709 13:00:23.913990  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55002.6us
I20260709 13:00:23.926843  1635 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
W20260709 13:00:23.930856  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:00:23.946223  1674 leader_election.cc:343] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:23.946688  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.947472  1675 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:23.949352  1933 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.958032  1858 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:23.958557  1858 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:23.965556  2041 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:23.968560  1751 raft_consensus.cc:359] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.969305  1751 raft_consensus.cc:385] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.969831  1750 tablet_bootstrap.cc:492] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:23.969563  1751 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.971524  1750 ts_tablet_manager.cc:1403] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.166s	user 0.019s	sys 0.020s
I20260709 13:00:23.970960  1751 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.959355  1858 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.987006  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:23.987650  1708 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:23.987960  1750 raft_consensus.cc:359] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.988816  1750 raft_consensus.cc:385] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:23.989275  1750 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:23.988583  1455 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.990068  1750 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:23.993283  2037 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:23.993148  1750 ts_tablet_manager.cc:1434] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:00:23.995875  1750 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:24.007958  1970 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:24.010274  2037 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:24.014989  1802 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:24.018949  1970 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:24.022008  1751 ts_tablet_manager.cc:1434] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.103s	user 0.005s	sys 0.003s
I20260709 13:00:24.024111  1804 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:24.031203  1749 tablet_bootstrap.cc:654] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.035669  1708 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.048666  1872 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.049182  1872 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.052856  1751 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
W20260709 13:00:24.056798  1674 consensus_peers.cc:597] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:24.075938  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.079327  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.087193  1750 tablet_bootstrap.cc:654] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.076960  1629 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.092267  2037 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.092432  1872 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.092753  2037 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.098457  1872 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:24.161192  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62226us
W20260709 13:00:24.164163  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63897.8us
I20260709 13:00:24.164347  2041 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.164889  2041 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:24.161567  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53454.5us
I20260709 13:00:24.165141  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:24.168244  1872 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:24.168928  2021 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.169342  2021 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.169776  2021 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.192428  2041 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.193758  2041 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.194280  2041 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.204533  2041 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:00:24.165710  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51421.5us
I20260709 13:00:24.213670  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.214350  1707 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.215118  1749 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.215682  1749 ts_tablet_manager.cc:1403] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.303s	user 0.020s	sys 0.028s
I20260709 13:00:24.213689  1872 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.230656  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.274259  1751 tablet_bootstrap.cc:654] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.275540  1749 raft_consensus.cc:359] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.276271  1749 raft_consensus.cc:385] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.276558  1749 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
W20260709 13:00:24.312397  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82707.3us
I20260709 13:00:24.334900  2040 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:00:24.284890  2037 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.372010  2021 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.374223  2021 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.345841  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.375793  1704 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.341315  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:24.378507  1457 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:24.385373  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:24.390825  1525 leader_election.cc:343] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.386286  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:24.402025  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.403147  1804 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.403533  1804 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.403901  1804 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.404942  1749 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.430135  1749 ts_tablet_manager.cc:1434] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.214s	user 0.016s	sys 0.003s
I20260709 13:00:24.413683  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.433720  1557 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:24.444178  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:24.445107  1709 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.450368  1749 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.453569  1750 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:24.454027  1750 ts_tablet_manager.cc:1403] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.459s	user 0.037s	sys 0.039s
I20260709 13:00:24.456467  1751 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:24.457074  1751 ts_tablet_manager.cc:1403] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.425s	user 0.049s	sys 0.052s
I20260709 13:00:24.445845  1634 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:24.469218  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:24.469813  1634 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:24.471002  1675 leader_election.cc:343] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:24.477528  1674 leader_election.cc:343] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.477919  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.479446  1933 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:24.486486  1709 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.488334  2055 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:24.489198  1750 raft_consensus.cc:359] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.490659  1750 raft_consensus.cc:385] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.490906  1750 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.492590  1750 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.495361  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:24.497344  1933 raft_consensus.cc:2764] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:00:24.499475  1749 tablet_bootstrap.cc:654] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.499886  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:24.501083  2037 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:24.502038  2037 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:24.502421  1750 ts_tablet_manager.cc:1434] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 13:00:24.503438  1750 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:24.502794  2037 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.511158  1751 raft_consensus.cc:359] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.512993  1751 raft_consensus.cc:385] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.513295  1751 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.513924  1751 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.512527  1634 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.516641  1751 ts_tablet_manager.cc:1434] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.002s
I20260709 13:00:24.517624  1751 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:24.474897  1804 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.522753  1933 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:24.533308  2040 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:24.552496  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:24.556715  1804 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:24.560052  1525 leader_election.cc:343] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.570914  1751 tablet_bootstrap.cc:654] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.566164  1458 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.578881  1458 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.600884  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:24.601466  1709 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.616326  1750 tablet_bootstrap.cc:654] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:24.619606  1674 consensus_peers.cc:597] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:24.621260  1749 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.621707  1749 ts_tablet_manager.cc:1403] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.171s	user 0.027s	sys 0.007s
I20260709 13:00:24.624079  1749 raft_consensus.cc:359] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.624706  1749 raft_consensus.cc:385] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.624951  1749 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.626449  1749 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.620293  1709 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.631503  1751 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:24.631518  2036 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:24.634065  1750 tablet_bootstrap.cc:492] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:24.635565  1749 ts_tablet_manager.cc:1434] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:00:24.636605  1749 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.636699  1751 ts_tablet_manager.cc:1403] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.119s	user 0.007s	sys 0.013s
I20260709 13:00:24.637385  1750 ts_tablet_manager.cc:1403] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.134s	user 0.016s	sys 0.002s
I20260709 13:00:24.632568  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.639218  2036 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:24.639691  2036 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:24.640516  2036 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.644511  1750 raft_consensus.cc:359] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.645203  1750 raft_consensus.cc:385] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.645422  1750 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.646023  1750 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.648115  1750 ts_tablet_manager.cc:1434] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:24.648531  1751 raft_consensus.cc:359] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.649034  1750 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:24.649217  1751 raft_consensus.cc:385] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.649557  1751 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.650585  1751 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.659278  1749 tablet_bootstrap.cc:654] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.662163  1751 ts_tablet_manager.cc:1434] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:00:24.669548  1750 tablet_bootstrap.cc:654] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:24.669988  1525 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:24.677521  2036 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.678027  2036 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.678129  1455 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.684696  1970 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.685195  1970 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.687465  2037 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.689426  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.692451  2054 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.691036  1970 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.691587  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:24.696810  1675 leader_election.cc:343] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.689532  2037 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.699388  2037 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.699553  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.700225  1709 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.692929  2054 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.701380  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:00:24.702473  1674 leader_election.cc:343] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.702955  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.704698  1933 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:24.703608  1751 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:24.708434  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:24.710633  1525 leader_election.cc:343] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.702827  2037 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.723207  2054 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.713366  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:24.726727  1525 leader_election.cc:343] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.718698  2037 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.732645  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:24.734268  2036 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.738346  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.740487  1633 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:24.741483  1749 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.736950  2037 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.742456  1749 ts_tablet_manager.cc:1403] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.106s	user 0.021s	sys 0.024s
I20260709 13:00:24.745187  1749 raft_consensus.cc:359] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.745947  1749 raft_consensus.cc:385] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.746289  1749 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
W20260709 13:00:24.738760  1675 leader_election.cc:343] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.747162  1749 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.750198  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:24.751061  2064 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.751513  2064 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.750980  1749 ts_tablet_manager.cc:1434] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:24.751922  2064 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.752140  1750 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:24.752646  1750 ts_tablet_manager.cc:1403] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.104s	user 0.031s	sys 0.013s
I20260709 13:00:24.752856  1749 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.755658  1751 tablet_bootstrap.cc:654] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.759407  1750 raft_consensus.cc:359] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.760170  1750 raft_consensus.cc:385] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.760471  1750 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.760952  1749 tablet_bootstrap.cc:654] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.738916  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.767175  1751 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:24.767422  1709 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.767640  1751 ts_tablet_manager.cc:1403] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.003s
I20260709 13:00:24.770916  1751 raft_consensus.cc:359] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.771849  1751 raft_consensus.cc:385] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.772212  1751 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.772922  1751 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.774377  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.775482  2036 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.775879  2036 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.776242  2036 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.777426  1751 ts_tablet_manager.cc:1434] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:00:24.761240  1750 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.780267  1751 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:24.792557  1750 ts_tablet_manager.cc:1434] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:00:24.802170  1750 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:24.804622  1751 tablet_bootstrap.cc:654] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.807505  2064 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.810386  1749 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.810834  1749 ts_tablet_manager.cc:1403] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.004s
I20260709 13:00:24.812919  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:24.811298  2064 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.814119  1633 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.816143  1749 raft_consensus.cc:359] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.816965  1749 raft_consensus.cc:385] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.816689  2036 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.817268  1749 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.817379  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:24.817934  1749 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:24.819641  1675 leader_election.cc:343] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.820461  1749 ts_tablet_manager.cc:1434] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:00:24.824179  2036 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.824368  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:24.825758  1525 leader_election.cc:343] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.826735  1633 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:24.828879  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:24.829573  1705 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.829691  1749 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.832950  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:00:24.839005  2064 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:24.841388  2064 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:24.842024  1705 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.842232  2064 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.843006  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.842875  2037 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.844390  2036 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:24.845466  2036 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:24.846192  2036 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.846908  1749 tablet_bootstrap.cc:654] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.851948  1750 tablet_bootstrap.cc:654] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.859601  2037 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.860533  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:24.861646  1525 leader_election.cc:343] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.863296  1751 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:24.863847  1751 ts_tablet_manager.cc:1403] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.005s
I20260709 13:00:24.863154  1455 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.863868  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:24.864713  1705 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.869012  1751 raft_consensus.cc:359] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.869725  1751 raft_consensus.cc:385] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.869998  1751 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.870736  1751 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.873170  1751 ts_tablet_manager.cc:1434] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:00:24.873534  1705 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:24.873844  1750 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:24.874171  1751 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:24.874289  1750 ts_tablet_manager.cc:1403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.072s	user 0.004s	sys 0.010s
I20260709 13:00:24.874624  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.876204  2037 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:24.876781  2037 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:24.877573  2037 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.879989  1750 raft_consensus.cc:359] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.880689  1750 raft_consensus.cc:385] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.880971  1750 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.881669  1750 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.882815  1749 tablet_bootstrap.cc:492] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.883297  1749 ts_tablet_manager.cc:1403] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.009s
I20260709 13:00:24.884667  1751 tablet_bootstrap.cc:654] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.886935  1749 raft_consensus.cc:359] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.887631  1749 raft_consensus.cc:385] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.887920  1749 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.891459  1749 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.893486  1456 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:24.900867  1749 ts_tablet_manager.cc:1434] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 13:00:24.902321  1749 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.907001  1751 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:24.907585  1751 ts_tablet_manager.cc:1403] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.012s
I20260709 13:00:24.908876  1749 tablet_bootstrap.cc:654] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.911239  1751 raft_consensus.cc:359] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.912070  1751 raft_consensus.cc:385] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.912460  2054 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.913110  2054 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.916847  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:24.917317  2054 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.912456  1751 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
W20260709 13:00:24.918260  1675 leader_election.cc:343] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.918699  1751 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.921741  1751 ts_tablet_manager.cc:1434] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:00:24.923045  1749 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:24.923452  1749 ts_tablet_manager.cc:1403] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.002s
I20260709 13:00:24.917609  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:24.926750  1674 leader_election.cc:343] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.927156  1674 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.927796  1933 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:24.928380  1749 raft_consensus.cc:359] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.930007  1749 raft_consensus.cc:385] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:24.930284  1749 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:24.930953  1749 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.932649  1749 ts_tablet_manager.cc:1434] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:00:24.933561  1749 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:24.939100  1751 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:24.944818  1751 tablet_bootstrap.cc:654] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.944778  1750 ts_tablet_manager.cc:1434] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:00:24.948125  2064 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.948585  2064 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.949119  1749 tablet_bootstrap.cc:654] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.950695  2064 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:24.952122  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:24.954191  1674 leader_election.cc:343] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.955001  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:24.955802  1557 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:24.957706  1750 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:24.957892  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:24.959239  2064 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:24.959695  2064 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.960100  2064 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.971562  1750 tablet_bootstrap.cc:654] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:24.973940  2064 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.974485  2036 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.978422  2036 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.980320  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:24.980885  1557 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.982792  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:24.982792  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.984594  1705 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:24.982959  2036 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:24.985925  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:24.986828  2036 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:24.987044  1525 leader_election.cc:343] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:24.987190  2036 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.987524  2036 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:24.994454  2036 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:24.996498  2036 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.000771  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:25.001339  1709 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.002436  1749 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.002887  1749 ts_tablet_manager.cc:1403] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.010s
I20260709 13:00:25.003561  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:25.004882  1525 leader_election.cc:343] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.005419  2036 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.006538  1749 raft_consensus.cc:359] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.007190  1749 raft_consensus.cc:385] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.007444  1749 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.008172  1749 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.010237  1749 ts_tablet_manager.cc:1434] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:00:25.011154  1749 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.016873  1557 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:25.017652  1633 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.023211  1749 tablet_bootstrap.cc:654] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.028182  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:00:25.029335  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:25.032243  2022 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.032699  2022 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.034417  2036 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.038220  2022 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.031945  1933 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.043759  1749 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.046142  1749 ts_tablet_manager.cc:1403] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.006s
I20260709 13:00:25.047562  1933 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:25.048856  1933 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.051015  1749 raft_consensus.cc:359] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.054505  1749 raft_consensus.cc:385] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.055163  1749 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.018252  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:25.056479  1749 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.021471  2064 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.057884  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:24.976743  1456 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:25.021188  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.058702  1705 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:25.058421  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.059612  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.059813  2036 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.060490  2036 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:25.061524  2036 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.064850  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:25.066769  2062 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.067998  1749 ts_tablet_manager.cc:1434] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:00:25.069576  1749 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.083608  1751 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.084200  1751 ts_tablet_manager.cc:1403] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.145s	user 0.041s	sys 0.063s
I20260709 13:00:25.083681  1749 tablet_bootstrap.cc:654] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:25.088312  1674 leader_election.cc:343] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.087735  2022 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.089774  2022 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.090663  1751 raft_consensus.cc:359] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.091300  1751 raft_consensus.cc:385] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.091563  1751 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.092172  1751 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.092695  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.094209  2022 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.095474  1751 ts_tablet_manager.cc:1434] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
W20260709 13:00:25.096275  1661 tablet.cc:2406] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:25.096560  1751 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.105051  1456 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:25.122532  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.137521  1601 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:25.138445  2022 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:25.148236  2062 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:25.192088  1455 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:25.197875  1750 tablet_bootstrap.cc:492] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.198462  1750 ts_tablet_manager.cc:1403] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.241s	user 0.016s	sys 0.008s
I20260709 13:00:25.202153  1749 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.202589  1749 ts_tablet_manager.cc:1403] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.008s
I20260709 13:00:25.209260  2021 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.209810  2021 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.198029  1872 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:25.212229  2053 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.212869  2021 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.212733  2053 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.213565  1634 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.213798  1750 raft_consensus.cc:359] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.215075  1750 raft_consensus.cc:385] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.215292  1750 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.221944  2053 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.229215  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.229992  1749 raft_consensus.cc:359] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.230836  1749 raft_consensus.cc:385] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.230330  1750 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.231196  1749 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.234591  1749 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.237787  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.236366  1559 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:25.238384  1709 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.240780  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.245703  1750 ts_tablet_manager.cc:1434] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.047s	user 0.002s	sys 0.005s
I20260709 13:00:25.254819  1749 ts_tablet_manager.cc:1434] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.002s
I20260709 13:00:25.252700  1804 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.259600  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.256286  2064 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.263943  1933 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.267616  1933 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.267983  1933 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.274215  1751 tablet_bootstrap.cc:654] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.260129  1804 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.283676  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.284487  1872 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.284945  1872 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.286901  1872 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.286798  2037 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.276270  2064 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.289904  2064 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.278475  1804 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.291911  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.291949  1750 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.297254  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.299391  1970 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:25.298691  1674 leader_election.cc:343] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.301685  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.302937  1749 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
W20260709 13:00:25.304848  1674 leader_election.cc:343] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.306035  1804 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.308708  1804 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.318831  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:25.319499  1708 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.328246  1749 tablet_bootstrap.cc:654] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:25.332136  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65341.6us
I20260709 13:00:25.292615  1705 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.335378  1523 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.335915  1750 tablet_bootstrap.cc:654] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.336333  1804 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.336741  1804 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.337134  1804 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:25.344172  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58157us
I20260709 13:00:25.345367  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:25.345957  1525 leader_election.cc:343] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.349457  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.336633  1708 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:25.351578  1558 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.352005  1933 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.354277  1933 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.354708  1751 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.355145  1751 ts_tablet_manager.cc:1403] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.259s	user 0.033s	sys 0.018s
I20260709 13:00:25.357507  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.355463  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.359315  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:25.359911  1553 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.360237  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.360787  1631 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:25.361213  1674 leader_election.cc:343] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.363382  1804 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.367110  2021 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.371968  1557 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:25.369877  2036 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:25.376168  1872 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.377785  1872 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.385543  1804 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.387849  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:25.388399  1709 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.391608  2021 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:25.391880  1872 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.395783  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.396291  1675 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.396420  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.398074  2064 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.398701  2021 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.399662  1709 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:25.400170  2054 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:25.402601  1525 leader_election.cc:343] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.371620  1553 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:25.409370  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.422865  1523 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:00:25.423132  1525 leader_election.cc:343] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.425093  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.428905  2105 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:25.430500  1525 leader_election.cc:343] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.430125  2105 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:25.432281  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.433903  1804 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.432312  2105 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.432973  1933 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.458036  1933 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:25.457963  1751 raft_consensus.cc:359] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.458738  1751 raft_consensus.cc:385] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.458998  1933 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.459017  1751 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.461831  1751 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.464092  1749 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.464661  1749 ts_tablet_manager.cc:1403] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.162s	user 0.011s	sys 0.003s
I20260709 13:00:25.465937  1751 ts_tablet_manager.cc:1434] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.002s
I20260709 13:00:25.468367  1749 raft_consensus.cc:359] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.468828  1750 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.469188  1751 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.480311  1749 raft_consensus.cc:385] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.480597  1750 ts_tablet_manager.cc:1403] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.189s	user 0.034s	sys 0.010s
I20260709 13:00:25.481594  1749 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
W20260709 13:00:25.485188  1525 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:25.486016  1749 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.490201  2100 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:25.493788  1455 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:25.497784  1457 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:25.500190  1751 tablet_bootstrap.cc:654] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.503096  1750 raft_consensus.cc:359] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.503805  1750 raft_consensus.cc:385] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.504081  1750 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.504736  1750 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.507339  1749 ts_tablet_manager.cc:1434] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.002s
I20260709 13:00:25.514163  1749 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.527985  1750 ts_tablet_manager.cc:1434] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:00:25.530810  1750 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.533385  1749 tablet_bootstrap.cc:654] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.536970  1750 tablet_bootstrap.cc:654] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.541154  1804 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.541782  1804 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.543967  2105 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:25.546622  1804 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.548873  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.549563  1706 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.550817  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.551713  2105 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.551671  1458 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:25.552407  2105 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.553542  2105 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.578197  1631 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.578685  1749 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.579195  1749 ts_tablet_manager.cc:1403] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.010s
I20260709 13:00:25.583606  1751 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.584101  1751 ts_tablet_manager.cc:1403] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.014s
I20260709 13:00:25.586649  1751 raft_consensus.cc:359] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.587256  1751 raft_consensus.cc:385] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.587508  1751 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.589293  1751 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.591379  1751 ts_tablet_manager.cc:1434] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:25.592312  1751 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.594640  1749 raft_consensus.cc:359] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.596546  1749 raft_consensus.cc:385] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.596817  1749 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.597182  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.597561  1749 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.604491  1749 ts_tablet_manager.cc:1434] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:00:25.605490  1749 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.612222  1751 tablet_bootstrap.cc:654] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.619060  1749 tablet_bootstrap.cc:654] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.623827  2105 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.624540  1751 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.627812  1751 ts_tablet_manager.cc:1403] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 13:00:25.628665  2100 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.630035  2105 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:00:25.628690  1525 leader_election.cc:343] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.633000  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:25.633581  1705 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.633755  1751 raft_consensus.cc:359] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.635473  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.637077  1751 raft_consensus.cc:385] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.637400  1751 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.638114  1751 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.640132  1751 ts_tablet_manager.cc:1434] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:25.642783  1750 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.643213  1750 ts_tablet_manager.cc:1403] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.017s
I20260709 13:00:25.646035  1750 raft_consensus.cc:359] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.646634  1750 raft_consensus.cc:385] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.646871  1750 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.647512  1750 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.649559  1750 ts_tablet_manager.cc:1434] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:00:25.650466  1750 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.656581  1751 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
W20260709 13:00:25.659271  1674 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:25.661237  1525 leader_election.cc:343] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.662382  1750 tablet_bootstrap.cc:654] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:25.662551  1525 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:25.663156  1749 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.663595  1749 ts_tablet_manager.cc:1403] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.009s
I20260709 13:00:25.665880  2053 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.666281  1751 tablet_bootstrap.cc:654] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.647514  1705 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:25.676841  1750 tablet_bootstrap.cc:492] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.677314  1750 ts_tablet_manager.cc:1403] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.005s
I20260709 13:00:25.680299  1750 raft_consensus.cc:359] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.680939  1750 raft_consensus.cc:385] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.681190  1750 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.681777  1750 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.683954  1750 ts_tablet_manager.cc:1434] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:00:25.683943  1749 raft_consensus.cc:359] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.682019  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.685047  1750 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.685009  1749 raft_consensus.cc:385] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.686307  2037 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.686797  2037 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:25.687572  2037 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.691437  1750 tablet_bootstrap.cc:654] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.685550  1749 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.693362  1749 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.696930  1749 ts_tablet_manager.cc:1434] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:00:25.698006  1749 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.703941  1751 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.705284  1751 ts_tablet_manager.cc:1403] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.000s
I20260709 13:00:25.706969  1749 tablet_bootstrap.cc:654] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.708039  1751 raft_consensus.cc:359] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.708860  1751 raft_consensus.cc:385] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.709158  1751 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.710500  1751 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.712033  1751 ts_tablet_manager.cc:1434] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:25.712960  1751 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.715164  1750 tablet_bootstrap.cc:492] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.716094  1750 ts_tablet_manager.cc:1403] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.004s
I20260709 13:00:25.719082  1751 tablet_bootstrap.cc:654] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.722129  1750 raft_consensus.cc:359] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.722995  1750 raft_consensus.cc:385] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.723341  1750 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.724162  1750 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.725970  1750 ts_tablet_manager.cc:1434] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:00:25.726894  1750 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.727849  1751 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.728277  1751 ts_tablet_manager.cc:1403] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:00:25.733093  1751 raft_consensus.cc:359] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.733716  1751 raft_consensus.cc:385] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.733961  1751 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.734609  1751 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.737695  1751 ts_tablet_manager.cc:1434] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:00:25.738602  1751 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.742774  1749 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.743217  1749 ts_tablet_manager.cc:1403] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.001s
I20260709 13:00:25.744153  1750 tablet_bootstrap.cc:654] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.745795  1749 raft_consensus.cc:359] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.746433  1749 raft_consensus.cc:385] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.746685  1749 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.760596  1751 tablet_bootstrap.cc:654] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.755784  1749 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.766242  1749 ts_tablet_manager.cc:1434] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 13:00:25.775276  1749 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.785862  1749 tablet_bootstrap.cc:654] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.785991  1458 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:25.796659  1749 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.797293  1749 ts_tablet_manager.cc:1403] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 13:00:25.800612  1749 raft_consensus.cc:359] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.801512  1749 raft_consensus.cc:385] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.801844  1749 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.804111  1751 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.804629  1751 ts_tablet_manager.cc:1403] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.000s
I20260709 13:00:25.805939  1750 tablet_bootstrap.cc:492] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.806447  1750 ts_tablet_manager.cc:1403] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.080s	user 0.034s	sys 0.031s
I20260709 13:00:25.807181  1751 raft_consensus.cc:359] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.807888  1751 raft_consensus.cc:385] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.808156  1751 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.808849  1751 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.809391  1750 raft_consensus.cc:359] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.810176  1750 raft_consensus.cc:385] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.810479  1750 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.810588  2036 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.811085  2036 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.811327  1750 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.811971  1749 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.811328  1751 ts_tablet_manager.cc:1434] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:25.813230  2036 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.813539  1751 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.814643  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.815232  1709 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.816362  1523 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.816655  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.817982  2103 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:25.818624  1525 leader_election.cc:343] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.818434  2103 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.820242  1751 tablet_bootstrap.cc:654] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.820633  2103 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.821548  1750 ts_tablet_manager.cc:1434] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:00:25.823048  1750 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.823024  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.824669  2036 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.825060  2036 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.825181  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.825413  2036 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.826680  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:25.827724  2103 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:25.830835  1750 tablet_bootstrap.cc:654] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.834833  1749 ts_tablet_manager.cc:1434] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:00:25.836182  1749 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.837051  1631 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.843098  2036 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.845207  2036 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.846252  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:25.846622  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:25.847162  1709 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.847664  1750 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.848126  1750 ts_tablet_manager.cc:1403] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.012s
W20260709 13:00:25.852173  1525 leader_election.cc:343] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.853433  1709 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:25.854372  1523 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.855289  2037 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.855909  1750 raft_consensus.cc:359] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.856561  1750 raft_consensus.cc:385] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.856827  1750 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.857424  1750 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.856580  2037 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:25.859191  2037 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:25.859462  1750 ts_tablet_manager.cc:1434] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 13:00:25.859965  2037 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.860399  1750 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.860472  1749 tablet_bootstrap.cc:654] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.867031  1751 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:25.867558  1751 ts_tablet_manager.cc:1403] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.004s
I20260709 13:00:25.868352  1750 tablet_bootstrap.cc:654] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.870487  1751 raft_consensus.cc:359] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.871142  1751 raft_consensus.cc:385] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.871264  2053 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.871428  1751 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.871703  2053 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.872104  1751 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.873716  2053 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.874078  1751 ts_tablet_manager.cc:1434] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:00:25.874958  1751 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Bootstrap starting.
I20260709 13:00:25.876405  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:25.878233  1674 leader_election.cc:343] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.884231  1633 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.885030  1751 tablet_bootstrap.cc:654] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.890192  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.890784  1557 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:25.897758  1749 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.898198  1749 ts_tablet_manager.cc:1403] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.062s	user 0.006s	sys 0.012s
I20260709 13:00:25.898932  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.899941  2099 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.900300  2099 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.900619  2099 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.900697  1749 raft_consensus.cc:359] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.901407  1749 raft_consensus.cc:385] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.901696  1749 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.903563  1749 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.905366  1970 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.905614  1749 ts_tablet_manager.cc:1434] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:25.905843  1970 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.906556  1749 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Bootstrap starting.
I20260709 13:00:25.906998  1750 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:25.907574  1750 ts_tablet_manager.cc:1403] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.047s	user 0.027s	sys 0.017s
I20260709 13:00:25.912381  1750 raft_consensus.cc:359] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.913543  1749 tablet_bootstrap.cc:654] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:25.913586  1750 raft_consensus.cc:385] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.914037  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.914099  1750 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
W20260709 13:00:25.915575  1674 leader_election.cc:343] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:25.916318  1750 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.919212  1750 ts_tablet_manager.cc:1434] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:00:25.919651  2100 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.921707  1750 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:25.921509  2100 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.923990  2100 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.924751  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.925354  1557 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:25.929104  2100 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.929482  1933 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.929716  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.929987  1933 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.930536  1706 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.933200  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:25.936543  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.935186  2103 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.938067  2103 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.939217  1933 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.937040  1970 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.940145  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.941169  2100 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.940956  1558 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 0.
I20260709 13:00:25.941939  2105 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.942374  2105 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.942921  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.941566  2100 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.944646  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:25.944329  2100 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.940783  2054 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.944345  2040 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:25.948416  2040 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.948771  2040 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.951062  1749 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: No bootstrap required, opened a new log
I20260709 13:00:25.951478  1749 ts_tablet_manager.cc:1403] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.009s
I20260709 13:00:25.946554  2054 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.952850  2054 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:25.949138  2105 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.958626  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.961439  2103 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.963265  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.974531  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.974676  2100 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.976356  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:25.978099  1749 raft_consensus.cc:359] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.978818  1749 raft_consensus.cc:385] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:25.978693  2099 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.979106  1749 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Initialized, Role: FOLLOWER
I20260709 13:00:25.979750  1749 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.981021  2037 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.981436  2037 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.981949  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:25.984409  2103 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:25.984680  1933 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.983690  2037 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.985177  1933 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:25.982080  1749 ts_tablet_manager.cc:1434] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:00:25.987735  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:25.988448  1705 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:25.988173  1933 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:25.989861  2100 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:25.990381  2100 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:25.992851  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:25.996892  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:26.000644  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:26.001504  1750 tablet_bootstrap.cc:654] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:26.000672  1751 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: No bootstrap required, opened a new log
I20260709 13:00:26.002483  1751 ts_tablet_manager.cc:1403] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent bootstrapping tablet: real 0.128s	user 0.034s	sys 0.022s
W20260709 13:00:26.005342  1674 leader_election.cc:343] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:25.992921  1525 leader_election.cc:343] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:26.017042  1525 leader_election.cc:343] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.018262  1804 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.018832  1804 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.034397  1751 raft_consensus.cc:359] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.035140  1751 raft_consensus.cc:385] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:26.035413  1751 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Initialized, Role: FOLLOWER
I20260709 13:00:26.036067  1751 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.040544  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:26.017184  2105 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:26.053508  1804 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.055828  2099 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:26.062796  2054 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.041947  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:26.066201  1525 leader_election.cc:343] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.037943  2036 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:00:26.067384  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.051477  2105 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.069069  2105 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:26.069150  1525 leader_election.cc:343] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.058990  2040 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.079162  1709 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:26.080084  2040 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:26.081151  1705 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:26.081766  2054 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:26.093500  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:00:26.094255  1674 leader_election.cc:343] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.094681  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.096141  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.096728  1704 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:26.098616  1628 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.102895  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:26.104416  2036 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:26.104928  2036 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.105319  2036 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.095674  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.110463  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:00:26.110574  1674 leader_election.cc:343] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:26.111387  1674 leader_election.cc:343] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.116391  1751 ts_tablet_manager.cc:1434] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent starting tablet: real 0.114s	user 0.012s	sys 0.014s
I20260709 13:00:26.087975  2105 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:26.118675  1525 leader_election.cc:343] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:26.126688  1674 leader_election.cc:343] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.145985  1456 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:26.168949  1635 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.144033  1634 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.172369  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.188350  2037 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:26.170773  2040 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:00:26.191574  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:00:26.197760  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:26.198874  2037 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.208446  1872 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.209035  1872 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.211483  1872 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.247170  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:26.252240  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:26.252997  1709 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:26.257733  1525 leader_election.cc:343] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.258293  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:26.259274  1557 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:26.223528  2105 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.268831  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:26.256186  2143 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.276229  2143 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.267400  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:26.295969  1709 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:26.277158  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.310781  1750 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:26.311480  1750 ts_tablet_manager.cc:1403] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.390s	user 0.017s	sys 0.009s
I20260709 13:00:26.304177  1558 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.317559  1626 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:26.343344  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46329.8us
I20260709 13:00:26.310105  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:26.284771  2036 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.350198  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:26.350731  2036 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.351194  2143 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.315270  2037 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:26.357425  1525 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:26.320874  1704 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 0.
I20260709 13:00:26.361961  1750 raft_consensus.cc:359] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.362941  1750 raft_consensus.cc:385] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:26.363317  1750 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:26.364342  1750 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.367728  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:26.371812  2037 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:26.372262  2037 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:26.373143  2037 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.350842  1554 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:00:26.419687  1557 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:26.369833  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.426017  1750 ts_tablet_manager.cc:1434] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.000s
I20260709 13:00:26.416729  1555 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:26.432544  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:00:26.436215  1525 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:26.427156  1558 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:26.436905  2054 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:26.437261  2097 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:00:26.437063  2053 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:00:26.456598  1750 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:26.471668  1601 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:26.484897  2150 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:26.485335  2150 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.485718  2150 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.492657  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.500191  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.459369  2097 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.514273  1750 tablet_bootstrap.cc:654] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:26.517637  2160 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:26.508044  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.524231  1523 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:00:26.530378  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 80741.2us
I20260709 13:00:26.530712  2037 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:26.529989  2040 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:26.543185  2037 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.574095  2037 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.556685  2040 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:00:26.602975  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58524.3us
W20260709 13:00:26.613921  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68795.2us
I20260709 13:00:26.624290  2040 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.629909  2154 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:00:26.623565  2150 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.635779  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.681341  2037 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.688230  1629 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:00:26.701938  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:26.706527  1750 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:26.707089  1750 ts_tablet_manager.cc:1403] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.251s	user 0.017s	sys 0.006s
I20260709 13:00:26.709196  2150 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.711403  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:26.719682  1750 raft_consensus.cc:359] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.720356  1750 raft_consensus.cc:385] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:26.720636  1750 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:26.721279  1750 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.722803  1750 ts_tablet_manager.cc:1434] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.002s
I20260709 13:00:26.723073  1601 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:26.725821  2158 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.727039  2037 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.733537  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:26.764612  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.767913  2144 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:26.768471  2144 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:26.769282  2144 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:26.770668  1525 leader_election.cc:343] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:26.775321  1525 leader_election.cc:343] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.766228  2097 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.781594  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:26.796514  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.791272  1704 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:26.802798  1525 leader_election.cc:343] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:26.814172  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:26.796686  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:26.817276  1705 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.820504  2143 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.821034  2143 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.825453  2143 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.826272  1750 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:26.827421  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:26.831243  2143 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.837098  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:26.863399  1630 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.849514  2143 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.887408  1750 tablet_bootstrap.cc:654] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:26.893049  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.903890  1707 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:26.885459  2143 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.904942  1704 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:26.905736  1598 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:26.906555  2143 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:26.907060  1627 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:26.910264  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:26.919677  2154 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:26.921411  1523 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.926317  2155 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:26.928576  2155 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:26.926632  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:26.929411  2155 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.930348  2143 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:26.941877  2146 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:00:26.942904  2144 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:26.943382  2144 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:26.945088  2144 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:26.946082  1525 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:26.946950  2144 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.947126  1601 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:26.944545  1705 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:00:26.951452  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:26.954391  1750 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:26.954558  2154 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:26.955067  1750 ts_tablet_manager.cc:1403] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.129s	user 0.008s	sys 0.017s
I20260709 13:00:26.956097  2154 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:26.959502  1750 raft_consensus.cc:359] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.960322  1750 raft_consensus.cc:385] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:26.960664  1750 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:26.961694  1750 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:26.964257  1750 ts_tablet_manager.cc:1434] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:26.967185  1750 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:26.978344  2036 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:00:26.999058  2154 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.000501  1458 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.004503  1458 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.015879  2103 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:27.050590  1750 tablet_bootstrap.cc:654] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:27.063534  2145 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:27.079432  1674 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:27.082432  1750 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:27.082880  1750 ts_tablet_manager.cc:1403] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.117s	user 0.011s	sys 0.012s
I20260709 13:00:27.083818  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.085093  1750 raft_consensus.cc:359] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.085632  1750 raft_consensus.cc:385] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:27.085862  1750 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:27.085363  1457 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.086483  1750 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.088289  1750 ts_tablet_manager.cc:1434] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:00:27.089071  1750 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:27.097544  1750 tablet_bootstrap.cc:654] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:27.110145  1750 tablet_bootstrap.cc:492] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:27.110598  1750 ts_tablet_manager.cc:1403] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 13:00:27.113754  1750 raft_consensus.cc:359] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.114285  1750 raft_consensus.cc:385] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:27.114559  1750 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:27.115173  1750 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.117254  1750 ts_tablet_manager.cc:1434] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:00:27.118234  1750 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:27.124607  1750 tablet_bootstrap.cc:654] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:27.130393  1458 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 0 to 1, leader changed from <none> to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.133080  1750 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:27.133566  1750 ts_tablet_manager.cc:1403] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:00:27.136335  1750 raft_consensus.cc:359] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.137130  1750 raft_consensus.cc:385] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:27.137337  1750 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:27.137936  1750 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.139952  1750 ts_tablet_manager.cc:1434] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:00:27.140951  1750 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:27.146755  2103 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.147218  2103 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.149176  2103 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:27.150696  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:27.151974  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:27.152576  1750 tablet_bootstrap.cc:654] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:27.153919  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:27.154719  2103 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:27.157461  1750 tablet_bootstrap.cc:492] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:27.157871  1750 ts_tablet_manager.cc:1403] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:00:27.160667  1750 raft_consensus.cc:359] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.161126  1750 raft_consensus.cc:385] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:27.161332  1750 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:27.161926  1750 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.163314  1750 ts_tablet_manager.cc:1434] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.002s
I20260709 13:00:27.164255  1750 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Bootstrap starting.
I20260709 13:00:27.174458  1750 tablet_bootstrap.cc:654] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:27.179478  1750 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: No bootstrap required, opened a new log
I20260709 13:00:27.179970  1750 ts_tablet_manager.cc:1403] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:00:27.183497  1750 raft_consensus.cc:359] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.184085  1750 raft_consensus.cc:385] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:27.184320  1750 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Initialized, Role: FOLLOWER
I20260709 13:00:27.184957  1750 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.186542  1750 ts_tablet_manager.cc:1434] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:00:27.195940  2159 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:27.208081  2103 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.208575  2103 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.210390  2103 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:27.210865  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:27.212442  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:27.213456  1630 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:27.216679  1598 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:27.217634  2103 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:27.222900  2159 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:27.250927  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:27.388149  1804 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:27.398336  1633 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:27.406724  2103 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.407248  2103 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.414168  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:27.416107  2103 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:27.417636  1804 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:27.419447  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:27.420940  1598 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:27.421922  2103 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:27.452191  1633 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:27.476722  2103 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.477187  2103 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.478682  2103 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:27.479285  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:27.479842  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:27.480567  1705 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:27.481494  1598 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:27.482090  2103 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:27.486743  2103 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:27.559857  2154 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:27.575901  1804 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:27.584018  2168 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.584570  2168 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:27.586628  2168 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:27.588124  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:27.588968  1704 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:00:27.591953  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:27.592847  1555 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 1: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:00:27.594211  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:00:27.595577  2168 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:27.606298  2168 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:27.690162  1633 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:28.282083  1506 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:28.313166  2159 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.313747  2159 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.316125  2159 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:28.316356  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:00:28.316988  1555 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:28.317662  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:28.318086  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:28.318246  1633 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:00:28.318758  2159 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:28.319151  2159 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.319458  2159 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.325953  2159 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.327950  2159 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:28.329876  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:28.330379  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:28.330967  1558 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.330416  1633 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.335994  1558 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:28.337174  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:28.337800  2159 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:28.338229  2159 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:28.338959  2159 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.340746  1633 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:28.348228  1457 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 0 to 2, leader changed from <none> to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:28.388020  2100 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:00:28.388579  2100 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.390326  2100 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:28.391103  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:28.391322  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:28.391741  1633 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:00:28.392004  1704 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:28.393098  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:28.393764  2100 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:28.394075  2100 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:00:28.394361  2100 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.401165  2100 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.403789  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:28.403940  2100 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:28.404508  1633 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:00:28.405365  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:28.410425  1704 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:28.411468  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:00:28.412796  2100 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:28.413304  2100 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:28.414148  2100 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:28.425114  1457 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:28.777504  1633 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:28.778916  2040 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:28.788587  1555 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:28.789978  2040 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:28.802671  1707 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:28.805455  2216 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:28.819293  1630 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:28.820628  2216 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:29.962945   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:30.041921  2248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:30.064868  2249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:30.085924   431 server_base.cc:1061] running on GCE node
W20260709 13:00:30.089241  2254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:30.093721   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:30.094194   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:30.094493   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602030094472 us; error 0 us; skew 500 ppm
I20260709 13:00:30.095335   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:30.105648   431 webserver.cc:533] Webserver started at http://127.0.107.196:39993/ using document root <none> and password file <none>
I20260709 13:00:30.106293   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:30.106521   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:30.106911   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:30.108296   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-3-root/instance:
uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
format_stamp: "Formatted at 2026-07-09 13:00:30 on dist-test-slave-kgkf"
I20260709 13:00:30.113068   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:00:30.116667  2259 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:30.117501   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:00:30.117838   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-3-root
uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
format_stamp: "Formatted at 2026-07-09 13:00:30 on dist-test-slave-kgkf"
I20260709 13:00:30.118099   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:30.141682   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:30.143082   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:30.144693   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:30.147061   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:30.147245   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:30.147548   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:30.147737   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:30.231984   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:44509
I20260709 13:00:30.232134  2321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:44509 every 8 connection(s)
I20260709 13:00:30.238822   431 auto_leader_rebalancer-test.cc:384] add a tserver: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:30.254339  2322 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37587
I20260709 13:00:30.254889  2322 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:30.255942  2322 heartbeater.cc:507] Master 127.0.107.254:37587 requested a full tablet report, sending...
I20260709 13:00:30.258249  1456 ts_manager.cc:194] Registered new tserver with Master: 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:30.260181  1456 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54540
W20260709 13:00:30.323002  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800084240 after lost signal to thread 434
W20260709 13:00:30.324523  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080021cc20 after lost signal to thread 1584
W20260709 13:00:30.325408  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002733e0 after lost signal to thread 1659
W20260709 13:00:30.326188  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800284840 after lost signal to thread 1733
I20260709 13:00:31.015031  1627 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.023109  1707 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.023963  1556 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.024745  2326 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.025671  2225 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.038209  2252 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.056056  1553 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.099563  1702 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.103708  1456 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.124610  1627 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:31.129319  1599 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 334958c5aee349629d63e898f13ba95b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:31.148149  1702 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.149111  1556 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.150676  2252 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:00:31.176492  2326 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:31.180089  1599 consensus_peers.cc:597] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e27b39f413f94e9192458b7bef0a86b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:31.197021  2336 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.199184  1553 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.207319  1702 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.227530  1457 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.264428  2322 heartbeater.cc:499] Master 127.0.107.254:37587 was elected leader, sending a full tablet report...
I20260709 13:00:31.271128  1553 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.285210  1627 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.285789  1702 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.287552  2234 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.288564  2231 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:31.301837  1524 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce44b855d6d442778079778cd0e68c34. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:31.305135  2231 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.327498  1627 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.333949  1457 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.320118  1702 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.390427  1703 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.415297  1627 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.416831  1553 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:31.418254  2240 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.419090  2250 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:31.435000  1673 consensus_peers.cc:597] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61539cc4a0c645ed9fb29927e85f3e20. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:31.440610  2246 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.442662  1553 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.449401  1629 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:31.466423  1458 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.579916  2352 ts_tablet_manager.cc:933] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:31.587872  2352 tablet_copy_client.cc:323] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:31.612887  2356 ts_tablet_manager.cc:933] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:31.622949  2356 tablet_copy_client.cc:323] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:31.637579  1645 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet e27b39f413f94e9192458b7bef0a86b7 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:31.638244  1645 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet e27b39f413f94e9192458b7bef0a86b7 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-e27b39f413f94e9192458b7bef0a86b7
I20260709 13:00:31.640094  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet 334958c5aee349629d63e898f13ba95b from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:31.640705  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet 334958c5aee349629d63e898f13ba95b from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-334958c5aee349629d63e898f13ba95b
I20260709 13:00:31.646759  1645 tablet_copy_source_session.cc:215] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:31.649102  1644 tablet_copy_source_session.cc:215] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:31.651631  2352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27b39f413f94e9192458b7bef0a86b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.671001  2352 tablet_copy_client.cc:806] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:31.671778  2352 tablet_copy_client.cc:670] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:31.682932  2356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334958c5aee349629d63e898f13ba95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.689352  2352 tablet_copy_client.cc:538] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:31.702405  2352 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:31.714206  2356 tablet_copy_client.cc:806] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:31.714867  2356 tablet_copy_client.cc:670] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:31.728591  2356 tablet_copy_client.cc:538] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:31.742959  2356 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:31.743425  2352 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:31.744941  2352 tablet_bootstrap.cc:492] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:31.746125  2352 ts_tablet_manager.cc:1403] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.044s	user 0.030s	sys 0.006s
I20260709 13:00:31.749887  2352 raft_consensus.cc:359] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.750782  2352 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:31.751483  2352 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.762521  2352 ts_tablet_manager.cc:1434] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.006s
I20260709 13:00:31.769114  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-e27b39f413f94e9192458b7bef0a86b7 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:31.769598  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-e27b39f413f94e9192458b7bef0a86b7 on tablet e27b39f413f94e9192458b7bef0a86b7 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:31.777120  2352 ts_tablet_manager.cc:933] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:31.797133  2356 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:31.799582  2356 tablet_bootstrap.cc:492] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:31.800277  2356 ts_tablet_manager.cc:1403] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.007s
I20260709 13:00:31.801512  2352 tablet_copy_client.cc:323] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:31.815444  2356 raft_consensus.cc:359] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.819671  2356 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:31.823729  2356 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:31.832146  2356 ts_tablet_manager.cc:1434] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.032s	user 0.001s	sys 0.007s
I20260709 13:00:31.833855  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-334958c5aee349629d63e898f13ba95b received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:31.834285  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-334958c5aee349629d63e898f13ba95b on tablet 334958c5aee349629d63e898f13ba95b with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:31.853194  2356 ts_tablet_manager.cc:933] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:31.858387  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet ce44b855d6d442778079778cd0e68c34 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:31.858991  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet ce44b855d6d442778079778cd0e68c34 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-ce44b855d6d442778079778cd0e68c34
I20260709 13:00:31.873287  1570 tablet_copy_source_session.cc:215] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:31.880905  2352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce44b855d6d442778079778cd0e68c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.881574  2356 tablet_copy_client.cc:323] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:31.903035  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 61539cc4a0c645ed9fb29927e85f3e20 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:31.903625  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 61539cc4a0c645ed9fb29927e85f3e20 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-61539cc4a0c645ed9fb29927e85f3e20
I20260709 13:00:31.919965  2352 tablet_copy_client.cc:806] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:31.921016  2352 tablet_copy_client.cc:670] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:31.921460  1719 tablet_copy_source_session.cc:215] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:31.925580  2352 tablet_copy_client.cc:538] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:31.926995  2356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61539cc4a0c645ed9fb29927e85f3e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.936295  2352 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:31.956064  2356 tablet_copy_client.cc:806] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:31.956660  2356 tablet_copy_client.cc:670] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:31.964951  2356 tablet_copy_client.cc:538] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:31.993721  2356 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:32.003156  2352 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:32.012710  2352 tablet_bootstrap.cc:492] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:32.013576  2352 ts_tablet_manager.cc:1403] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.010s
I20260709 13:00:32.016793  2352 raft_consensus.cc:359] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:32.020867  2352 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:32.024349  2352 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:32.029685  2352 ts_tablet_manager.cc:1434] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:00:32.033977  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-ce44b855d6d442778079778cd0e68c34 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:32.034430  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-ce44b855d6d442778079778cd0e68c34 on tablet ce44b855d6d442778079778cd0e68c34 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:32.053285  2356 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:32.054289  2356 tablet_bootstrap.cc:492] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:32.055019  2356 ts_tablet_manager.cc:1403] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.011s
I20260709 13:00:32.057858  2356 raft_consensus.cc:359] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:32.058737  2356 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:32.059423  2356 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:32.064399  2356 ts_tablet_manager.cc:1434] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:32.066040  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-61539cc4a0c645ed9fb29927e85f3e20 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:32.066483  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-61539cc4a0c645ed9fb29927e85f3e20 on tablet 61539cc4a0c645ed9fb29927e85f3e20 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:32.138106  2297 raft_consensus.cc:1217] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:32.217166  2297 raft_consensus.cc:1217] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:32.288295  2297 raft_consensus.cc:1217] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:32.297646  2297 raft_consensus.cc:1217] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:32.625243  2368 raft_consensus.cc:1064] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:32.669896  2368 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:32.687557  2252 raft_consensus.cc:1064] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:32.702956  2252 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:32.743696  1703 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.744268  2297 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.747363  2336 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:32.748467  1703 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.749553  2336 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:32.757449  2235 raft_consensus.cc:1064] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:32.764025  2252 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:00:32.778589  1556 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.773324  2235 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:32.797621  2376 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:00:32.815640  1556 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.820298  2297 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.822880  2252 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:32.825011  2336 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:32.832963  2377 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.843886  2296 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.880923  1556 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.893191  1707 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.900391  2295 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.900838  1627 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.902876  1703 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:32.904004  2373 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:32.910980  1804 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:32.911900  2234 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:32.910152  2368 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.924753  2246 raft_consensus.cc:1064] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:32.927477  2246 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:32.914996  2297 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:32.977411  1456 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:32.987592  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58406.8us
I20260709 13:00:33.026119  1703 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.069454  1629 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:33.072304  2295 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.111917  2392 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:00:33.070878  2373 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:00:33.142409  2390 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.058s	user 0.003s	sys 0.000s
I20260709 13:00:33.290958  1458 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.312520  2295 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:33.315310  1457 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.305006  1553 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.327217  2250 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:33.367863  1554 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:33.374468  1633 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:33.434840  2240 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:33.471964  1704 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.475191  1631 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.486800  2377 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.506822  1704 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.526201  2297 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.547636  1554 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:33.562863  2380 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.584785  1627 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:33.600191  2295 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.600936  2377 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.604633  2377 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.607857  2295 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.597007  1702 raft_consensus.cc:2955] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.636288  2296 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.656405  1554 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.660084  2406 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:33.658219  2376 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.662175  1457 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.674616  1628 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.678062  1700 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.667730  2370 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.690372  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e27b39f413f94e9192458b7bef0a86b7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:33.692255  2406 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.697328  2252 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.698951  2296 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.716536  2297 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.719472  1457 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.731752  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 334958c5aee349629d63e898f13ba95b with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:33.749727  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ce44b855d6d442778079778cd0e68c34 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:33.734740  2373 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:00:33.759881  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 334958c5aee349629d63e898f13ba95b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:33.766636  1555 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.769143  1556 raft_consensus.cc:2955] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.773217  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet e27b39f413f94e9192458b7bef0a86b7 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:33.782455  1458 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:33.791743  2295 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.794646  1702 raft_consensus.cc:2955] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.799463  2413 tablet_replica.cc:333] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:33.809195  2413 raft_consensus.cc:2243] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:33.811040  2413 raft_consensus.cc:2272] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:33.811327  1456 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:33.833949  1704 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:33.840884  1628 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.840946  2295 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:33.842618  2370 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.843549  2246 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.845659  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet ce44b855d6d442778079778cd0e68c34 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:33.849321  2413 ts_tablet_manager.cc:1916] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:33.876389  2246 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.882403  1628 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.887853  2297 raft_consensus.cc:2955] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.894603  2414 tablet_replica.cc:333] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:33.898118  2415 tablet_replica.cc:333] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:33.899735  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 61539cc4a0c645ed9fb29927e85f3e20 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:33.919313  2415 raft_consensus.cc:2243] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:33.923247  1457 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:33.895738  2414 raft_consensus.cc:2243] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:33.928709  2414 raft_consensus.cc:2272] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:33.929776  2415 raft_consensus.cc:2272] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:33.943893  2415 ts_tablet_manager.cc:1916] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:33.956110  2413 ts_tablet_manager.cc:1929] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:33.956849  2413 log.cc:1199] T 334958c5aee349629d63e898f13ba95b P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/334958c5aee349629d63e898f13ba95b
I20260709 13:00:33.959429  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 334958c5aee349629d63e898f13ba95b (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:33.971617  2415 ts_tablet_manager.cc:1929] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:33.972254  2415 log.cc:1199] T ce44b855d6d442778079778cd0e68c34 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/ce44b855d6d442778079778cd0e68c34
I20260709 13:00:33.974144  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet ce44b855d6d442778079778cd0e68c34 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:33.979315  2414 ts_tablet_manager.cc:1916] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:34.002379  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 61539cc4a0c645ed9fb29927e85f3e20 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:34.014274  2414 ts_tablet_manager.cc:1929] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:34.014798  2414 log.cc:1199] T e27b39f413f94e9192458b7bef0a86b7 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/e27b39f413f94e9192458b7bef0a86b7
I20260709 13:00:34.017702  2414 tablet_replica.cc:333] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:34.017637  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet e27b39f413f94e9192458b7bef0a86b7 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:34.019722  2414 raft_consensus.cc:2243] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:34.020586  2414 raft_consensus.cc:2272] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:34.023947  2414 ts_tablet_manager.cc:1916] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:34.044368  2414 ts_tablet_manager.cc:1929] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:34.044785  2414 log.cc:1199] T 61539cc4a0c645ed9fb29927e85f3e20 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/61539cc4a0c645ed9fb29927e85f3e20
I20260709 13:00:34.046304  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 61539cc4a0c645ed9fb29927e85f3e20 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:00:35.276012  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64038.3us
I20260709 13:00:35.287683  1554 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:35.308755  1524 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9df64a488003414aac04146e56487c81. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:35.310271  1631 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.313474  2409 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.330835  1703 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.337934  2409 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.339502  2406 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.342779  1632 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.357347  1554 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:35.363204  1457 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.366775  1703 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
W20260709 13:00:35.403874  1524 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c25c5b342d344f3ac325ab5073d1747. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:35.409065  1632 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.409111  1703 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.411072  2409 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.412914  2409 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.445118  2420 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.447419  1632 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.447219  1704 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.463737  1554 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:35.479846  1457 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
W20260709 13:00:35.502110  1524 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcd4e3d43c7d4c97a3f229082334515f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:35.502748  1704 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.505686  1632 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.505473  2420 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.507198  2409 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:35.520684  2409 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.522104  1703 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.524121  1632 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.538800  1456 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.541901  1704 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:35.564055  1673 consensus_peers.cc:597] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 288b67d6ad124d2aa18d641ed4e1f3df. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:35.564419  1554 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.566133  2423 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.567366  1632 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:35.568706  2417 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.576315  2246 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.579226  1554 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.585562  1632 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:35.606561  1458 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.856509  2444 ts_tablet_manager.cc:933] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:35.868196  2444 tablet_copy_client.cc:323] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:35.870749  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 9df64a488003414aac04146e56487c81 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:35.871384  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 9df64a488003414aac04146e56487c81 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9df64a488003414aac04146e56487c81
I20260709 13:00:35.894450  1570 tablet_copy_source_session.cc:215] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:35.898689  2444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df64a488003414aac04146e56487c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:35.913569  2446 ts_tablet_manager.cc:933] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:35.916219  2446 tablet_copy_client.cc:323] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:35.918020  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet bcd4e3d43c7d4c97a3f229082334515f from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:35.918622  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet bcd4e3d43c7d4c97a3f229082334515f from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-bcd4e3d43c7d4c97a3f229082334515f
I20260709 13:00:35.918809  2444 tablet_copy_client.cc:806] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:35.919411  2444 tablet_copy_client.cc:670] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:35.923111  2444 tablet_copy_client.cc:538] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:35.932214  2444 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:35.933187  1570 tablet_copy_source_session.cc:215] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:35.936757  2446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd4e3d43c7d4c97a3f229082334515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:35.941346  2448 ts_tablet_manager.cc:933] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:35.950301  2448 tablet_copy_client.cc:323] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:35.952450  2449 ts_tablet_manager.cc:933] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:35.953531  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 9c25c5b342d344f3ac325ab5073d1747 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:35.955478  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 9c25c5b342d344f3ac325ab5073d1747 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9c25c5b342d344f3ac325ab5073d1747
I20260709 13:00:35.962636  1570 tablet_copy_source_session.cc:215] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:35.965658  2448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c25c5b342d344f3ac325ab5073d1747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:35.974066  2446 tablet_copy_client.cc:806] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:35.974727  2446 tablet_copy_client.cc:670] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:35.979733  2446 tablet_copy_client.cc:538] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:35.981165  2448 tablet_copy_client.cc:806] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:35.981792  2448 tablet_copy_client.cc:670] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:35.990748  2448 tablet_copy_client.cc:538] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:35.990895  2449 tablet_copy_client.cc:323] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:35.991637  2446 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:35.993121  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 288b67d6ad124d2aa18d641ed4e1f3df from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:35.993716  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 288b67d6ad124d2aa18d641ed4e1f3df from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-288b67d6ad124d2aa18d641ed4e1f3df
I20260709 13:00:36.001556  1719 tablet_copy_source_session.cc:215] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:35.993170  2444 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:36.005438  2444 tablet_bootstrap.cc:492] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:36.006125  2448 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:36.006157  2444 ts_tablet_manager.cc:1403] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.007s
I20260709 13:00:36.019114  2444 raft_consensus.cc:359] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.020035  2444 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:36.020954  2444 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.025290  2449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 288b67d6ad124d2aa18d641ed4e1f3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:36.031802  2444 ts_tablet_manager.cc:1434] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:00:36.034178  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9df64a488003414aac04146e56487c81 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:36.034669  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9df64a488003414aac04146e56487c81 on tablet 9df64a488003414aac04146e56487c81 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:36.041967  2446 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:36.042940  2446 tablet_bootstrap.cc:492] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:36.043613  2446 ts_tablet_manager.cc:1403] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.011s
I20260709 13:00:36.046536  2446 raft_consensus.cc:359] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.047339  2446 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:36.048004  2446 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.050529  2446 ts_tablet_manager.cc:1434] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:36.052202  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-bcd4e3d43c7d4c97a3f229082334515f received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:36.052781  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-bcd4e3d43c7d4c97a3f229082334515f on tablet bcd4e3d43c7d4c97a3f229082334515f with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:36.059933  2449 tablet_copy_client.cc:806] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:36.060602  2449 tablet_copy_client.cc:670] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:36.062213  2448 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:36.062992  2448 tablet_bootstrap.cc:492] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:36.063573  2448 ts_tablet_manager.cc:1403] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.017s
I20260709 13:00:36.064877  2449 tablet_copy_client.cc:538] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:36.066057  2448 raft_consensus.cc:359] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.066813  2448 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:36.067442  2448 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.071918  2448 ts_tablet_manager.cc:1434] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:36.076984  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9c25c5b342d344f3ac325ab5073d1747 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:36.077448  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9c25c5b342d344f3ac325ab5073d1747 on tablet 9c25c5b342d344f3ac325ab5073d1747 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:36.084388  2449 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:36.109238  2449 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:36.110015  2449 tablet_bootstrap.cc:492] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:36.110610  2449 ts_tablet_manager.cc:1403] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.002s
I20260709 13:00:36.113440  2449 raft_consensus.cc:359] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.114281  2449 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:36.115592  2449 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:36.121366  2449 ts_tablet_manager.cc:1434] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:00:36.123399  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-288b67d6ad124d2aa18d641ed4e1f3df received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:36.123971  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-288b67d6ad124d2aa18d641ed4e1f3df on tablet 288b67d6ad124d2aa18d641ed4e1f3df with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:36.231783  2295 raft_consensus.cc:1217] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:36.411610  2295 raft_consensus.cc:1217] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:36.506479  2295 raft_consensus.cc:1217] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:36.533978  2295 raft_consensus.cc:1217] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:36.703476  2406 raft_consensus.cc:1064] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:36.705449  2406 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:36.713505  2295 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.714946  1703 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.715204  1627 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.715057  2406 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.716665  2409 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.717489  2416 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.730509  2416 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.742700  1627 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.781909  1456 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.785180  1703 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.791326  2295 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.827143  1554 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:36.831385  2295 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:36.831965  1631 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:36.833369  2464 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:36.835424  2464 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:36.842993  2406 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.849663  2295 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.852694  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9df64a488003414aac04146e56487c81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:36.857154  1631 raft_consensus.cc:2955] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.875703  1458 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:36.925455  2419 raft_consensus.cc:1064] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:36.930725  2419 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:36.940091  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 9df64a488003414aac04146e56487c81 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:36.947536  2417 raft_consensus.cc:1064] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:36.951089  2417 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:36.963204  2470 tablet_replica.cc:333] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:36.967598  2295 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.967787  1632 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.967972  1554 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.969741  2443 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.970847  2417 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.973207  2462 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.977043  2297 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.980688  1631 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.983843  2419 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.985805  2416 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.988907  1702 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:36.997692  2464 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:37.006330  2416 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.007516  2470 raft_consensus.cc:2243] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:37.008355  2470 raft_consensus.cc:2272] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:37.011934  2470 ts_tablet_manager.cc:1916] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:37.017169  1702 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.017169  2297 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.021541  1627 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.028606  2462 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.032145  1457 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:37.042804  1627 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.055222  1554 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:37.059556  2409 raft_consensus.cc:1064] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:37.062536  2409 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:37.070783  1631 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:37.072026  2296 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:37.070802  1559 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.077333  1455 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:37.078685  2406 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.079825  2464 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.062261  2295 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.087970  2470 ts_tablet_manager.cc:1929] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:37.088507  2470 log.cc:1199] T 9df64a488003414aac04146e56487c81 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/9df64a488003414aac04146e56487c81
I20260709 13:00:37.090193  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 9df64a488003414aac04146e56487c81 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:37.092824  2419 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.096966  1632 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.107343  1702 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:37.108227  2296 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:37.111835  2406 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:37.116557  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bcd4e3d43c7d4c97a3f229082334515f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:37.115931  2296 raft_consensus.cc:2955] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.125128  1627 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:37.123060  2419 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:37.135002  1458 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.149165  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet bcd4e3d43c7d4c97a3f229082334515f with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:37.150133  2470 tablet_replica.cc:333] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:37.150322  2464 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:37.151221  2470 raft_consensus.cc:2243] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:37.152009  2470 raft_consensus.cc:2272] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:37.155040  2470 ts_tablet_manager.cc:1916] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:37.156193  1702 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:37.166729  1631 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:00:37.167618  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47797.6us
I20260709 13:00:37.169278  2462 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.175340  2297 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:37.179563  2462 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.182636  1628 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.184175  2417 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.184378  2470 ts_tablet_manager.cc:1929] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:37.187158  2470 log.cc:1199] T bcd4e3d43c7d4c97a3f229082334515f P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/bcd4e3d43c7d4c97a3f229082334515f
I20260709 13:00:37.188738  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet bcd4e3d43c7d4c97a3f229082334515f (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:37.193305  2297 raft_consensus.cc:2955] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.207674  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 288b67d6ad124d2aa18d641ed4e1f3df with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:37.217625  2297 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.218523  1627 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.222337  2419 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.224359  1455 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.258534  1703 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.281032  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 288b67d6ad124d2aa18d641ed4e1f3df with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:37.274477  1456 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:37.291178  2484 tablet_replica.cc:333] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:37.292351  2484 raft_consensus.cc:2243] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:37.294143  2484 raft_consensus.cc:2272] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:37.298707  2484 ts_tablet_manager.cc:1916] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:37.299057  1559 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:37.311973  2297 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:37.313552  2409 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.313365  1703 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:37.328560  2464 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.334091  2297 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.336728  2484 ts_tablet_manager.cc:1929] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:37.337319  2484 log.cc:1199] T 288b67d6ad124d2aa18d641ed4e1f3df P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/288b67d6ad124d2aa18d641ed4e1f3df
I20260709 13:00:37.350652  2419 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.351833  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 288b67d6ad124d2aa18d641ed4e1f3df (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:37.360016  1703 raft_consensus.cc:2955] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.363735  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9c25c5b342d344f3ac325ab5073d1747 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:37.363490  1455 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:37.375690  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c25c5b342d344f3ac325ab5073d1747 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:37.377522  2485 tablet_replica.cc:333] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:37.378832  2485 raft_consensus.cc:2243] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:37.379506  2485 raft_consensus.cc:2272] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:37.383723  2485 ts_tablet_manager.cc:1916] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:37.401464  2485 ts_tablet_manager.cc:1929] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:37.401821  2485 log.cc:1199] T 9c25c5b342d344f3ac325ab5073d1747 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/9c25c5b342d344f3ac325ab5073d1747
I20260709 13:00:37.403259  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 9c25c5b342d344f3ac325ab5073d1747 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:38.455729  1559 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:38.462301  1524 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78699c8b654944aa9788917e2a177d14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.462699  1702 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:38.464779  1627 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:38.463893  2464 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.466250  2486 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.474018  2486 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.476117  1627 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.488262  1702 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:38.488451  1703 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.494704  1457 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.498589  1554 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:38.499830  1673 consensus_peers.cc:597] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41b83284fac147339ac4124c2eedee11. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.499982  1627 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:38.501272  2463 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.515743  2463 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:00:38.527875  2463 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.530094  1554 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.530114  1627 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.541502  1559 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:38.548261  1703 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:38.548662  1524 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dea1928bdd634efaa34550f21ff564f3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.549817  1627 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:38.549939  2464 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.551550  2419 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.567914  2486 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.578930  1702 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.582176  1458 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.608578  1458 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.609050  1559 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:38.608925  1632 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.619582  1628 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:00:38.619711  1524 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de80dd9c151b41e88e1469985d074b81. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.621261  2419 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:38.626430  1702 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:38.630625  2503 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:38.635990  2486 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.637773  1632 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.655956  1458 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.658046  1703 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:38.950706  2507 ts_tablet_manager.cc:933] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:38.952661  2507 tablet_copy_client.cc:323] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:38.958220  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 41b83284fac147339ac4124c2eedee11 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:38.958741  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 41b83284fac147339ac4124c2eedee11 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-41b83284fac147339ac4124c2eedee11
I20260709 13:00:38.982164  1719 tablet_copy_source_session.cc:215] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:38.987041  2507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b83284fac147339ac4124c2eedee11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:39.002890  2507 tablet_copy_client.cc:806] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:39.003472  2507 tablet_copy_client.cc:670] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:39.007555  2507 tablet_copy_client.cc:538] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:39.009297  2510 ts_tablet_manager.cc:933] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:39.011356  2510 tablet_copy_client.cc:323] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:39.013087  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 78699c8b654944aa9788917e2a177d14 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:39.013631  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 78699c8b654944aa9788917e2a177d14 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-78699c8b654944aa9788917e2a177d14
I20260709 13:00:39.015977  2507 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:39.021040  1570 tablet_copy_source_session.cc:215] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:39.035501  2513 ts_tablet_manager.cc:933] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:39.039724  2510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78699c8b654944aa9788917e2a177d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:39.079600  2515 ts_tablet_manager.cc:933] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:39.101467  2513 tablet_copy_client.cc:323] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:39.103240  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet dea1928bdd634efaa34550f21ff564f3 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:39.103828  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet dea1928bdd634efaa34550f21ff564f3 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-dea1928bdd634efaa34550f21ff564f3
I20260709 13:00:39.112108  1570 tablet_copy_source_session.cc:215] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:39.112259  2515 tablet_copy_client.cc:323] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:39.114097  2510 tablet_copy_client.cc:806] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:39.115964  2510 tablet_copy_client.cc:670] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:39.116158  2513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea1928bdd634efaa34550f21ff564f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:39.115145  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet de80dd9c151b41e88e1469985d074b81 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:39.118062  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet de80dd9c151b41e88e1469985d074b81 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-de80dd9c151b41e88e1469985d074b81
I20260709 13:00:39.120260  2510 tablet_copy_client.cc:538] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:39.122833  2507 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:39.124534  2507 tablet_bootstrap.cc:492] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:39.125216  2507 ts_tablet_manager.cc:1403] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.109s	user 0.042s	sys 0.033s
I20260709 13:00:39.127523  1570 tablet_copy_source_session.cc:215] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:39.128208  2507 raft_consensus.cc:359] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.129132  2507 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:39.129876  2507 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.130656  2515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de80dd9c151b41e88e1469985d074b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:39.140239  2507 ts_tablet_manager.cc:1434] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.012s
I20260709 13:00:39.142227  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-41b83284fac147339ac4124c2eedee11 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:39.142688  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-41b83284fac147339ac4124c2eedee11 on tablet 41b83284fac147339ac4124c2eedee11 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:39.146703  2510 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:39.156944  2513 tablet_copy_client.cc:806] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:39.157613  2513 tablet_copy_client.cc:670] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:39.162596  2513 tablet_copy_client.cc:538] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:39.176594  2513 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:39.178512  2515 tablet_copy_client.cc:806] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:39.179414  2515 tablet_copy_client.cc:670] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:39.185767  2515 tablet_copy_client.cc:538] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:39.193485  2510 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:39.194478  2510 tablet_bootstrap.cc:492] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:39.195173  2510 ts_tablet_manager.cc:1403] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.050s	user 0.029s	sys 0.011s
I20260709 13:00:39.203787  2515 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:39.207139  2510 raft_consensus.cc:359] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.210489  2510 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:39.211190  2510 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.213626  2510 ts_tablet_manager.cc:1434] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.005s
I20260709 13:00:39.217341  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-78699c8b654944aa9788917e2a177d14 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:39.217757  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-78699c8b654944aa9788917e2a177d14 on tablet 78699c8b654944aa9788917e2a177d14 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:39.223605  2513 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:39.233073  2513 tablet_bootstrap.cc:492] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:39.233839  2513 ts_tablet_manager.cc:1403] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.014s
I20260709 13:00:39.238516  2513 raft_consensus.cc:359] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.250062  2513 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:39.250752  2513 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.253845  2513 ts_tablet_manager.cc:1434] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:00:39.260192  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-dea1928bdd634efaa34550f21ff564f3 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:39.260610  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-dea1928bdd634efaa34550f21ff564f3 on tablet dea1928bdd634efaa34550f21ff564f3 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:39.267524  2515 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:39.268492  2515 tablet_bootstrap.cc:492] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:39.269207  2515 ts_tablet_manager.cc:1403] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.074s	user 0.033s	sys 0.017s
I20260709 13:00:39.272248  2515 raft_consensus.cc:359] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.273159  2515 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:39.273850  2515 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:39.276568  2515 ts_tablet_manager.cc:1434] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:39.282245  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-de80dd9c151b41e88e1469985d074b81 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:39.282693  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-de80dd9c151b41e88e1469985d074b81 on tablet de80dd9c151b41e88e1469985d074b81 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:39.360381  2295 raft_consensus.cc:1217] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:39.451162  2295 raft_consensus.cc:1217] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:39.478788  2295 raft_consensus.cc:1217] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:39.625710  2295 raft_consensus.cc:1217] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:39.824024  2521 raft_consensus.cc:1064] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:39.826737  2521 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:39.833750  1631 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.834635  1552 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.837030  2457 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.837809  2295 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.838718  2522 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.841583  2522 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:39.852409  2521 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:39.866539  1628 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:39.867107  2295 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:39.864321  1552 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:39.887219  1455 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:39.921303  2503 raft_consensus.cc:1064] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:39.934293  2503 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:39.946540  1704 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:39.969269  2295 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:39.972324  2521 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:39.975419  2295 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.977910  1703 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.979882  2464 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.982504  1632 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:39.969378  1628 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:39.985016  2486 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.991230  2464 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.996415  2527 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.008584  2522 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.010689  2295 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.017400  1627 raft_consensus.cc:2955] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.043006  1455 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.050872  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 41b83284fac147339ac4124c2eedee11 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:40.056825  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57318.8us
I20260709 13:00:40.057214  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 41b83284fac147339ac4124c2eedee11 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:40.082219  2486 raft_consensus.cc:1064] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:40.084849  2486 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:40.088783  2464 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.089900  2539 tablet_replica.cc:333] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:40.090158  2296 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.093923  2539 raft_consensus.cc:2243] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:40.094702  2539 raft_consensus.cc:2272] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:40.099501  2539 ts_tablet_manager.cc:1916] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:40.108217  2295 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:40.109827  1627 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:40.111864  1704 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:40.109881  1628 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.114295  1703 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.119262  1458 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.122743  2536 raft_consensus.cc:1064] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:40.125329  2536 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:40.116904  2486 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.128144  2503 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.130944  2540 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.165608  2539 ts_tablet_manager.cc:1929] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:40.170301  1552 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:40.187500  2539 log.cc:1199] T 41b83284fac147339ac4124c2eedee11 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/41b83284fac147339ac4124c2eedee11
I20260709 13:00:40.175869  1704 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:40.177412  1627 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:40.189462  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 41b83284fac147339ac4124c2eedee11 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:00:40.185599  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 55859.2us
I20260709 13:00:40.191439  2503 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.193763  2295 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:40.195182  2536 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:40.196628  2295 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:40.197321  1627 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.209168  2540 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:40.209232  1704 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.202215  1632 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:40.216449  2540 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.218554  2542 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:40.221891  2543 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:40.225235  1458 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.233016  2295 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.228292  2486 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.251425  2297 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.268467  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dea1928bdd634efaa34550f21ff564f3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:40.275136  1554 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:40.282732  1458 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.287534  2542 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.293460  2293 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:40.297537  2294 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.299925  1702 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:40.300513  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet dea1928bdd634efaa34550f21ff564f3 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:40.309653  2536 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:40.318722  2464 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:40.318831  1632 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.319623  1627 raft_consensus.cc:2955] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.340476  2540 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.350041  1702 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.338155  1458 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.372138  2551 tablet_replica.cc:333] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:40.373574  2551 raft_consensus.cc:2243] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:40.374464  2551 raft_consensus.cc:2272] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:40.381752  2551 ts_tablet_manager.cc:1916] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:40.385886  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet de80dd9c151b41e88e1469985d074b81 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:40.369041  2549 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:00:40.369059  2295 raft_consensus.cc:2955] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.415135  1458 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.419060  2551 ts_tablet_manager.cc:1929] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:40.419623  2551 log.cc:1199] T dea1928bdd634efaa34550f21ff564f3 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/dea1928bdd634efaa34550f21ff564f3
I20260709 13:00:40.421293  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet dea1928bdd634efaa34550f21ff564f3 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:40.424618  1703 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.445086  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet de80dd9c151b41e88e1469985d074b81 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:40.448742  2552 tablet_replica.cc:333] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:40.449627  2552 raft_consensus.cc:2243] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:40.450404  2552 raft_consensus.cc:2272] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:40.460544  2552 ts_tablet_manager.cc:1916] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:40.464576  1552 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:40.470620  2295 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:40.472642  2542 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.474512  1704 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:40.478513  2542 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:40.480316  2552 ts_tablet_manager.cc:1929] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:00:40.481981  2552 log.cc:1199] T de80dd9c151b41e88e1469985d074b81 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/de80dd9c151b41e88e1469985d074b81
I20260709 13:00:40.483670  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet de80dd9c151b41e88e1469985d074b81 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:40.482923  2543 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.483259  2295 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.489526  1703 raft_consensus.cc:2955] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.502215  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78699c8b654944aa9788917e2a177d14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:40.506676  1456 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:40.518106  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 78699c8b654944aa9788917e2a177d14 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:40.519621  2552 tablet_replica.cc:333] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:40.520773  2552 raft_consensus.cc:2243] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:40.521394  2552 raft_consensus.cc:2272] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:40.525866  2552 ts_tablet_manager.cc:1916] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:40.540158  2552 ts_tablet_manager.cc:1929] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:40.540620  2552 log.cc:1199] T 78699c8b654944aa9788917e2a177d14 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/78699c8b654944aa9788917e2a177d14
I20260709 13:00:40.542069  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 78699c8b654944aa9788917e2a177d14 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:41.588680  1704 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:41.595294  1673 consensus_peers.cc:597] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d1d1686090d4b3abcf51f94f1a798d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:41.595517  1628 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.597158  2522 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:41.597684  1552 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.599025  2557 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:41.623198  2522 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.629148  1552 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.637902  1554 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:41.639642  1632 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.648252  1627 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.651309  2542 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:41.652410  1457 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
W20260709 13:00:41.655031  1524 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21235338303a443e8f10b8773c68e2db. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:41.658921  1704 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.661571  2542 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:41.683701  2542 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.691538  1627 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.712419  1704 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.731434  1458 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.735302  1552 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:41.742952  1632 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.744791  1704 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.744792  2570 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:41.746099  2542 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:41.747244  1524 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35904a4f4e3747dd85997b0dae5ae7bc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:41.766201  2570 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.767254  1632 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.767406  1704 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.780354  1632 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:41.780794  1457 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
W20260709 13:00:41.792320  1599 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1f19e2b30024564847382b872fc80e1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:41.793443  1552 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.798010  1704 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:41.799335  2509 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:41.801046  2578 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:41.809715  2509 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.813815  1552 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.811321  1704 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:41.827515  1457 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:42.019258  2581 ts_tablet_manager.cc:933] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:42.021164  2581 tablet_copy_client.cc:323] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:42.022737  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 3d1d1686090d4b3abcf51f94f1a798d4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:42.023267  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 3d1d1686090d4b3abcf51f94f1a798d4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-3d1d1686090d4b3abcf51f94f1a798d4
I20260709 13:00:42.030648  1719 tablet_copy_source_session.cc:215] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:42.033571  2581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1d1686090d4b3abcf51f94f1a798d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.049539  2581 tablet_copy_client.cc:806] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:42.050119  2581 tablet_copy_client.cc:670] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:42.053543  2581 tablet_copy_client.cc:538] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:42.062103  2581 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:42.081776  2581 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:42.082661  2581 tablet_bootstrap.cc:492] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:42.083302  2581 ts_tablet_manager.cc:1403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.013s
I20260709 13:00:42.086119  2581 raft_consensus.cc:359] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.086908  2581 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:42.087574  2581 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.090823  2581 ts_tablet_manager.cc:1434] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.005s
I20260709 13:00:42.092844  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-3d1d1686090d4b3abcf51f94f1a798d4 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:42.093259  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-3d1d1686090d4b3abcf51f94f1a798d4 on tablet 3d1d1686090d4b3abcf51f94f1a798d4 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:42.193627  2581 ts_tablet_manager.cc:933] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:42.195451  2581 tablet_copy_client.cc:323] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:42.196923  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 21235338303a443e8f10b8773c68e2db from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:42.197459  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 21235338303a443e8f10b8773c68e2db from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-21235338303a443e8f10b8773c68e2db
I20260709 13:00:42.204821  1570 tablet_copy_source_session.cc:215] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:42.207527  2581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21235338303a443e8f10b8773c68e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.220705  2581 tablet_copy_client.cc:806] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:42.221293  2581 tablet_copy_client.cc:670] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:42.224577  2581 tablet_copy_client.cc:538] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:42.231630  2581 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:42.259858  2581 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:42.260762  2581 tablet_bootstrap.cc:492] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:42.261425  2581 ts_tablet_manager.cc:1403] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.000s
I20260709 13:00:42.264318  2581 raft_consensus.cc:359] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.265254  2581 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:42.265868  2581 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.268254  2581 ts_tablet_manager.cc:1434] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:42.271250  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-21235338303a443e8f10b8773c68e2db received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:42.271694  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-21235338303a443e8f10b8773c68e2db on tablet 21235338303a443e8f10b8773c68e2db with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:42.310204  2581 ts_tablet_manager.cc:933] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:42.312218  2581 tablet_copy_client.cc:323] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:42.314114  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet b1f19e2b30024564847382b872fc80e1 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:42.314617  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet b1f19e2b30024564847382b872fc80e1 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-b1f19e2b30024564847382b872fc80e1
I20260709 13:00:42.322829  1644 tablet_copy_source_session.cc:215] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:42.325989  2588 ts_tablet_manager.cc:933] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:42.327965  2588 tablet_copy_client.cc:323] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:42.327592  2581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f19e2b30024564847382b872fc80e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.330415  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 35904a4f4e3747dd85997b0dae5ae7bc from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:42.331004  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 35904a4f4e3747dd85997b0dae5ae7bc from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-35904a4f4e3747dd85997b0dae5ae7bc
I20260709 13:00:42.340288  1570 tablet_copy_source_session.cc:215] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:42.347373  2588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35904a4f4e3747dd85997b0dae5ae7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.350862  2581 tablet_copy_client.cc:806] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:42.351517  2581 tablet_copy_client.cc:670] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:42.356899  2581 tablet_copy_client.cc:538] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:42.366943  2581 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:42.367317  2588 tablet_copy_client.cc:806] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:42.368040  2588 tablet_copy_client.cc:670] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:42.372203  2588 tablet_copy_client.cc:538] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:00:42.423794  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56923.1us
I20260709 13:00:42.436517  2588 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:42.459075  2588 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:42.459947  2588 tablet_bootstrap.cc:492] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:42.460829  2588 ts_tablet_manager.cc:1403] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.000s
I20260709 13:00:42.464653  2588 raft_consensus.cc:359] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.465435  2588 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:42.466066  2588 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.468722  2588 ts_tablet_manager.cc:1434] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:42.470974  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-35904a4f4e3747dd85997b0dae5ae7bc received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:42.471355  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-35904a4f4e3747dd85997b0dae5ae7bc on tablet 35904a4f4e3747dd85997b0dae5ae7bc with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:42.475914  2581 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:42.479070  2581 tablet_bootstrap.cc:492] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:42.479707  2581 ts_tablet_manager.cc:1403] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.113s	user 0.028s	sys 0.007s
I20260709 13:00:42.482245  2581 raft_consensus.cc:359] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.483019  2581 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:42.483623  2581 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:42.485529  2581 ts_tablet_manager.cc:1434] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:00:42.487207  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-b1f19e2b30024564847382b872fc80e1 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:42.487635  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-b1f19e2b30024564847382b872fc80e1 on tablet b1f19e2b30024564847382b872fc80e1 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:42.523036  2295 raft_consensus.cc:1217] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:42.672868  2295 raft_consensus.cc:1217] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:42.759379  2295 raft_consensus.cc:1217] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:42.869362  2294 raft_consensus.cc:1217] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:43.044603  2522 raft_consensus.cc:1064] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:43.047323  2522 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.057969  2294 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.058151  1627 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.060556  1554 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.061810  2594 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:43.063526  2560 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:43.065517  2522 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.087728  2522 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.089708  1627 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.090246  2294 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.100865  2597 raft_consensus.cc:1064] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:43.103682  2597 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.115617  1457 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.131558  1703 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.131912  1556 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.133860  2570 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:43.140769  2294 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.141659  1628 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.142290  2596 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:43.144161  2596 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.167476  2596 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.169224  1628 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.172931  1703 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.182780  2295 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.221891  1456 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.224263  1703 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.235131  1628 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.237320  2522 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.237980  2295 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.245990  2560 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.251374  2560 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.252532  1627 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.263128  2295 raft_consensus.cc:2955] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.274250  1556 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.281985  2294 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.285251  1627 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.284266  1455 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.294489  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3d1d1686090d4b3abcf51f94f1a798d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:43.297066  2568 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:43.302927  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d1d1686090d4b3abcf51f94f1a798d4 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:43.305788  2568 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:43.308777  2611 tablet_replica.cc:333] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:43.321668  2596 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.327337  1627 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.360077  2295 raft_consensus.cc:2955] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.363561  1458 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.372150  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 21235338303a443e8f10b8773c68e2db with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:43.376618  2611 raft_consensus.cc:2243] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:43.377488  2611 raft_consensus.cc:2272] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:43.380177  2590 raft_consensus.cc:1064] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:43.381628  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 21235338303a443e8f10b8773c68e2db with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:43.383179  2590 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.388899  2611 ts_tablet_manager.cc:1916] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:43.400164  2612 tablet_replica.cc:333] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:43.403251  2612 raft_consensus.cc:2243] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:43.404161  2612 raft_consensus.cc:2272] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:43.405717  1702 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.408078  2294 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.410336  1554 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.411662  2591 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:43.412952  2570 raft_consensus.cc:1064] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:43.413656  2591 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.414917  2611 ts_tablet_manager.cc:1929] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:43.415555  2611 log.cc:1199] T 3d1d1686090d4b3abcf51f94f1a798d4 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/3d1d1686090d4b3abcf51f94f1a798d4
I20260709 13:00:43.416154  2570 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.417320  2509 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.419294  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 3d1d1686090d4b3abcf51f94f1a798d4 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:43.433131  2612 ts_tablet_manager.cc:1916] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:43.439548  1627 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.440747  2293 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.453245  2570 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.456156  2596 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.444849  1703 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:43.463992  2570 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:43.466070  2590 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.482472  1704 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.467880  1552 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.500216  2568 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.506075  2612 ts_tablet_manager.cc:1929] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:43.506542  2612 log.cc:1199] T 21235338303a443e8f10b8773c68e2db P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/21235338303a443e8f10b8773c68e2db
I20260709 13:00:43.509169  2295 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.511350  1457 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.516733  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 21235338303a443e8f10b8773c68e2db (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:43.528012  1704 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.526103  1631 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.537748  2295 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.546911  1552 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.548038  2293 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.548271  2509 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.549786  2590 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.554113  1629 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.557154  2509 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.568166  1552 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.578052  1457 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.580879  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b1f19e2b30024564847382b872fc80e1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:43.580475  2295 raft_consensus.cc:2955] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.605122  1456 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.615851  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f19e2b30024564847382b872fc80e1 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:43.616775  2612 tablet_replica.cc:333] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:43.617832  2612 raft_consensus.cc:2243] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:43.618525  2612 raft_consensus.cc:2272] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:43.621946  2612 ts_tablet_manager.cc:1916] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:43.639675  2612 ts_tablet_manager.cc:1929] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:43.640111  2612 log.cc:1199] T b1f19e2b30024564847382b872fc80e1 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/b1f19e2b30024564847382b872fc80e1
I20260709 13:00:43.641767  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet b1f19e2b30024564847382b872fc80e1 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:43.662981  1552 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:43.671042  2295 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.672276  2597 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.672497  1703 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:43.673661  2568 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:43.679212  2596 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.680506  2295 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.684203  1703 raft_consensus.cc:2955] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:43.688380  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35904a4f4e3747dd85997b0dae5ae7bc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:43.694779  1455 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.713361  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 35904a4f4e3747dd85997b0dae5ae7bc with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:43.724174  2621 tablet_replica.cc:333] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:43.724917  2621 raft_consensus.cc:2243] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:43.725410  2621 raft_consensus.cc:2272] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:43.728195  2621 ts_tablet_manager.cc:1916] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:43.739966  2621 ts_tablet_manager.cc:1929] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:43.740270  2621 log.cc:1199] T 35904a4f4e3747dd85997b0dae5ae7bc P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/35904a4f4e3747dd85997b0dae5ae7bc
I20260709 13:00:43.741642  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 35904a4f4e3747dd85997b0dae5ae7bc (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:44.781940  1552 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:44.788667  1702 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:44.789217  1628 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:44.789217  1524 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88dfd27965e145dcb810423d7396b403. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:44.790081  2597 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:44.790930  2622 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:44.800217  2623 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.803135  1702 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.809017  1628 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.825600  1702 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:44.835517  1673 consensus_peers.cc:597] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4481ad55e16e48b7a3442111d9636f55. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:44.835517  1552 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:44.837565  1628 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:44.837670  2626 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:44.837247  1457 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.838738  2625 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:44.855634  2624 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.860896  1552 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.860167  1633 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.905305  1559 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:44.909796  1456 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.962819  1633 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:44.963040  1524 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d254c50f08d431298fe57ef1dde64be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:44.964172  2641 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:44.965026  1704 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:44.967276  2623 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:44.983047  2597 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.984930  1633 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.986264  1704 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:44.999161  1559 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.006945  1455 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:45.012292  1524 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd868018972045628b3604762f904e05. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:45.015733  1628 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:45.015885  1704 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:45.017411  2641 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:45.018334  2623 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:45.044215  2623 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:45.045989  1628 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:45.053164  1702 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:45.065495  1456 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.326511  2647 ts_tablet_manager.cc:933] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:45.328657  2647 tablet_copy_client.cc:323] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:45.330580  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 4481ad55e16e48b7a3442111d9636f55 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:45.331111  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 4481ad55e16e48b7a3442111d9636f55 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-4481ad55e16e48b7a3442111d9636f55
I20260709 13:00:45.337198  1719 tablet_copy_source_session.cc:215] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:45.340498  2647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4481ad55e16e48b7a3442111d9636f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:45.357524  2647 tablet_copy_client.cc:806] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:45.358111  2647 tablet_copy_client.cc:670] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:45.362169  2647 tablet_copy_client.cc:538] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:45.368891  2647 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:45.386102  2647 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:45.386960  2647 tablet_bootstrap.cc:492] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:45.387655  2647 ts_tablet_manager.cc:1403] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.001s
I20260709 13:00:45.390050  2647 raft_consensus.cc:359] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.390861  2647 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:45.391520  2647 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:45.393774  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800027700 after lost signal to thread 434
W20260709 13:00:45.394826  1656 debug-util.cc:398] Leaking SignalData structure 0x7b080009e920 after lost signal to thread 1492
W20260709 13:00:45.395982  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800255740 after lost signal to thread 1584
W20260709 13:00:45.397125  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800301a20 after lost signal to thread 1659
W20260709 13:00:45.398165  1656 debug-util.cc:398] Leaking SignalData structure 0x7b080016ad00 after lost signal to thread 1733
W20260709 13:00:45.399261  1656 debug-util.cc:398] Leaking SignalData structure 0x7b080023ebe0 after lost signal to thread 2321
I20260709 13:00:45.404312  2647 ts_tablet_manager.cc:1434] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:00:45.409651  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-4481ad55e16e48b7a3442111d9636f55 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:45.410157  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-4481ad55e16e48b7a3442111d9636f55 on tablet 4481ad55e16e48b7a3442111d9636f55 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:45.414040  2647 ts_tablet_manager.cc:933] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:45.423486  2647 tablet_copy_client.cc:323] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:45.426143  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 88dfd27965e145dcb810423d7396b403 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:45.426697  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 88dfd27965e145dcb810423d7396b403 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-88dfd27965e145dcb810423d7396b403
I20260709 13:00:45.435233  1570 tablet_copy_source_session.cc:215] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:45.438645  2647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88dfd27965e145dcb810423d7396b403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:45.457544  2647 tablet_copy_client.cc:806] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:45.458175  2647 tablet_copy_client.cc:670] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:45.463449  2647 tablet_copy_client.cc:538] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:45.475113  2647 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:45.489775  2651 ts_tablet_manager.cc:933] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:45.499820  2651 tablet_copy_client.cc:323] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:45.501382  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 9d254c50f08d431298fe57ef1dde64be from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:45.501997  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 9d254c50f08d431298fe57ef1dde64be from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9d254c50f08d431298fe57ef1dde64be
I20260709 13:00:45.510787  1570 tablet_copy_source_session.cc:215] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:45.511387  2647 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:45.512353  2647 tablet_bootstrap.cc:492] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:45.513478  2647 ts_tablet_manager.cc:1403] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.039s	user 0.026s	sys 0.000s
I20260709 13:00:45.514582  2651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d254c50f08d431298fe57ef1dde64be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:45.517530  2647 raft_consensus.cc:359] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.518978  2647 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:45.519619  2647 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.522512  2647 ts_tablet_manager.cc:1434] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:00:45.536741  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-88dfd27965e145dcb810423d7396b403 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:45.537266  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-88dfd27965e145dcb810423d7396b403 on tablet 88dfd27965e145dcb810423d7396b403 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:45.570775  2651 tablet_copy_client.cc:806] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:45.574342  2651 tablet_copy_client.cc:670] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:45.578405  2651 tablet_copy_client.cc:538] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:45.588310  2651 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:45.612771  2651 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:45.618042  2651 tablet_bootstrap.cc:492] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:45.618875  2651 ts_tablet_manager.cc:1403] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.004s
I20260709 13:00:45.623104  2647 ts_tablet_manager.cc:933] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:45.622097  2651 raft_consensus.cc:359] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.629444  2651 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:45.631212  2651 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.639569  2651 ts_tablet_manager.cc:1434] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:00:45.643505  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9d254c50f08d431298fe57ef1dde64be received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:45.643961  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9d254c50f08d431298fe57ef1dde64be on tablet 9d254c50f08d431298fe57ef1dde64be with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:45.643884  2647 tablet_copy_client.cc:323] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:45.646186  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet bd868018972045628b3604762f904e05 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:45.646708  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet bd868018972045628b3604762f904e05 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-bd868018972045628b3604762f904e05
I20260709 13:00:45.656965  1570 tablet_copy_source_session.cc:215] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:45.660252  2647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd868018972045628b3604762f904e05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:45.679312  2647 tablet_copy_client.cc:806] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:45.679883  2647 tablet_copy_client.cc:670] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:45.684072  2647 tablet_copy_client.cc:538] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:45.693104  2647 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:45.715060  2647 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:45.715919  2647 tablet_bootstrap.cc:492] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:45.716549  2647 ts_tablet_manager.cc:1403] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:00:45.719713  2647 raft_consensus.cc:359] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.720656  2647 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:45.721275  2647 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:45.727314  2647 ts_tablet_manager.cc:1434] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:45.729343  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-bd868018972045628b3604762f904e05 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:45.729811  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-bd868018972045628b3604762f904e05 on tablet bd868018972045628b3604762f904e05 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:45.766993  2293 raft_consensus.cc:1217] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:45.847028  2293 raft_consensus.cc:1217] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:46.020495  2293 raft_consensus.cc:1217] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:46.035596  2293 raft_consensus.cc:1217] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:46.373972  2624 raft_consensus.cc:1064] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:46.384755  2624 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.405139  2293 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.405809  1634 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.406701  2624 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.420881  1559 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.425987  2627 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.430035  2627 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.444063  2622 raft_consensus.cc:1064] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:46.448107  2622 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.455140  2624 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.460662  1552 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.478293  1457 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.485332  1634 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.491742  2293 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.499497  2294 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.503355  1704 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.510763  1628 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.512245  2622 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.514031  2622 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.518585  2641 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.589269  1704 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.587050  2622 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.597700  2641 raft_consensus.cc:1064] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:46.602246  2641 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.628973  1628 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.631198  2597 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.637780  1634 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.695869  1704 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.709697  2293 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.712071  2624 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.712599  1633 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.714534  2624 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.717197  1455 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.720762  2658 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.725251  1633 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.725812  2295 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:00:46.735685  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134523us
I20260709 13:00:46.741919  2295 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.743481  1700 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.740657  2293 raft_consensus.cc:2955] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.767084  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4481ad55e16e48b7a3442111d9636f55 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:46.774160  2673 raft_consensus.cc:1064] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:46.777149  2673 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.780803  2677 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.788578  2675 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:46.791579  1456 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.808579  2294 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.813333  1703 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.816529  2678 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:46.816648  1552 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.818481  2675 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.823395  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 4481ad55e16e48b7a3442111d9636f55 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:46.830838  1704 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.835505  2294 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.839196  2674 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:46.834167  1628 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:46.844614  2294 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.848222  2677 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:46.852435  1633 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.860247  1456 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.867105  2685 tablet_replica.cc:333] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:46.860146  2677 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:00:46.875936  2673 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.882866  2675 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.894313  2293 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.904405  1704 raft_consensus.cc:2955] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.920518  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 88dfd27965e145dcb810423d7396b403 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:46.931692  1552 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:46.939437  2641 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.941990  1634 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.943572  2675 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.950793  1454 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.953194  2673 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.955241  2293 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:46.957536  2622 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.961651  1634 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.963864  2294 raft_consensus.cc:2955] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.965536  2293 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.965628  1707 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:46.971395  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 88dfd27965e145dcb810423d7396b403 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:46.999893  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9d254c50f08d431298fe57ef1dde64be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:47.012421  1635 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.017832  1455 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.031692  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d254c50f08d431298fe57ef1dde64be with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:47.060873  2685 raft_consensus.cc:2243] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:47.061708  2685 raft_consensus.cc:2272] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:47.063091  2689 tablet_replica.cc:333] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:47.066295  2689 raft_consensus.cc:2243] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:47.074579  2690 tablet_replica.cc:333] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:47.062601  1456 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.072278  2689 raft_consensus.cc:2272] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:47.068225  2685 ts_tablet_manager.cc:1916] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:47.084342  2689 ts_tablet_manager.cc:1916] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:47.088191  2690 raft_consensus.cc:2243] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:47.091933  2690 pending_rounds.cc:70] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Trying to abort 1 pending ops.
I20260709 13:00:47.092351  2690 pending_rounds.cc:77] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305633982869422080 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "9d254c50f08d431298fe57ef1dde64be" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } } }
I20260709 13:00:47.093694  2690 raft_consensus.cc:2917] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:00:47.094184  2690 raft_consensus.cc:2889] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:00:47.094751  2690 raft_consensus.cc:2272] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:47.102335  1552 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:47.103106  2685 ts_tablet_manager.cc:1929] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:47.103520  2685 log.cc:1199] T 4481ad55e16e48b7a3442111d9636f55 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/4481ad55e16e48b7a3442111d9636f55
I20260709 13:00:47.105265  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 4481ad55e16e48b7a3442111d9636f55 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:47.112843  2293 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:47.117695  2641 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:47.122315  2690 ts_tablet_manager.cc:1916] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:47.127254  2676 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.147743  2294 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.155930  1704 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:47.163568  2641 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:47.167860  2690 ts_tablet_manager.cc:1929] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:47.167932  1704 raft_consensus.cc:2955] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.173677  2689 ts_tablet_manager.cc:1929] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:47.177804  2689 log.cc:1199] T 88dfd27965e145dcb810423d7396b403 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/88dfd27965e145dcb810423d7396b403
I20260709 13:00:47.179744  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 88dfd27965e145dcb810423d7396b403 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:47.183924  2690 log.cc:1199] T 9d254c50f08d431298fe57ef1dde64be P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/9d254c50f08d431298fe57ef1dde64be
I20260709 13:00:47.185616  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 9d254c50f08d431298fe57ef1dde64be (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:47.193523  1456 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:47.203580  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet bd868018972045628b3604762f904e05 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:47.204392  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bd868018972045628b3604762f904e05 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:47.204583  2689 tablet_replica.cc:333] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:47.205577  2689 raft_consensus.cc:2243] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:47.206182  2689 raft_consensus.cc:2272] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:47.209342  2689 ts_tablet_manager.cc:1916] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:47.231062  2689 ts_tablet_manager.cc:1929] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:47.231575  2689 log.cc:1199] T bd868018972045628b3604762f904e05 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/bd868018972045628b3604762f904e05
I20260709 13:00:47.233243  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet bd868018972045628b3604762f904e05 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:48.276782  1635 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.284876  1704 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.285111  1560 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.287268  2590 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:48.288149  2660 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:48.292914  1599 consensus_peers.cc:597] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23585cacd4d249b4aa2ca97e2fb3ea8d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:48.303135  2660 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.304934  1560 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.305538  1704 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.322794  1552 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.329706  1455 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.338012  1704 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.339249  1635 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.340659  2641 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:48.342823  2676 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:48.343160  1524 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7346e6ca66624e10898433387e0bbe68. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:48.357366  2673 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.365590  1704 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.372669  1560 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.372288  1631 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.389333  1635 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.391240  2641 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:48.395553  1524 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26aae1e2780d42cda794411c9c1c0f32. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:48.389350  1707 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.399484  2641 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:48.429618  2641 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.431516  1635 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.440518  1707 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.451416  1704 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.475674  1458 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:48.477818  1458 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:48.488463  1552 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.493019  2658 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:48.496014  1673 consensus_peers.cc:597] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f8c2a4b167d431aac3de9d8166bab1f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:48.531277  1635 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:48.535143  2624 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.536296  1552 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.569518  2709 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:48.573947  1458 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.597767  1635 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:48.741976  2713 ts_tablet_manager.cc:933] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:48.749495  2713 tablet_copy_client.cc:323] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:48.751222  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 7346e6ca66624e10898433387e0bbe68 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:48.751811  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 7346e6ca66624e10898433387e0bbe68 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-7346e6ca66624e10898433387e0bbe68
I20260709 13:00:48.761729  1570 tablet_copy_source_session.cc:215] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:48.766086  2713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7346e6ca66624e10898433387e0bbe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.781787  2713 tablet_copy_client.cc:806] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:48.782409  2713 tablet_copy_client.cc:670] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:48.786197  2713 tablet_copy_client.cc:538] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:48.794723  2713 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:48.815397  2713 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:48.816332  2713 tablet_bootstrap.cc:492] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:48.817071  2713 ts_tablet_manager.cc:1403] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.004s
I20260709 13:00:48.821213  2713 raft_consensus.cc:359] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.822140  2713 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:48.822844  2713 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:48.833755  2713 ts_tablet_manager.cc:1434] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:00:48.836431  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-7346e6ca66624e10898433387e0bbe68 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:48.836877  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-7346e6ca66624e10898433387e0bbe68 on tablet 7346e6ca66624e10898433387e0bbe68 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:48.852875  2713 ts_tablet_manager.cc:933] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:48.862037  2713 tablet_copy_client.cc:323] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:48.865815  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:48.866387  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet 23585cacd4d249b4aa2ca97e2fb3ea8d from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-23585cacd4d249b4aa2ca97e2fb3ea8d
I20260709 13:00:48.880988  2718 ts_tablet_manager.cc:933] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031)
I20260709 13:00:48.880206  1644 tablet_copy_source_session.cc:215] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:48.887633  2713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.892555  2718 tablet_copy_client.cc:323] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:38031
I20260709 13:00:48.899035  1570 tablet_copy_service.cc:140] P 444b494bb53842b59408a7230884398a: Received BeginTabletCopySession request for tablet 26aae1e2780d42cda794411c9c1c0f32 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:39480)
I20260709 13:00:48.899576  1570 tablet_copy_service.cc:161] P 444b494bb53842b59408a7230884398a: Beginning new tablet copy session on tablet 26aae1e2780d42cda794411c9c1c0f32 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:39480: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-26aae1e2780d42cda794411c9c1c0f32
I20260709 13:00:48.912699  1570 tablet_copy_source_session.cc:215] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:48.921108  2718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26aae1e2780d42cda794411c9c1c0f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.951110  2718 tablet_copy_client.cc:806] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:48.951759  2718 tablet_copy_client.cc:670] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:48.966312  2713 tablet_copy_client.cc:806] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:48.971908  2713 tablet_copy_client.cc:670] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:48.992071  2713 tablet_copy_client.cc:538] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:48.999861  2718 tablet_copy_client.cc:538] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:49.019079  2718 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:49.022855  2713 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:49.060320  2722 ts_tablet_manager.cc:933] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:49.067624  2713 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:49.068740  2713 tablet_bootstrap.cc:492] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:49.069532  2713 ts_tablet_manager.cc:1403] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.047s	user 0.033s	sys 0.008s
I20260709 13:00:49.077167  2722 tablet_copy_client.cc:323] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:49.081303  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 7f8c2a4b167d431aac3de9d8166bab1f from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:49.081851  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 7f8c2a4b167d431aac3de9d8166bab1f from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-7f8c2a4b167d431aac3de9d8166bab1f
I20260709 13:00:49.079061  2713 raft_consensus.cc:359] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.082653  2713 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:49.083292  2713 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.085909  2713 ts_tablet_manager.cc:1434] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:00:49.088843  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-23585cacd4d249b4aa2ca97e2fb3ea8d received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:49.089329  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-23585cacd4d249b4aa2ca97e2fb3ea8d on tablet 23585cacd4d249b4aa2ca97e2fb3ea8d with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:49.091614  1719 tablet_copy_source_session.cc:215] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:49.093317  2718 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:49.094261  2718 tablet_bootstrap.cc:492] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:49.094914  2722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f8c2a4b167d431aac3de9d8166bab1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:49.095006  2718 ts_tablet_manager.cc:1403] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.076s	user 0.036s	sys 0.024s
I20260709 13:00:49.104653  2718 raft_consensus.cc:359] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.105526  2718 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:49.106189  2718 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.109073  2718 ts_tablet_manager.cc:1434] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:00:49.110931  1570 tablet_copy_service.cc:342] P 444b494bb53842b59408a7230884398a: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-26aae1e2780d42cda794411c9c1c0f32 received from {username='slave'} at 127.0.0.1:39480
I20260709 13:00:49.111339  1570 tablet_copy_service.cc:434] P 444b494bb53842b59408a7230884398a: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-26aae1e2780d42cda794411c9c1c0f32 on tablet 26aae1e2780d42cda794411c9c1c0f32 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:49.132073  2722 tablet_copy_client.cc:806] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:49.138774  2722 tablet_copy_client.cc:670] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:49.143975  2722 tablet_copy_client.cc:538] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:49.160229  2722 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:49.220932  2294 raft_consensus.cc:1217] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:49.235474  2722 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:49.236464  2722 tablet_bootstrap.cc:492] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:49.237171  2722 ts_tablet_manager.cc:1403] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.077s	user 0.032s	sys 0.021s
I20260709 13:00:49.241135  2722 raft_consensus.cc:359] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.242067  2722 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:49.242700  2722 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:49.257835  2722 ts_tablet_manager.cc:1434] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:00:49.262034  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-7f8c2a4b167d431aac3de9d8166bab1f received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:49.263191  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-7f8c2a4b167d431aac3de9d8166bab1f on tablet 7f8c2a4b167d431aac3de9d8166bab1f with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:49.290153  2294 raft_consensus.cc:1217] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:49.344808  2294 raft_consensus.cc:1217] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:49.507135  2293 raft_consensus.cc:1217] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:49.765280  2590 raft_consensus.cc:1064] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:49.767549  2590 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:49.774633  1707 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.775873  2692 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.779615  2293 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.781349  1560 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.782945  2692 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.785778  2728 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.787235  2725 raft_consensus.cc:1064] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:49.790192  2725 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:49.847275  1704 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.848928  2641 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.852468  2293 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.856771  2692 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.858670  1635 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.860550  2641 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.861325  1560 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.866719  2726 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.877789  1703 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.884896  2641 raft_consensus.cc:1064] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:49.889724  2641 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:49.892063  2294 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.896099  1455 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.900534  2725 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.919010  2294 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.926928  1704 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.930981  1635 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:49.934540  2293 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.936981  1703 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.937754  1634 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.938328  2641 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.939622  2726 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.957088  1634 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:49.972114  2709 raft_consensus.cc:1064] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:49.974706  2709 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:49.981238  2295 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.983075  2641 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.986729  2590 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.003723  1628 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:50.005504  2296 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:50.005754  2729 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.007223  2624 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:50.010468  2624 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:50.015056  2297 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.055486  1455 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.066833  1458 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.079979  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:50.084239  2709 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:00:50.085755  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68276.4us
I20260709 13:00:50.090054  1707 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.094077  2750 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.098726  1634 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.103305  1706 raft_consensus.cc:2955] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.106637  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23585cacd4d249b4aa2ca97e2fb3ea8d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:50.128863  2733 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.133966  1458 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.134810  1707 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:00:50.134807  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107492us
W20260709 13:00:50.154417  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52527.6us
I20260709 13:00:50.160243  1555 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:50.166196  1634 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.176287  2294 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.178071  2754 tablet_replica.cc:333] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:50.178472  1458 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.184371  2754 raft_consensus.cc:2243] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:50.185222  2754 raft_consensus.cc:2272] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:50.198804  1556 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:50.205957  2754 ts_tablet_manager.cc:1916] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:50.205358  2293 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.207398  1634 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.213611  2733 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.218822  2710 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:50.224611  2755 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.230911  2754 ts_tablet_manager.cc:1929] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:50.231420  1634 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.234660  2294 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.239065  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7346e6ca66624e10898433387e0bbe68 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:50.243569  1556 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:50.243943  2733 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.245208  1704 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:50.247743  2754 log.cc:1199] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/23585cacd4d249b4aa2ca97e2fb3ea8d
I20260709 13:00:50.253144  2294 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.258476  1458 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.259684  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 23585cacd4d249b4aa2ca97e2fb3ea8d (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:50.258356  2293 raft_consensus.cc:2955] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.261576  2297 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.261605  1703 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.263963  2756 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.266480  1634 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:50.268597  2733 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.269488  2624 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.270906  2733 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:50.278079  2709 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.286718  1556 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.288687  2726 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.286821  2294 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.293294  1635 raft_consensus.cc:2955] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.299690  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 7346e6ca66624e10898433387e0bbe68 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:50.306085   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:50.312736  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26aae1e2780d42cda794411c9c1c0f32 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:50.313133  2293 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.328572  1703 raft_consensus.cc:2955] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.332423  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f8c2a4b167d431aac3de9d8166bab1f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:50.337513  1456 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:50.350406  2758 tablet_replica.cc:333] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:50.351497  2758 raft_consensus.cc:2243] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:50.361886  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f8c2a4b167d431aac3de9d8166bab1f with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:50.362982  2754 tablet_replica.cc:333] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:50.364042  2754 raft_consensus.cc:2243] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:50.365283  2754 raft_consensus.cc:2272] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:50.373852  2754 ts_tablet_manager.cc:1916] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:50.371626  1454 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.394162  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 26aae1e2780d42cda794411c9c1c0f32 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:50.399293  2758 raft_consensus.cc:2272] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:50.403749  2754 ts_tablet_manager.cc:1929] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:50.404290  2754 log.cc:1199] T 7f8c2a4b167d431aac3de9d8166bab1f P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/7f8c2a4b167d431aac3de9d8166bab1f
I20260709 13:00:50.405164  2758 ts_tablet_manager.cc:1916] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:50.412846  2759 tablet_replica.cc:333] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:50.414587  2759 raft_consensus.cc:2243] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:50.407131  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 7f8c2a4b167d431aac3de9d8166bab1f (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:50.419406  2759 raft_consensus.cc:2272] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:50.426378  2759 ts_tablet_manager.cc:1916] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:50.443161  2759 ts_tablet_manager.cc:1929] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:50.443636  2759 log.cc:1199] T 26aae1e2780d42cda794411c9c1c0f32 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/26aae1e2780d42cda794411c9c1c0f32
I20260709 13:00:50.450909  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 26aae1e2780d42cda794411c9c1c0f32 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:50.459455  2758 ts_tablet_manager.cc:1929] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:50.460251  2758 log.cc:1199] T 7346e6ca66624e10898433387e0bbe68 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/7346e6ca66624e10898433387e0bbe68
I20260709 13:00:50.463827  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 7346e6ca66624e10898433387e0bbe68 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:50.484628  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:50.485504  1703 raft_consensus.cc:606] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:50.500293  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:50.500825  1559 raft_consensus.cc:606] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:50.503203  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:50.503655  1703 raft_consensus.cc:606] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:50.506053  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:50.506476  1703 raft_consensus.cc:606] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:50.509801  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:50.510346  1703 raft_consensus.cc:606] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:50.512651  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:50.513163  1703 raft_consensus.cc:606] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:50.515992   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:00:50.517428   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:50.662484  2624 raft_consensus.cc:993] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.663040  2710 raft_consensus.cc:1081] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.664422  1628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:50.664867  1628 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.665114  1628 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.670279  1628 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.672665  1628 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:50.673585  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:50.673627  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:50.674360  1704 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.674422  1559 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.674641  1704 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:50.675381  1704 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.676347  1704 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.680002  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.681046  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:50.681687  2764 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.681788  1704 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.682199  2764 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:50.683060  2764 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.692407  1456 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.701390  2725 raft_consensus.cc:993] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.701941  2733 raft_consensus.cc:1081] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.703423  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36804
I20260709 13:00:50.703912  1635 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.704236  1635 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.709240  1635 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.710727  1635 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:50.711395  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:50.711725  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:50.712035  1559 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.712306  1704 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.712349  1559 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:50.713102  1559 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.714092  1559 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.720580  1559 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.721628  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.721624  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:50.722573  2764 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.723049  2764 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:50.723918  2764 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.743549  1456 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.793632  2624 raft_consensus.cc:993] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.794117  2709 raft_consensus.cc:1081] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.795583  1628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:50.796169  1628 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.796499  1628 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.801493  1628 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.803867  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:50.804176  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:50.804468  1559 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.804742  1703 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.805070  1703 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:50.805838  1703 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.806856  1703 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.810883  1559 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.811687  1628 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:50.811967  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:50.812836  2764 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.813308  1703 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.813506  2764 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:50.814507  2764 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.824522  1456 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.840545  2709 raft_consensus.cc:993] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.841054  2710 raft_consensus.cc:1081] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:50.842396  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:50.842903  1635 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.843240  1635 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.849045  1635 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.850839  1635 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:50.851660  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:50.851758  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:50.852583  1703 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.852805  1703 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:50.853478  1703 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.854568  1703 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.856007  1559 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.859567  1703 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.860635  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:50.860816  1559 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:50.861322  2764 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:50.861811  2764 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:50.862607  2764 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:50.871881  1456 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.978381  2710 raft_consensus.cc:993] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:50.978791  2624 raft_consensus.cc:1081] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:50.980139  2294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57952
I20260709 13:00:50.980589  2294 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:50.980865  2294 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.986019  2294 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:50.987814  2294 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:50.988525  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:50.989253  1635 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.989177  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:50.989895  1703 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:50.990212  1703 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:50.991025  1703 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:50.992233  1703 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:50.996201  1635 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:50.997407  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:50.998924  1703 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:51.001112  2769 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:51.003007  2769 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:51.003906  2769 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:51.014875  1456 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:51.037322  2710 raft_consensus.cc:993] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:51.037873  2710 raft_consensus.cc:1081] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:51.039508  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:51.040148  1635 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:51.040479  1635 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.045017  1635 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:51.046928  1635 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:51.048215  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:51.048571  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:51.049197  1703 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:51.049437  1703 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:51.049538  1559 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.050297  1703 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:51.051543  1703 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.055215  1559 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:51.056246  1601 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:51.056972  1703 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:51.057293  2764 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:51.057832  2764 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:51.058707  2764 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:51.067299  1456 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:51.141976  1703 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.143244  2763 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.155822  1560 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.157073  2764 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.187283  1703 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.188345  2777 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.198093  1560 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.199517  2763 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.217751  1560 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.222581  2763 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.236002  1703 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.240875  2777 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.461277  1635 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:51.462373  2769 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:51.470871  1635 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:51.474774  1703 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:51.478848  2769 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:51.487226  1704 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.487334  1559 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:51.488489  1599 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af498956d398433a8c293e398bd1255f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:51.489272  2777 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.490825  1704 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.496493  2763 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.492420  1559 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.493686  2764 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.499866  2777 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.501281  2777 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.503415  1706 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.519055   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:51.559255  1559 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.578835  1704 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
W20260709 13:00:51.597807  1673 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9438873b8e1a4f0780fff8b93a9d89b8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:51.598404  1560 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.599992  2624 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.599545  1455 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.601246  1635 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.603749  2624 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.626333  1559 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.635751  2792 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.641472  2624 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.644809  1559 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.656018  1635 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.657783  1704 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:51.658442   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:51.663007  2792 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:51.709127  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60298.2us
I20260709 13:00:51.713009  1455 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.735108  1635 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:51.784746  1706 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.786423  2777 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:51.790360  1560 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.791888  2764 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:51.788648  1599 consensus_peers.cc:597] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab336cb98e24470eb828d9e9fdbcb001. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:51.805209  2763 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.808737  1706 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.811995  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:51.812604  1703 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:51.820755  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:51.821313  1703 raft_consensus.cc:606] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:51.825253  1634 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:51.833847  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:51.833331  1559 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.834568  1703 raft_consensus.cc:606] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:51.840071  1455 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.843828  1556 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:00:51.845206  1599 consensus_peers.cc:597] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a986b2c96ad0416f8fc5927d45ebadc4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:51.845652  2792 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:51.841526  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:51.842675  1704 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:51.851274  2764 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:51.854512  1560 raft_consensus.cc:606] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:51.859604  2777 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.862054  1704 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.865023  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:51.865585  1703 raft_consensus.cc:606] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:51.866631   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:51.868212   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:51.867306  1559 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.887399  1456 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:51.911377  2810 ts_tablet_manager.cc:933] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:51.913803  2810 tablet_copy_client.cc:323] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:51.915244  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet af498956d398433a8c293e398bd1255f from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:51.915808  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet af498956d398433a8c293e398bd1255f from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-af498956d398433a8c293e398bd1255f
I20260709 13:00:51.923290  1644 tablet_copy_source_session.cc:215] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:51.925910  2810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af498956d398433a8c293e398bd1255f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:51.941213  2810 tablet_copy_client.cc:806] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:51.942037  2810 tablet_copy_client.cc:670] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:51.950563  2810 tablet_copy_client.cc:538] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:51.959196  2810 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:51.980435  2810 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:51.980837  2772 raft_consensus.cc:993] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:51.981369  2725 raft_consensus.cc:1081] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:51.981549  2810 tablet_bootstrap.cc:492] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:51.982437  2810 ts_tablet_manager.cc:1403] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.004s
I20260709 13:00:51.983170  2296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57942
I20260709 13:00:51.983754  2296 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:51.984822  2296 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.985953  2810 raft_consensus.cc:359] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:51.986878  2810 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:51.988982  2810 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:51.994925  2810 ts_tablet_manager.cc:1434] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:51.996685  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-af498956d398433a8c293e398bd1255f received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:51.997139  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-af498956d398433a8c293e398bd1255f on tablet af498956d398433a8c293e398bd1255f with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:51.996845  2296 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.002718  2813 ts_tablet_manager.cc:933] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:52.006664  2296 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:52.008438  2813 tablet_copy_client.cc:323] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:52.009809  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 9438873b8e1a4f0780fff8b93a9d89b8 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:52.010247  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 9438873b8e1a4f0780fff8b93a9d89b8 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9438873b8e1a4f0780fff8b93a9d89b8
I20260709 13:00:52.011994  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:52.012603  1635 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.015581  1719 tablet_copy_source_session.cc:215] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:52.018514  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:52.019047  1556 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:52.019304  1556 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:52.019399  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:52.020628  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:52.021024  1556 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.021632  2769 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:52.022238  1556 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.034175  1556 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:52.036469  2813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9438873b8e1a4f0780fff8b93a9d89b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:52.046370  2769 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:52.047629  2769 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.058815  2813 tablet_copy_client.cc:806] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:52.059422  2813 tablet_copy_client.cc:670] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:52.065143  1456 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.075344  2813 tablet_copy_client.cc:538] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:52.085512  2813 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:52.118683  2813 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:52.119612  2813 tablet_bootstrap.cc:492] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:52.120438  2813 ts_tablet_manager.cc:1403] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.008s
I20260709 13:00:52.123510  2813 raft_consensus.cc:359] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.124339  2813 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:52.125005  2813 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.127425  2813 ts_tablet_manager.cc:1434] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:00:52.135736  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9438873b8e1a4f0780fff8b93a9d89b8 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:52.136229  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9438873b8e1a4f0780fff8b93a9d89b8 on tablet 9438873b8e1a4f0780fff8b93a9d89b8 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:52.196671  2805 raft_consensus.cc:993] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:52.197201  2805 raft_consensus.cc:1081] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:52.198698  2296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57952
I20260709 13:00:52.199277  2296 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:52.199607  2296 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.208101  2296 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.210368  2296 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:52.210850  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:52.211606  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:52.212028  1635 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.212275  1706 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:52.213183  1706 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:52.214023  1706 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.215266  1706 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.220412  2799 raft_consensus.cc:993] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.220942  2799 raft_consensus.cc:1081] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.222108  1635 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:52.223485  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:52.223587  1634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:52.224161  1634 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:52.224632  1634 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:52.224664  2769 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:52.225358  2769 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:52.226227  2769 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.232259  1634 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.232282  1706 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:52.239514  1634 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:52.239735  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:52.240808  1559 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:52.243479  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:52.244119  1706 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:52.244481  1706 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:52.245595  1706 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.246740  1706 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:52.251360  1456 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.253021  1706 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:00:52.254525  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:52.255414  2777 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:52.255954  2777 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:52.256922  2777 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.257378  1559 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:00:52.271648  1456 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 3, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:52.281716  2805 raft_consensus.cc:993] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.282217  2799 raft_consensus.cc:1081] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.284838  1634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:52.285354  1634 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:52.285696  1634 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.292081  1634 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.294323  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:52.294864  1559 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.295046  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:52.295586  1706 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:52.295910  1706 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:52.296660  1706 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.297622  1706 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.301074  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:52.302326  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:52.303664  2792 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:52.307354  2792 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:52.308298  2792 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.309214  1706 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:52.310410  1634 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:52.321568  1456 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:52.328228  2813 ts_tablet_manager.cc:933] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:52.333552  2813 tablet_copy_client.cc:323] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:52.335376  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet ab336cb98e24470eb828d9e9fdbcb001 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:52.335973  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet ab336cb98e24470eb828d9e9fdbcb001 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-ab336cb98e24470eb828d9e9fdbcb001
I20260709 13:00:52.340467  2810 ts_tablet_manager.cc:933] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:52.348385  2805 raft_consensus.cc:993] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.349082  2805 raft_consensus.cc:1081] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:52.352097  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:52.352747  1635 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:52.353173  1635 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.354730  2296 raft_consensus.cc:1217] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:52.358467  1644 tablet_copy_source_session.cc:215] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:52.365126  1635 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.370546  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:52.371097  1559 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.371847  1635 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:52.376756  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:52.377491  1704 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:52.377808  1704 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:52.378531  1704 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.379513  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:52.382683  2810 tablet_copy_client.cc:323] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:52.387037  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet a986b2c96ad0416f8fc5927d45ebadc4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:52.387547  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet a986b2c96ad0416f8fc5927d45ebadc4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-a986b2c96ad0416f8fc5927d45ebadc4
I20260709 13:00:52.393256  2813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab336cb98e24470eb828d9e9fdbcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:52.395529  1644 tablet_copy_source_session.cc:215] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:52.404120  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:52.407881  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:52.396366  1559 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:52.410627  2792 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:52.411144  2792 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:52.411990  2792 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:52.417152  2813 tablet_copy_client.cc:806] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:52.417896  2813 tablet_copy_client.cc:670] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:52.422705  2296 raft_consensus.cc:1217] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:52.431923  2810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a986b2c96ad0416f8fc5927d45ebadc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:52.457648  2813 tablet_copy_client.cc:538] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:52.460724  1635 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:52.465906  2769 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:52.487188  1456 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:52.537124  2813 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:52.553174  2810 tablet_copy_client.cc:806] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:52.553929  2810 tablet_copy_client.cc:670] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:00:52.554178  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68337.4us
I20260709 13:00:52.594993  1556 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:52.596065  2810 tablet_copy_client.cc:538] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:52.596765  2769 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:52.605535  2810 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:52.630522  2813 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:52.636385  2813 tablet_bootstrap.cc:492] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:52.637074  2813 ts_tablet_manager.cc:1403] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.104s	user 0.024s	sys 0.006s
I20260709 13:00:52.643493  2810 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:52.644541  2810 tablet_bootstrap.cc:492] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:52.645233  2810 ts_tablet_manager.cc:1403] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.025s
I20260709 13:00:52.645105  2813 raft_consensus.cc:359] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.646148  2813 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:52.646813  2813 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.648753  2810 raft_consensus.cc:359] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.650115  2810 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:52.650823  2810 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:52.653293  2810 ts_tablet_manager.cc:1434] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:52.654855  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-a986b2c96ad0416f8fc5927d45ebadc4 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:52.655287  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-a986b2c96ad0416f8fc5927d45ebadc4 on tablet a986b2c96ad0416f8fc5927d45ebadc4 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:52.680459  2813 ts_tablet_manager.cc:1434] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:00:52.682068  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-ab336cb98e24470eb828d9e9fdbcb001 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:52.682101  1706 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:52.682552  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-ab336cb98e24470eb828d9e9fdbcb001 on tablet ab336cb98e24470eb828d9e9fdbcb001 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:52.683564  2763 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:52.735579  2294 raft_consensus.cc:1217] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:52.780094  1703 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:52.785079  2792 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:52.817675  1560 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:52.820719  2763 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:52.824447  1559 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:52.826085  2763 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:52.836202  1704 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:52.836431  1635 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:52.861310  2769 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:52.872403   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:52.881120  2769 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:52.897572  2792 raft_consensus.cc:1064] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:52.905738  2792 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:52.962168  2296 raft_consensus.cc:1217] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:52.982077  1560 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:52.985766  2763 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:53.001060  1703 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:53.003011  2831 raft_consensus.cc:1064] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:53.003067  2821 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:53.008450  2831 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:53.044454  1703 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.047757  2764 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.049762  2296 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.051335  1560 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.052884  2805 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.055006  2799 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.061883  2294 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.062856  1560 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.068864  2792 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.070797  2763 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.096665  1631 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.109314  2804 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.122501  2799 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.125739  2792 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.130224  1559 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.134482  1560 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.151793  2294 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.159545  1631 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.160755  1457 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.170466  2296 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.177660  1458 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.260748  1703 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:53.268273  2296 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:53.269789  2831 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:53.278124  1560 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:00:53.281554  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 88739.2us
I20260709 13:00:53.282080  2804 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:53.280433  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 89050.7us
I20260709 13:00:53.290205   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:53.314258  2764 raft_consensus.cc:1064] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:53.323870  2764 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:53.329694  2864 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.345819  1703 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.355844  1631 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:00:53.360778  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 66063.2us
I20260709 13:00:53.361850  2296 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:53.362645  2293 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:00:53.365759  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66795.9us
I20260709 13:00:53.368009  2763 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:53.377532  2296 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.383809  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9438873b8e1a4f0780fff8b93a9d89b8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:53.384037  1456 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.375682  2859 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.394899  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 9438873b8e1a4f0780fff8b93a9d89b8 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:53.404433  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:53.405167  1559 raft_consensus.cc:606] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:53.412842  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:53.413843  1559 raft_consensus.cc:606] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:53.420888  1559 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:53.423213  2858 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.428700  1559 raft_consensus.cc:2955] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.421324  1560 raft_consensus.cc:2955] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.453446  2827 raft_consensus.cc:993] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:53.453943  2827 raft_consensus.cc:1081] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:53.471498  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36804
I20260709 13:00:53.472316  1635 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.472728  1635 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.482430  1703 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.483906  1556 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.484571  2293 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:53.485332  2764 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.486303  2860 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.487514  2858 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.517426  1456 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.525573  1635 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.622726  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet af498956d398433a8c293e398bd1255f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:53.623566  1444 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100867us
W20260709 13:00:53.625126  2739 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:00:53.635504  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:53.636296  1634 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
W20260709 13:00:53.640663  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81889.2us
I20260709 13:00:53.647197  2869 tablet_replica.cc:333] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:53.648484  2869 raft_consensus.cc:2243] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.648747  1635 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:53.653409  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet af498956d398433a8c293e398bd1255f with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
W20260709 13:00:53.669905  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 135997us
I20260709 13:00:53.670210  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:53.676682  1703 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.670996  2869 raft_consensus.cc:2272] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.684602  2869 ts_tablet_manager.cc:1916] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.696223  2832 raft_consensus.cc:993] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.696946  2725 raft_consensus.cc:1081] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.698887  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ce0393bceaa489d896dd65113130131"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:53.699997  2878 tablet_replica.cc:333] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:53.699546  1559 raft_consensus.cc:606] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:53.706717  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:53.707281  1556 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:53.707585  1556 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:53.708365  1556 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.709599  1556 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.712575  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:53.713228  1559 raft_consensus.cc:606] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:53.715658  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:53.716189  1559 raft_consensus.cc:606] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:53.717340   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:00:53.718856   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:53.719300  2857 raft_consensus.cc:1064] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
W20260709 13:00:53.720383  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45298.3us
I20260709 13:00:53.721942  2857 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:53.719295  2859 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.740338  1559 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.748427  1703 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:53.749678  1706 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:53.752435  1556 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:53.756644  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:53.760751  2763 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:53.761634  2763 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:53.762501  2763 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.772336  2293 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.777796  2878 raft_consensus.cc:2243] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.778688  2878 raft_consensus.cc:2272] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.784578  1703 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.788431  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:00:53.789047  1704 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.789427  1704 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.789677  2821 raft_consensus.cc:993] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.790127  2821 raft_consensus.cc:1081] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.793970  2879 raft_consensus.cc:993] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.794308  2880 raft_consensus.cc:993] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.794466  2879 raft_consensus.cc:1081] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.796528  1704 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.794803  2880 raft_consensus.cc:1081] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:53.801610  2878 ts_tablet_manager.cc:1916] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.802459  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:53.803056  1556 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:53.803372  1556 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:53.804111  1556 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.804976  1556 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.811286  1556 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:53.812171  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44866
I20260709 13:00:53.812568  1702 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:53.812798  1706 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.813242  1706 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.814873  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:53.816377  2867 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:53.819608  1702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:00:53.816893  2867 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:53.820246  1702 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.820446  1700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ce0393bceaa489d896dd65113130131"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:00:53.822054  1702 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.822499  1700 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:53.822892  1700 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.821573  2792 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:00:53.824862  2867 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.832389  1700 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.839844  1704 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:53.832939  1702 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.841641  1456 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:53.850713  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:53.851284  1628 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.856003  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:53.856550  1559 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:53.856822  1559 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:53.858259  1702 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:53.847051  1700 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:53.861335  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:53.861848  1631 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.862079  1555 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:53.863170  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:53.863797  1635 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.869176  1706 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.872651  2763 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.877332  1706 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:53.877140  1559 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.886768  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:53.887444  1556 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:53.887784  1556 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:53.882071  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:53.890719  1555 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.891623  1631 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:53.896356  1559 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.893482  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:53.898176  1634 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:53.907179  2869 ts_tablet_manager.cc:1929] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.898469  1634 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:53.926383  1458 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:53.923727  2878 ts_tablet_manager.cc:1929] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.941103  1674 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:53.935442  1555 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:00:53.942126  1556 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.928936  1628 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:53.931799  1635 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:53.927484  1559 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:53.952164  2296 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:53.955165  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:53.956053  2831 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:53.956538  2831 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:53.951510  2878 log.cc:1199] T af498956d398433a8c293e398bd1255f P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/af498956d398433a8c293e398bd1255f
I20260709 13:00:53.929006  2869 log.cc:1199] T 9438873b8e1a4f0780fff8b93a9d89b8 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/9438873b8e1a4f0780fff8b93a9d89b8
I20260709 13:00:53.961309  2860 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.957394  2831 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.927884  1634 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:53.965446  1634 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:53.960435  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet af498956d398433a8c293e398bd1255f (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:00:53.945340  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 101397us
I20260709 13:00:53.978346  1675 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:53.987203  1556 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:53.982594  2805 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:54.013073  2876 raft_consensus.cc:993] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:53.970247  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet 9438873b8e1a4f0780fff8b93a9d89b8 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:00:54.027933  1444 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57678.9us
I20260709 13:00:54.026062  1556 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:54.050706  1634 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
W20260709 13:00:54.030090  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56150.5us
I20260709 13:00:54.029490  2864 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:54.031011  2805 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:54.039001  2880 raft_consensus.cc:1081] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:54.112175  2864 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:54.118404  2864 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:00:54.120018  2739 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.136s	user 0.003s	sys 0.000s
I20260709 13:00:54.120975  2805 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:54.164577  2293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57942
I20260709 13:00:54.165134  2293 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.165458  2293 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:54.193439  2296 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.244899  2293 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:54.277834  2293 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:54.292472  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:54.293162  1631 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:54.300815  1700 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.194123  2859 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.307446  1631 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:54.313388  2898 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:00:54.312826  1706 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.334600  1456 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.346714  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:54.350359  2875 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:54.350958  2875 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:54.356443  2875 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:00:54.364482  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 275589us
I20260709 13:00:54.368058  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:54.368896  1559 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:54.369247  1559 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:54.377596  1559 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:00:54.385557  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.390614  1635 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:54.396695  1559 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:54.422215  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 219464us
I20260709 13:00:54.435591  1559 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:54.439713  1560 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.446084  2294 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:54.459930  2857 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:54.464641  1702 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:54.471720  2792 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:00:54.476900  1557 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.483083  2891 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:00:54.480792  1552 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.524770  1631 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.526005  1635 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.533246  2891 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.534761  2891 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:00:54.541982  1560 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:54.543042  2857 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.545629  2866 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:54.548740  1458 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.550626  1458 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 3, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.552246  2885 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.569468  1559 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.558478  1458 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.570773  1702 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.573689  2864 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.588239  2293 raft_consensus.cc:2955] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.590806  1633 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:54.596635  1703 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:54.598995  2898 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:54.604243  2297 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:54.567497  1456 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.642319  1627 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.646277  2763 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:54.654666  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab336cb98e24470eb828d9e9fdbcb001 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:54.647993  2898 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.663254  2905 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:00:54.669078  1705 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.669715  1456 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.686468  1557 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.689783  2905 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.691115  1634 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:54.703814  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a986b2c96ad0416f8fc5927d45ebadc4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:54.703979  2864 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.690922  2297 raft_consensus.cc:2955] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:54.719942   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:54.727329  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet ab336cb98e24470eb828d9e9fdbcb001 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:54.735707  1458 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.844362  1631 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:54.884131  2769 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:55.146711  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 159497us
I20260709 13:00:55.166736   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:55.127830  2920 tablet_replica.cc:333] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:00:55.196436  2739 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:00:55.269821  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72735.6us
W20260709 13:00:55.338941  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 181608us
W20260709 13:00:55.393532  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.009s	sys 0.021s
W20260709 13:00:55.395303  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.010s	sys 0.021s
W20260709 13:00:55.402667  2918 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:00:55.440538  1559 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:55.449653  2920 raft_consensus.cc:2243] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:55.465039  2920 raft_consensus.cc:2272] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:55.472189  2769 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:55.481338  2920 ts_tablet_manager.cc:1916] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:00:55.549319  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58412.5us
W20260709 13:00:55.562162  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 169928us
I20260709 13:00:55.628909  2905 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:00:55.636590  2905 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:55.638798  2905 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:00:55.648197  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 80279.3us
W20260709 13:00:55.670133  1558 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.062s	sys 0.088s
W20260709 13:00:55.676685  1558 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.062s	sys 0.088s
W20260709 13:00:55.688685  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 146971us
W20260709 13:00:55.703235  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 155112us
I20260709 13:00:55.692886  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 705) took 1068 ms. Trace:
W20260709 13:00:55.715366  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 174941us
I20260709 13:00:55.735589  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:00:55.799718  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 172851us
W20260709 13:00:55.818240  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 239839us
W20260709 13:00:55.771039  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 152972us
I20260709 13:00:55.845237  2920 ts_tablet_manager.cc:1929] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:55.846391  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:55.849469  1705 raft_consensus.cc:606] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:55.710376  1558 rpcz_store.cc:276] 0709 13:00:54.620172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.635033 (+ 14861us) service_pool.cc:224] Handling call
0709 13:00:54.635342 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.636451 (+  1109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.636493 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.641242 (+  4749us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.816995 (+175753us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:55.677196 (+860201us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.677563 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.677680 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.688204 (+ 10524us) raft_consensus.cc:1685] finished
0709 13:00:55.688229 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.688465 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":743,"prepare.queue_time_us":195316,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":2830,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":3491,"thread_start_us":1025116,"threads_started":2,"wal-append.queue_time_us":305962}
I20260709 13:00:55.866243  2920 log.cc:1199] T ab336cb98e24470eb828d9e9fdbcb001 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/ab336cb98e24470eb828d9e9fdbcb001
I20260709 13:00:55.913681  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:00:55.978822  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97624.6us
I20260709 13:00:55.998158  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 237) took 1335 ms. Trace:
I20260709 13:00:55.999402  1456 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:56.006332  2920 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51902 (request call id 81) took 1262 ms. Trace:
I20260709 13:00:55.979389  2827 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:00:56.042546  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 128966us
I20260709 13:00:56.016386  1458 rpcz_store.cc:276] 0709 13:00:54.662878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.663589 (+   711us) service_pool.cc:224] Handling call
0709 13:00:55.998037 (+1334448us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:54.748062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:54.917202 (+169140us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:54.920780 (+  3578us) write_op.cc:435] Acquired schema lock
0709 13:00:54.921904 (+  1124us) tablet.cc:662] Decoding operations
0709 13:00:54.987078 (+ 65174us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:54.990739 (+  3661us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:54.993165 (+  2426us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:55.004903 (+ 11738us) tablet.cc:701] Row locks acquired
0709 13:00:55.008057 (+  3154us) write_op.cc:260] PREPARE: finished
0709 13:00:55.008207 (+   150us) write_op.cc:270] Start()
0709 13:00:55.008317 (+   110us) write_op.cc:276] Timestamp: P: 1783602055008166 usec, L: 0
0709 13:00:55.008340 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:00:55.008951 (+   611us) log.cc:844] Serialized 537 byte log entry
0709 13:00:55.029077 (+ 20126us) op_driver.cc:464] REPLICATION: finished
0709 13:00:55.034868 (+  5791us) write_op.cc:301] APPLY: starting
0709 13:00:55.034942 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:55.035047 (+   105us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:55.035134 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:55.035338 (+   204us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:55.035483 (+   145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:55.035529 (+    46us) write_op.cc:312] APPLY: finished
0709 13:00:55.035701 (+   172us) log.cc:844] Serialized 23 byte log entry
0709 13:00:55.049081 (+ 13380us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:55.137093 (+ 88012us) write_op.cc:454] Released schema lock
0709 13:00:55.142989 (+  5896us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:55.545289 (+402300us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b18002b80a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:00:55.545435 (+   146us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:55.545476 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9697,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":1965,"apply.run_cpu_time_us":1795,"apply.run_wall_time_us":155940,"num_ops":1,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":10229,"prepare.run_wall_time_us":275244,"replication_time_us":15102,"spinlock_wait_cycles":2568576}]]}
W20260709 13:00:56.061513  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57722us
I20260709 13:00:56.050884  2827 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.073292  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:00:56.076588  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:56.078019  2897 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:56.082468  2827 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:56.083941  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:00:56.087218  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:56.023936  2920 rpcz_store.cc:276] 0709 13:00:54.722447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.727070 (+  4623us) service_pool.cc:224] Handling call
0709 13:00:55.018569 (+291499us) ts_tablet_manager.cc:1151] Deleting tablet ab336cb98e24470eb828d9e9fdbcb001
0709 13:00:55.022510 (+  3941us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:00:55.508407 (+485897us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:55.845033 (+336626us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:55.982592 (+137559us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127338,"tablet-delete.queue_time_us":276108,"thread_start_us":230954,"threads_started":1}
I20260709 13:00:56.093047  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet ab336cb98e24470eb828d9e9fdbcb001 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:56.112727  1525 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:00:56.127528  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52111.1us
I20260709 13:00:56.117552  2876 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:56.158835  2959 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:00:56.172094  2961 raft_consensus.cc:993] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.172825  2961 raft_consensus.cc:1081] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.175275  2295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57952
I20260709 13:00:56.176050  2295 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.176492  2295 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.177802  2953 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:00:56.178557  2953 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:56.197121  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:00:56.206861  2953 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.216459  2295 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:00:56.220316  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71524.8us
I20260709 13:00:56.221395  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:56.222005  1632 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.249333  1632 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.260793  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet a986b2c96ad0416f8fc5927d45ebadc4 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:56.261747  2920 tablet_replica.cc:333] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:56.262138  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:56.250241  2295 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.260142  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 148) took 1445 ms. Trace:
I20260709 13:00:56.262823  2920 raft_consensus.cc:2243] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:56.263623  2920 raft_consensus.cc:2272] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:56.263072  1456 rpcz_store.cc:276] 0709 13:00:54.814420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.814614 (+   194us) service_pool.cc:224] Handling call
0709 13:00:56.260079 (+1445465us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:56.026385 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:56.027109 (+   724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:56.027155 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:00:56.027182 (+    27us) tablet.cc:662] Decoding operations
0709 13:00:56.027586 (+   404us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:56.027655 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:56.027693 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:56.027932 (+   239us) tablet.cc:701] Row locks acquired
0709 13:00:56.027954 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:00:56.028112 (+   158us) write_op.cc:270] Start()
0709 13:00:56.028245 (+   133us) write_op.cc:276] Timestamp: P: 1783602056028064 usec, L: 0
0709 13:00:56.028283 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:00:56.029008 (+   725us) log.cc:844] Serialized 536 byte log entry
0709 13:00:56.102281 (+ 73273us) op_driver.cc:464] REPLICATION: finished
0709 13:00:56.256179 (+153898us) write_op.cc:301] APPLY: starting
0709 13:00:56.256264 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:56.256370 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:56.256489 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:56.256696 (+   207us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:56.256857 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:56.256902 (+    45us) write_op.cc:312] APPLY: finished
0709 13:00:56.257098 (+   196us) log.cc:844] Serialized 23 byte log entry
0709 13:00:56.257272 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:56.257357 (+    85us) write_op.cc:454] Released schema lock
0709 13:00:56.257394 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":450628,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":153612,"apply.run_cpu_time_us":1543,"apply.run_wall_time_us":7461,"num_ops":1,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":2955,"prepare.run_wall_time_us":2954,"replication_time_us":73841,"spinlock_wait_cycles":42752,"thread_start_us":109730,"threads_started":1}]]}
I20260709 13:00:56.267735  1454 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 266) took 1103 ms. Trace:
I20260709 13:00:56.275045  1454 rpcz_store.cc:276] 0709 13:00:55.164431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.189994 (+ 25563us) service_pool.cc:224] Handling call
0709 13:00:56.267506 (+1077512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298832,"reported_tablets":4}
I20260709 13:00:56.269040  2962 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:56.277550  2962 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:56.278764  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:00:56.280370  2962 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.270004  2920 ts_tablet_manager.cc:1916] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:56.286414  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.286931  1702 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:56.287212  1702 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:56.287863  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:56.287950  1702 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.288475  1703 raft_consensus.cc:606] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:56.289345  1702 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.304028  1702 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.318835  2920 ts_tablet_manager.cc:1929] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:00:56.319336  2920 log.cc:1199] T a986b2c96ad0416f8fc5927d45ebadc4 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/a986b2c96ad0416f8fc5927d45ebadc4
I20260709 13:00:56.323426  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:00:56.327565  1457 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.329051  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet a986b2c96ad0416f8fc5927d45ebadc4 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:56.329937  2953 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:56.340384  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88dfd27965e145dcb810423d7396b403"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:56.341041  1559 raft_consensus.cc:606] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:56.347868  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:56.348500  1704 raft_consensus.cc:606] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:56.351066  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ce0393bceaa489d896dd65113130131"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:56.351493  1704 raft_consensus.cc:606] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:56.353647  1632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:56.354094  1632 raft_consensus.cc:606] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:56.355124   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:00:56.356285   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:56.389967  2905 raft_consensus.cc:993] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.390630  2966 raft_consensus.cc:1081] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.392150  2295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57952
I20260709 13:00:56.392552  2295 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.392771  2295 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.397545  2295 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.399749  2295 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.400290  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:56.400900  1632 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.401139  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.401666  1704 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:56.401957  1704 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:56.402678  1704 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.403841  1704 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.406926  1632 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.408177  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:56.408933  2962 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:56.409363  2962 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:56.410184  2962 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.410952  1704 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.415985  2894 raft_consensus.cc:993] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.416589  2932 raft_consensus.cc:1081] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.418445  2295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57934
I20260709 13:00:56.418982  2295 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.419324  2295 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.420531  1457 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.424870  2295 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.429970  2295 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.436233  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:56.436793  1632 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:56.437089  1632 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:56.437798  1632 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.438910  1632 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.443815  1632 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.444674  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:56.445443  2962 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:56.445348  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.446040  2962 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:56.446102  1704 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.446846  2962 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.451512  1704 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.456171  1457 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.460013  2953 raft_consensus.cc:993] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.460501  2953 raft_consensus.cc:1081] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.461786  2296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88dfd27965e145dcb810423d7396b403"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57942
I20260709 13:00:56.462301  2296 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.462620  2296 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.468099  2296 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.472277  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:56.472474  2296 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.472843  1559 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:56.473156  1559 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:56.473800  1559 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.474658  1559 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.477876  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.478441  1702 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.480422  1559 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.486088  1702 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.487139  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:00:56.488667  2962 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:56.489145  2962 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:56.489971  2962 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.505363  1457 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.702454  2905 raft_consensus.cc:993] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.702898  2966 raft_consensus.cc:1081] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:56.704289  2296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57952
I20260709 13:00:56.704769  2296 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.705021  2296 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.712039  2296 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.713351  2905 raft_consensus.cc:993] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:56.713892  2966 raft_consensus.cc:1081] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:56.714119  2296 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.715035  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:56.715473  1559 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.716450  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.716866  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ce0393bceaa489d896dd65113130131"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:56.716984  1704 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:56.717329  1704 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:56.717428  1632 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:56.718256  1632 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:56.718044  1704 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.719192  1704 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:56.721469  1559 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.722461  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:56.725095  1704 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:56.725368  2962 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:56.725915  2962 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:56.726725  2962 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:56.730667  1632 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:56.732602  1632 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:56.733485  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:56.734035  1702 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:56.734287  1702 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:56.734947  1702 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:56.735947  1702 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:56.737649  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:56.738397  1555 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:56.744987  1457 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.746121  1702 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:00:56.749939  1555 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:00:56.750933  1598 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:56.753109  2932 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:56.761194  2932 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:56.762200  2932 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:56.774248  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 3, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.799105  1627 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.800439  2962 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:56.816896  1702 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.819253  2928 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:56.848361  1702 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.849594  2978 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:56.866919  1703 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.866919  1627 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.868484  2962 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:56.868631  2976 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:56.888599  1632 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.893458  2962 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:56.996881  1559 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:56.998092  2978 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:57.008793  1703 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:57.015355  2978 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:57.233583  1703 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:57.234741  2978 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:57.243139  1559 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:57.245213  2977 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:57.265198  1703 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:57.266680  2950 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.290740  1559 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:57.292166  2992 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.357118   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:57.405294  1704 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:57.407486   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:57.411609  1629 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:57.412983  1673 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e008100b0d36427c939df1036b5e0b67. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:57.413177  1559 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:57.414139  2891 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:57.415073  2905 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:57.427161  2891 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.429051  1629 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.437188  1559 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.451364  1458 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.453226  1629 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:57.466424  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21235338303a443e8f10b8773c68e2db"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:57.467319  1559 raft_consensus.cc:606] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:57.467799  1704 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:57.471464  1555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:57.472079  1555 raft_consensus.cc:606] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:57.473778  2932 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:57.476131  1599 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2dc331078c54a6aa71b5eab7cb93250. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:57.476011  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9df64a488003414aac04146e56487c81"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:57.477149  1559 raft_consensus.cc:606] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:57.483852  1629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:57.484408  1629 raft_consensus.cc:606] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:57.486148  1555 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:57.488610  1629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af498956d398433a8c293e398bd1255f"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:57.489185  1629 raft_consensus.cc:606] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:00:57.490319  2950 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:57.491142   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:57.491680  2932 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.492874   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:57.504269  1702 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.514959  1629 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:57.515164  1457 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.529878  1703 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:57.531376  2950 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:57.533296  1599 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ce0393bceaa489d896dd65113130131. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:57.536605  1555 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.540877  3004 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.544349  1558 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:57.544986  1703 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.558007  1627 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:57.559621  2950 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.566045  1555 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
W20260709 13:00:57.576321  1599 consensus_peers.cc:597] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd3b4e5c2bf43a7a1a186ee984fa7c4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:57.577601  1702 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:57.580436  1558 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:57.580106  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 4, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.582154  2932 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.585219  2932 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.606014  2932 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.608635  1558 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.613628  1702 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.622617  1454 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:00:57.692616  2932 raft_consensus.cc:993] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:57.693111  3004 raft_consensus.cc:1081] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:57.694624  1555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af498956d398433a8c293e398bd1255f"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:00:57.695135  1555 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.695493  1555 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.701961  1555 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.703835  1555 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:57.704665  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:57.704764  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:00:57.705369  1629 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:57.705418  2296 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.705669  1629 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:57.706490  1629 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.707340  1629 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.711601  2296 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:57.712810  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:57.713531  2826 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:57.713670  2832 raft_consensus.cc:993] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:57.714038  2826 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:57.714344  2832 raft_consensus.cc:1081] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:57.714944  2826 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.716208  1629 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:57.717561  1627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21235338303a443e8f10b8773c68e2db"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36804
I20260709 13:00:57.718116  1627 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.718451  1627 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.724623  1627 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.726790  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:57.727322  1555 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:57.727627  1555 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:57.728358  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:00:57.728497  1555 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.728964  2296 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.729570  1555 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.729525  1457 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.734113  1627 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:57.736208  1555 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:57.737921  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:57.738214  2932 raft_consensus.cc:993] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:57.738662  2932 raft_consensus.cc:1081] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:57.739776  2992 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:57.740252  2992 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:57.741074  2992 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.744438  1702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44866
I20260709 13:00:57.745023  1702 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.745342  1702 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.752151  1702 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.753448  2296 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:57.754537  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:57.755095  1629 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:57.755398  1629 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:57.755811  1702 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:57.756116  1629 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.757140  1629 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.757736  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:00:57.758261  2296 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:57.762984  1457 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.764214  2296 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:00:57.763491  1629 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:00:57.768843  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:00:57.769683  2905 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:57.770136  2905 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:57.770920  2905 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.786942  1457 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 3, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:57.862744  2826 raft_consensus.cc:993] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:57.863328  2953 raft_consensus.cc:1081] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:00:57.864787  2295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9df64a488003414aac04146e56487c81"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57942
I20260709 13:00:57.865237  2295 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.865501  2295 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.870620  2295 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.872969  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:57.873518  1555 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:57.873812  1555 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:57.874176  2832 raft_consensus.cc:993] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:57.874648  2832 raft_consensus.cc:1081] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:57.874567  1555 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.876757  2295 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:57.878732  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:00:57.879316  1704 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:57.879693  1704 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.880720  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:57.880245  1555 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.882679  1629 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.884580  3011 ts_tablet_manager.cc:933] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:57.889984  1704 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:57.892542  1629 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:57.893594  3011 tablet_copy_client.cc:323] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:00:57.894462  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:57.895390  2962 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:57.896085  2962 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:00:57.897059  2962 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:57.899529  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet e008100b0d36427c939df1036b5e0b67 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:00:57.900188  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet e008100b0d36427c939df1036b5e0b67 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-e008100b0d36427c939df1036b5e0b67
I20260709 13:00:57.895442  1555 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:00:57.910648  1704 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:57.913553  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:57.914997  1629 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.924363  1457 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:57.925411  1629 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:57.926715  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:57.928874  2905 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:57.929363  2905 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:57.930261  2905 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:57.936015  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:57.936618  1555 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:57.936949  1555 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:57.910518  1719 tablet_copy_source_session.cc:215] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:57.937736  1555 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:57.939169  1555 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:57.946117  1555 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:57.948807  3015 ts_tablet_manager.cc:933] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:57.953775  3011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e008100b0d36427c939df1036b5e0b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:57.954711  3015 tablet_copy_client.cc:323] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:57.957271  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet 9ce0393bceaa489d896dd65113130131 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:57.957890  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet 9ce0393bceaa489d896dd65113130131 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9ce0393bceaa489d896dd65113130131
I20260709 13:00:57.966809  1457 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:57.978061  3017 ts_tablet_manager.cc:933] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:57.975674  1644 tablet_copy_source_session.cc:215] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:57.986891  3011 tablet_copy_client.cc:806] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:57.987601  3011 tablet_copy_client.cc:670] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:57.988504  3017 tablet_copy_client.cc:323] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:57.991938  3015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0393bceaa489d896dd65113130131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:57.992656  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:57.997982  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-abd3b4e5c2bf43a7a1a186ee984fa7c4
I20260709 13:00:58.001992  3011 tablet_copy_client.cc:538] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.018477  3011 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:58.020180  3015 tablet_copy_client.cc:806] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.020816  3015 tablet_copy_client.cc:670] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.008327  1644 tablet_copy_source_session.cc:215] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.026149  3017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.039374  3015 tablet_copy_client.cc:538] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.047804  3011 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.048797  3011 tablet_bootstrap.cc:492] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:58.049708  3011 ts_tablet_manager.cc:1403] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.032s	user 0.025s	sys 0.003s
I20260709 13:00:58.052946  3011 raft_consensus.cc:359] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.053786  3011 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:58.054453  3011 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.056547  3017 tablet_copy_client.cc:806] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.057235  3017 tablet_copy_client.cc:670] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.057552  3011 ts_tablet_manager.cc:1434] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:58.059599  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-e008100b0d36427c939df1036b5e0b67 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:00:58.060067  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-e008100b0d36427c939df1036b5e0b67 on tablet e008100b0d36427c939df1036b5e0b67 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:58.051546  3015 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:58.078683  3017 tablet_copy_client.cc:538] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.080478  3011 ts_tablet_manager.cc:933] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:58.096153  3011 tablet_copy_client.cc:323] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:00:58.097970  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet c2dc331078c54a6aa71b5eab7cb93250 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:00:58.098577  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet c2dc331078c54a6aa71b5eab7cb93250 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-c2dc331078c54a6aa71b5eab7cb93250
I20260709 13:00:58.100018  3017 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:58.106488  3015 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.106534  1644 tablet_copy_source_session.cc:215] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.107663  3015 tablet_bootstrap.cc:492] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:58.108541  3015 ts_tablet_manager.cc:1403] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.057s	user 0.032s	sys 0.007s
I20260709 13:00:58.110124  3011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2dc331078c54a6aa71b5eab7cb93250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.111996  3015 raft_consensus.cc:359] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.112951  3015 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:58.113660  3015 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.133597  3015 ts_tablet_manager.cc:1434] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:00:58.135106  3011 tablet_copy_client.cc:806] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.135753  3011 tablet_copy_client.cc:670] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.139600  3011 tablet_copy_client.cc:538] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.149634  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9ce0393bceaa489d896dd65113130131 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:58.150444  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9ce0393bceaa489d896dd65113130131 on tablet 9ce0393bceaa489d896dd65113130131 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:58.154737  3011 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:00:58.160954  3017 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.161986  3017 tablet_bootstrap.cc:492] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:58.162953  3017 ts_tablet_manager.cc:1403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.063s	user 0.039s	sys 0.020s
I20260709 13:00:58.163442  2296 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.164361  1629 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.165836  2992 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.166674  3017 raft_consensus.cc:359] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.167598  3017 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:58.168318  3017 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.168721  2953 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.171968  3017 ts_tablet_manager.cc:1434] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:58.176236  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-abd3b4e5c2bf43a7a1a186ee984fa7c4 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:58.176632  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-abd3b4e5c2bf43a7a1a186ee984fa7c4 on tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:58.258776  2295 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.264572  2295 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:58.265822  2953 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:58.267931  2995 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:58.274341  1555 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.280568  3011 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.282110  3011 tablet_bootstrap.cc:492] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:00:58.282877  3011 ts_tablet_manager.cc:1403] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.128s	user 0.039s	sys 0.008s
I20260709 13:00:58.295418  3011 raft_consensus.cc:359] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.299057  3011 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:00:58.301262  3011 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:00:58.303020  3011 ts_tablet_manager.cc:1434] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:00:58.317198  2992 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.318111  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-c2dc331078c54a6aa71b5eab7cb93250 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:00:58.318552  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-c2dc331078c54a6aa71b5eab7cb93250 on tablet c2dc331078c54a6aa71b5eab7cb93250 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:58.333751  1629 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:58.338459  2891 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:58.392561  1558 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:58.396661  2905 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:58.409085  2295 raft_consensus.cc:1217] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:00:58.423377  2295 raft_consensus.cc:1217] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:58.438614  2296 raft_consensus.cc:1217] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:58.442744  1627 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.444139  2296 raft_consensus.cc:1217] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:58.444854  2977 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.448239  1629 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:58.450407  2891 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:58.528409   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:58.534067  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68520.9us
I20260709 13:00:58.672549  1555 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:58.685658  2976 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.734647   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:58.790701  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "334958c5aee349629d63e898f13ba95b"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:58.791344  1627 raft_consensus.cc:606] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:00:58.795207  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:58.796159  1703 raft_consensus.cc:606] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:58.799212  3032 raft_consensus.cc:1064] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:58.801652  3032 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.804111  1554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:58.804559  1554 raft_consensus.cc:606] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:58.805702   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:00:58.807336   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:58.812336  2295 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:58.813381  1703 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:58.813630  3047 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.814877  3031 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.815323  1554 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:58.817948  3031 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:58.879884  1554 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:58.886660  3004 raft_consensus.cc:993] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:58.887240  3004 raft_consensus.cc:1081] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:58.880219  3047 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:58.891975  1555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "334958c5aee349629d63e898f13ba95b"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:00:58.893647  1555 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:58.894004  1555 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:58.903981  1555 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.911614  1555 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:00:58.913223  1703 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:58.914554  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:58.914746  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:00:58.915130  1627 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:58.915396  2295 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:58.915495  1627 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:58.916368  1627 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.917658  1627 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:58.918756  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 4 to 5, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:58.923604  1627 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:58.925112  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:58.925834  2953 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:58.926335  2953 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:58.927044  2953 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.929527  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:00:58.947542  2295 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:58.959571  2932 raft_consensus.cc:1064] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:58.961982  2932 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.974156  3044 raft_consensus.cc:1064] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:58.977638  3044 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:58.996153  1627 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:59.001986  1555 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:59.005342  3047 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:59.008246  1702 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.008893  2297 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.011391  2992 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.012283  3004 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.044917  3064 raft_consensus.cc:1064] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:00:59.046401  3045 raft_consensus.cc:993] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:59.046954  3045 raft_consensus.cc:1081] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:00:59.049599  3064 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:59.054327  1702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:00:59.065514  2297 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.066222  1629 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.067862  2995 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.068387  1555 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.069926  2296 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:59.071143  2995 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.074312  2992 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:00:59.086601  2995 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.087172  3032 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.094712  1456 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.097347  2296 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.105822  2891 raft_consensus.cc:993] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:59.106587  2891 raft_consensus.cc:1081] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:00:59.107185  2297 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:59.107445  1703 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:59.108640  1629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:00:59.109489  1629 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:59.110024  1629 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.110292  3064 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:59.115664  3047 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.148142  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9ce0393bceaa489d896dd65113130131 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:59.149931  1454 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 5 to 6, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.153448  1629 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.054953  1702 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:59.156710  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:59.157354  1703 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:59.157706  1703 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:59.158521  1703 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.158644  1627 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.159652  1703 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.163241  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ce0393bceaa489d896dd65113130131 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44858
W20260709 13:00:59.165570  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123437us
I20260709 13:00:59.167220  1555 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.168458  2891 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.175738  1555 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:59.177322  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:59.178069  1555 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.177275  3072 tablet_replica.cc:333] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:59.179819  3072 raft_consensus.cc:2243] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:59.187593  3047 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.156920  1702 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.189733  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:59.182787  3032 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:00:59.185011  3072 raft_consensus.cc:2272] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:59.199647  1555 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:00:59.200366  1629 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:59.200017  3047 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.201848  1558 raft_consensus.cc:2955] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.202775  1700 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.211025  3072 ts_tablet_manager.cc:1916] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:59.219868  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:59.224706  3063 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:59.226877  3063 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:00:59.227738  3063 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.232970  3072 ts_tablet_manager.cc:1929] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:00:59.233531  3072 log.cc:1199] T 9ce0393bceaa489d896dd65113130131 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/9ce0393bceaa489d896dd65113130131
I20260709 13:00:59.237516  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 9ce0393bceaa489d896dd65113130131 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:59.236792  2297 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.261109  1554 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.264156  1458 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.225178  1457 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.305948  2932 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.308188  1706 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.319815  1554 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.325764  1555 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.325843  2297 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.345896  1456 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.383509  1457 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.385437  1702 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.392753  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:00:59.393347  1555 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:59.393637  1555 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:59.394327  1555 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.398020  1555 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.395527  1702 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:00:59.403472  1629 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:59.402969  2297 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.406574  1555 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:59.407729  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:00:59.409000  2905 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:59.409531  2905 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:59.410396  2905 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.410974  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:59.411522  1627 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:59.417912  1627 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:00:59.421430  2295 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.423215  3047 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.434531  3031 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.436262  2295 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.437125  1706 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:59.439464  2296 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:59.441735  1627 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:00:59.442378  1558 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.442935  2953 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:59.444357  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c2dc331078c54a6aa71b5eab7cb93250 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:59.445864  2296 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.446094  1554 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.449407  3063 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.451604  3044 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.453379  3044 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.455987  1554 raft_consensus.cc:2955] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.468696  1456 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 3 to 4, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:59.472414  1558 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.473389  2297 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.469282  2995 raft_consensus.cc:2955] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.496953  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet c2dc331078c54a6aa71b5eab7cb93250 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44858
I20260709 13:00:59.497867  3072 tablet_replica.cc:333] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:00:59.498996  3072 raft_consensus.cc:2243] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:59.499718  3072 raft_consensus.cc:2272] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:59.503793  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e008100b0d36427c939df1036b5e0b67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:59.504679  3072 ts_tablet_manager.cc:1916] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:59.512004  2297 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:59.513504  1706 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:59.513603  3031 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:59.515377  2932 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:59.521492  2932 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.524094  1706 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.534544  3072 ts_tablet_manager.cc:1929] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:59.545885  3072 log.cc:1199] T c2dc331078c54a6aa71b5eab7cb93250 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/c2dc331078c54a6aa71b5eab7cb93250
I20260709 13:00:59.547474  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet c2dc331078c54a6aa71b5eab7cb93250 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:59.521711  1456 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 3 to 4, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:59.549005  1627 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:59.550849  2953 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:59.549065  1456 catalog_manager.cc:5697] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:59.554678  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:59.556351  2295 raft_consensus.cc:2955] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:00:59.575645  1457 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index 4 to 5, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:59.583503  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet e008100b0d36427c939df1036b5e0b67 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36790
I20260709 13:00:59.590898  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51902
I20260709 13:00:59.602062  3083 tablet_replica.cc:333] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:00:59.603852  3083 raft_consensus.cc:2243] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:59.604465  3083 raft_consensus.cc:2272] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:59.605810  3084 tablet_replica.cc:333] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:00:59.608340  3084 raft_consensus.cc:2243] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:59.608886  3083 ts_tablet_manager.cc:1916] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:59.609858  3084 raft_consensus.cc:2272] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:59.614216  3084 ts_tablet_manager.cc:1916] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:59.626371  3083 ts_tablet_manager.cc:1929] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:59.626947  3083 log.cc:1199] T e008100b0d36427c939df1036b5e0b67 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/e008100b0d36427c939df1036b5e0b67
I20260709 13:00:59.628727  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet e008100b0d36427c939df1036b5e0b67 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:59.629616  3084 ts_tablet_manager.cc:1929] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:00:59.629944  3084 log.cc:1199] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/abd3b4e5c2bf43a7a1a186ee984fa7c4
I20260709 13:00:59.631094  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet abd3b4e5c2bf43a7a1a186ee984fa7c4 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:00:59.662029  1558 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:59.663240  2932 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:59.675487  1703 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:59.676819  3031 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:59.808331   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:59.816804  1627 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:59.818688  2891 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.830857  1554 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:59.834867  2891 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:59.894064   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:59.935348  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:59.935899  1703 raft_consensus.cc:606] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:00:59.938056  1554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78699c8b654944aa9788917e2a177d14"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:00:59.938527  1554 raft_consensus.cc:606] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:00:59.940593  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:00:59.941022  1703 raft_consensus.cc:606] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:00:59.943073  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:00:59.943535  1627 raft_consensus.cc:606] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:00:59.944659   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:59.946138   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:59.948523  2891 raft_consensus.cc:993] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:59.948908  3044 raft_consensus.cc:1081] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:00:59.950178  1554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51928
I20260709 13:00:59.950521  1554 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:59.950735  1554 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:59.955324  1554 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.957085  1554 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:00:59.957861  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:00:59.958135  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:00:59.958432  1627 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:59.958642  1703 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:59.958943  1703 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:00:59.959648  1703 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.960768  1703 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:59.964706  1627 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:00:59.965637  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:00:59.966303  3048 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:59.966742  3048 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:00:59.967061  1703 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:00:59.967461  3048 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:00:59.977113  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 3, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 3 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.059975  3031 raft_consensus.cc:993] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:01:00.060520  2932 raft_consensus.cc:1081] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:01:00.061995  1703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44866
I20260709 13:01:00.062479  1703 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.062774  1703 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:00.067165  1703 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.069172  1703 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:00.069447  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:00.069669  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:00.070066  1627 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:00.070134  2295 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:00.070432  1627 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:00.071153  1627 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.072299  1627 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:00.076673  2295 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:00.077867  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:00.078477  1627 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:00.078984  2891 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:00.079484  2891 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:00.080314  2891 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.088406  1457 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 3, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.219702  3048 raft_consensus.cc:993] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:01:00.220263  2953 raft_consensus.cc:1081] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:01:00.222169  2295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78699c8b654944aa9788917e2a177d14"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57942
I20260709 13:01:00.222708  2295 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.223093  2295 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.229280  2295 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.230901  2295 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:00.232033  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:00.232437  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:00.232628  1558 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.233074  1558 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:00.233109  1702 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.233582  1558 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.234377  1558 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.238679  1702 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:00.240885  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:00.241912  2976 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.242465  2976 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:00.242648  1558 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:00.243420  2976 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:00.254232  1457 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.357453  3044 raft_consensus.cc:993] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:01:00.357910  2995 raft_consensus.cc:1081] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:01:00.359232  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:01:00.359668  1632 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.359923  1632 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.364228  1632 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:00.365665  1632 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:00.366616  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:00.366616  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:00.367483  1558 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.367620  1702 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.368041  1702 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:00.368777  1702 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:00.369680  1702 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.372728  1558 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:00.373974  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:00.374544  1702 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:00.374799  3031 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.375365  3031 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:00.376466  3031 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:00.386608  1457 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.482805  1627 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:00.484031  2995 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:00.491508  2295 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:00.494442  2891 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:00.511785  1627 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:00.520270  3048 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.562003  1703 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:00.572902  2953 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:00.659754  1702 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:00.665467  1627 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:00.668118  1558 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:00.669456  3106 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.670398  3044 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:01:00.674719  1673 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4b89801fe624f179701cbd4b98f1490. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:00.683060  3044 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.683508  1627 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.686720  1558 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.703312  1454 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.705797  1627 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:00.719667  1703 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:00.719913  1702 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:00.721777  1599 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b765b4fb8a24da18c3dc1ec41dd5e80. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:00.722915  2976 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:00.722988  2992 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:00.723496  1558 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:00.724908  3031 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.727123  1554 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:00.730468  2976 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:00.761842  2992 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.765137  1554 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.770438  1702 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.781102  1706 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:00.783674  1454 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 3, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.803794  1554 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:00.806324  3031 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:00.820129  1627 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:00.821597  1673 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9d92cdb4b624c3bbc074afe1340b761. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:00.823021  2995 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:00.830807  1558 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:00.832860  2995 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:00.857908  3044 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.863340  1702 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:00.864748  1558 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.864883  1632 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.872421  1702 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:00.881773  2992 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:00.883456  1455 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.882066  1627 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:00.884832  1673 consensus_peers.cc:597] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c -> Peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Couldn't send request to peer 2579bae7bf1543bf9b10d68e93a6ebbb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ab7f997f0f54ab6bfebcbbd01c136d6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:00.885705  3044 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:00.888412  1558 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:00.889582  3044 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:00.897215  1629 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.895509  3044 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.907001  1558 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.918787  1458 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 reported cstate change: config changed from index -1 to 2, NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } } }
I20260709 13:01:00.946940   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:01.061264   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:01.139575  1558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:01:01.140208  1558 raft_consensus.cc:606] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:01.142004  3124 ts_tablet_manager.cc:933] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:01.143929  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:01.144727  1702 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:01:01.152256  1632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:01:01.152956  1632 raft_consensus.cc:606] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:01.154847   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:01.155151  3124 tablet_copy_client.cc:323] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:39515
I20260709 13:01:01.156853  1644 tablet_copy_service.cc:140] P 8b2c3934407d422495728f84374c6b53: Received BeginTabletCopySession request for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:42818)
I20260709 13:01:01.157147   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:01.157393  1644 tablet_copy_service.cc:161] P 8b2c3934407d422495728f84374c6b53: Beginning new tablet copy session on tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:42818: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-9b765b4fb8a24da18c3dc1ec41dd5e80
I20260709 13:01:01.164750  1644 tablet_copy_source_session.cc:215] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:01.167536  3124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:01.182675  3128 ts_tablet_manager.cc:933] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:01.183696  3124 tablet_copy_client.cc:806] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:01.184304  3124 tablet_copy_client.cc:670] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:01.188338  3124 tablet_copy_client.cc:538] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:01.189615  3126 raft_consensus.cc:993] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: : Instructing follower 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:01:01.190094  3097 raft_consensus.cc:1081] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Signalling peer 00f3fbc60b884078a3c219909b44f64c to start an election
I20260709 13:01:01.192822  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:44862
I20260709 13:01:01.193316  1706 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:01.193655  1706 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.198917  3124 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:01:01.205694  1706 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:01.208250  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:01.208786  1558 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:01.209062  1558 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:01.209759  1558 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:01.210781  1558 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.218935  3128 tablet_copy_client.cc:323] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:01:01.221055  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet c4b89801fe624f179701cbd4b98f1490 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:01:01.221583  1558 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:01.221589  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet c4b89801fe624f179701cbd4b98f1490 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-c4b89801fe624f179701cbd4b98f1490
I20260709 13:01:01.222415  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:01.223189  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:01.223409  2296 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.224102  3044 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:01.224565  3044 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:01.225391  3044 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:01.229480  1719 tablet_copy_source_session.cc:215] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:01.221369  1706 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:01.233090  3128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b89801fe624f179701cbd4b98f1490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:01.242609  3132 ts_tablet_manager.cc:933] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:01.259578  3124 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:01.260591  3124 tablet_bootstrap.cc:492] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:01:01.261255  3124 ts_tablet_manager.cc:1403] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.063s	user 0.021s	sys 0.009s
I20260709 13:01:01.263070  3044 raft_consensus.cc:993] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:01:01.263537  3044 raft_consensus.cc:1081] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Signalling peer 8b2c3934407d422495728f84374c6b53 to start an election
I20260709 13:01:01.265904  1629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:36806
I20260709 13:01:01.266392  1629 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:01.266709  1629 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:01.269907  3132 tablet_copy_client.cc:323] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:01:01.272286  3124 raft_consensus.cc:359] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.273339  3124 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:01:01.273809  3128 tablet_copy_client.cc:806] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:01.274477  3128 tablet_copy_client.cc:670] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:01.274031  3124 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.276356  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet f9d92cdb4b624c3bbc074afe1340b761 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:01:01.276878  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet f9d92cdb4b624c3bbc074afe1340b761 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-f9d92cdb4b624c3bbc074afe1340b761
I20260709 13:01:01.277383  3124 ts_tablet_manager.cc:1434] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:01:01.280457  1644 tablet_copy_service.cc:342] P 8b2c3934407d422495728f84374c6b53: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9b765b4fb8a24da18c3dc1ec41dd5e80 received from {username='slave'} at 127.0.0.1:42818
I20260709 13:01:01.280907  1644 tablet_copy_service.cc:434] P 8b2c3934407d422495728f84374c6b53: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-9b765b4fb8a24da18c3dc1ec41dd5e80 on tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:01.275537  1457 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.286273  1719 tablet_copy_source_session.cc:215] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:01.289640  3132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d92cdb4b624c3bbc074afe1340b761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:01.290832  2296 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:01.296662  3128 tablet_copy_client.cc:538] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:01.305133  1629 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:01.307279  1629 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:01.308426  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:01.309120  1706 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:01.309418  1706 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:01.310144  1706 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:01.310945  3128 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:01:01.311300  1706 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:01.312559  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:01.313115  1558 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:01.317876  1706 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:01.319286  1558 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:01.319283  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:01.321321  3117 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:01.321828  3117 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:01.323134  3117 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:01.345948  1457 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 3 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.355568  3132 tablet_copy_client.cc:806] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:01.356261  3132 tablet_copy_client.cc:670] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:01.356406  3124 ts_tablet_manager.cc:933] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Initiating tablet copy from peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:01.362864  3132 tablet_copy_client.cc:538] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:01.375191  3128 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:01.376613  3128 tablet_bootstrap.cc:492] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:01:01.378197  3124 tablet_copy_client.cc:323] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45021
I20260709 13:01:01.379593  3128 ts_tablet_manager.cc:1403] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.018s
I20260709 13:01:01.382097  1719 tablet_copy_service.cc:140] P 00f3fbc60b884078a3c219909b44f64c: Received BeginTabletCopySession request for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 from peer 2579bae7bf1543bf9b10d68e93a6ebbb ({username='slave'} at 127.0.0.1:58846)
I20260709 13:01:01.382653  1719 tablet_copy_service.cc:161] P 00f3fbc60b884078a3c219909b44f64c: Beginning new tablet copy session on tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 from peer 2579bae7bf1543bf9b10d68e93a6ebbb at {username='slave'} at 127.0.0.1:58846: session id = 2579bae7bf1543bf9b10d68e93a6ebbb-0ab7f997f0f54ab6bfebcbbd01c136d6
I20260709 13:01:01.382975  3132 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:01:01.382772  3128 raft_consensus.cc:359] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.383998  3128 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:01:01.384654  3128 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.387152  3128 ts_tablet_manager.cc:1434] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:01.389083  1718 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-c4b89801fe624f179701cbd4b98f1490 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:01:01.389488  1718 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-c4b89801fe624f179701cbd4b98f1490 on tablet c4b89801fe624f179701cbd4b98f1490 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:01.397379  1719 tablet_copy_source_session.cc:215] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:01.403367  3124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:01.422431  3124 tablet_copy_client.cc:806] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:01.423019  3124 tablet_copy_client.cc:670] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:01.427552  3124 tablet_copy_client.cc:538] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:01.435779  3132 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:01.436751  3132 tablet_bootstrap.cc:492] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:01:01.436934  3124 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap starting.
I20260709 13:01:01.437441  3132 ts_tablet_manager.cc:1403] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.022s
I20260709 13:01:01.440533  3132 raft_consensus.cc:359] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.441473  3132 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:01:01.444813  3132 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.454348  3132 ts_tablet_manager.cc:1434] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:01:01.457185  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-f9d92cdb4b624c3bbc074afe1340b761 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:01:01.457607  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-f9d92cdb4b624c3bbc074afe1340b761 on tablet f9d92cdb4b624c3bbc074afe1340b761 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:01.483444  3124 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:01.484359  3124 tablet_bootstrap.cc:492] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Bootstrap complete.
I20260709 13:01:01.485039  3124 ts_tablet_manager.cc:1403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.016s
I20260709 13:01:01.488230  3124 raft_consensus.cc:359] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.488971  3124 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Initialized, Role: LEARNER
I20260709 13:01:01.489583  3124 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:01.492022  3124 ts_tablet_manager.cc:1434] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:01.494223  1719 tablet_copy_service.cc:342] P 00f3fbc60b884078a3c219909b44f64c: Request end of tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-0ab7f997f0f54ab6bfebcbbd01c136d6 received from {username='slave'} at 127.0.0.1:58846
I20260709 13:01:01.494585  1719 tablet_copy_service.cc:434] P 00f3fbc60b884078a3c219909b44f64c: ending tablet copy session 2579bae7bf1543bf9b10d68e93a6ebbb-0ab7f997f0f54ab6bfebcbbd01c136d6 on tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with peer 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:01.582094  2295 raft_consensus.cc:1217] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:01.592541  3117 raft_consensus.cc:993] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:01:01.593146  3117 raft_consensus.cc:1081] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:01:01.638443  2295 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:01.652634  2891 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:02.177943   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:02.661211  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8d20 after lost signal to thread 434
I20260709 13:01:02.703181  3144 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.704217  3144 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.707244  3144 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.866442  3144 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.804976  3141 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:02.867120  3152 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.867429  3141 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.839222  3148 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.868764  3148 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.870290  3141 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.850780  3117 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.855010  3099 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:02.861235  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002cdba0 after lost signal to thread 1492
I20260709 13:01:02.871413  3117 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.871752  3099 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.867750  3152 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:02.867153  3144 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.865641  3142 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.876163  3099 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.876390  3142 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.884112  3117 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.884562  3148 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.886824  3117 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.887430  3148 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.887982  3142 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.887969  3148 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.887638  3117 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:02.888701  3142 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.889181  3142 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:02.890072  3148 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.891122  3142 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.897229  3153 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:02.893334  3117 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.902488  3148 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.904115  3148 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.905166  3152 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.892980  3142 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:02.906526  3148 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:02.906426  3142 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.910749  3141 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:02.924192  3141 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.915194  3142 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.915522  3144 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.932550  3158 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:02.914882  3153 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.933966  3158 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.936194  3158 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.992564  3162 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:02.994204  3142 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.996358  3158 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:02.997738  3152 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:02.998104  3156 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.997428  3142 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.985740  3140 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.999058  3152 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:02.999730  3140 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.984062  2992 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.002810  3144 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:02.997687  3158 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.005846  3099 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:02.984066  3097 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:02.984097  3126 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.007055  3145 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.007658  3148 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.009393  3147 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.010255  2992 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:02.999066  3156 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.012800  3140 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.015340  3152 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:02.997430  3149 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.001386  3142 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.021940  3099 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.003597  3162 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.022995  3145 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.025144  3148 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.025275  3097 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.026489  3147 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.029659  3126 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.035984  3149 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.023531  3158 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.045766  3156 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.042078  3144 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.030272  3145 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.048189  3145 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.048491  3158 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.048875  3145 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.049237  3158 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.049482  3144 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.053308  3161 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.055541  3148 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.037199  3153 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.059063  3145 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.064100  2992 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.054057  3161 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.074486  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.075666  1559 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:03.077665  3160 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.077494  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.081439  3099 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.070272  3153 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.083213  3153 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.085309  3147 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.072126  3145 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.086894  3153 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.086526  3145 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.087953  3149 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.078438  3160 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.088742  2297 raft_consensus.cc:1217] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:03.092669  3145 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.077510  3161 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.083491  3162 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.099128  3159 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.093211  3099 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.098515  3149 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.124056  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.126209  3160 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.113749  3163 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.130280  3166 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.105532  3159 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:03.110543  3145 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.100198  3153 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.136355  1631 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.129608  3162 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.130290  3147 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.139163  3162 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.140933  3147 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.143193  3162 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.143978  3162 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.130854  3163 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.153919  3162 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:03.144066  3145 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.154450  3149 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.131260  3166 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.137555  3160 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.169328  2932 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.177441  2295 raft_consensus.cc:1217] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.177166  3160 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.183353  3126 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.164336  3099 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.192687  3167 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.193579  3168 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.194834  3153 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.199556  3166 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.197918  3167 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.205668  3163 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.199282  3168 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.231302  2932 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.235463  2932 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.252440  2296 raft_consensus.cc:1217] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.206292  3165 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.268893  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.269364  3167 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.221175  3166 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.206815  3126 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.253827  3161 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.226784  3097 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.269740  1554 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.289606  3125 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.290050  3162 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.290609  3162 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.258554  3163 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.274824  3159 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.294639  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.295380  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.291150  3097 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.296183  3166 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.300478  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.307521  3097 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.300552  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.309052  3166 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.309897  3097 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.302985  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.289463  3165 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.290501  3125 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.304373  1704 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:03.293903  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.320683  3160 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.321807  3173 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.285120  3167 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.323114  2297 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:03.326490  3172 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.327138  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.327399  3172 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.328016  2296 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:03.303860  3168 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.325254  3159 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.337749  3159 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:03.348301  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.347373  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.352386  3145 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.340955  3163 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.353874  2295 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:03.357256  3163 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.316417  3162 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:03.359210  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.360263  1560 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.323185  3167 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.333537  3126 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.322883  3173 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.382043  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.390996  3171 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.361532  3162 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.394407  3153 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.341626  3149 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.317669  3161 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.335219  3097 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.383701  3147 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.398576  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.391580  2292 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.398981  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.405764  3174 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.405952  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.406703  1709 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:03.408708  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.404664  1703 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:03.427831  3178 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.398737  3172 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.391868  3171 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.439499  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.406700  3174 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.446086  3173 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.449239  3141 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.449815  3161 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.451138  2297 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:03.454347  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.456789  1560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:01:03.457883  3126 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.458542  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.459556  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.456825  3177 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.457916  1560 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:03.460558  1560 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.460482  3177 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.442269  3175 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.462190  3176 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.444532  3159 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.467694  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.442672  3097 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.509253  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.468568  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.501520  2932 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.506841  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.459594  3167 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.500988  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.507411  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.551463  1627 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.512735  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.551985  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.552649  1705 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:03.557216  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.557301  3179 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.558015  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.508828  3099 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.470420  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.561331  2932 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.508065  3165 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.558128  3179 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.573457  3181 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.574455  3179 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.574950  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.507428  3171 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.555905  1706 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:03.507584  3153 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.507477  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.500561  3174 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.578732  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.455475  3178 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.576366  1631 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.582715  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.582343  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.469944  3175 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.596424  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.597540  1627 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.598984  1703 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:03.574311  3181 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.575542  3173 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.600752  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.580103  3125 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.584056  3170 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.611387  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.611374  3170 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.612787  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.613380  1706 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:03.613790  3170 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.600737  3153 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.614347  1560 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.605167  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.605342  1633 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.605471  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.606606  3097 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.607484  2932 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.608182  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.610711  3175 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.602725  3177 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.579192  3183 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.569301  3176 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.636093  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.637351  3176 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.637503  3184 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.625329  3170 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.638219  3184 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.638676  3170 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.627519  3178 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.641443  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 202) took 2040 ms. Trace:
I20260709 13:01:03.641856  1455 rpcz_store.cc:276] 0709 13:01:01.601370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.601505 (+   135us) service_pool.cc:224] Handling call
0709 13:01:03.641243 (+2039738us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":12032}
I20260709 13:01:03.621935  1560 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.643487  1703 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:03.627869  1706 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:03.628571  3153 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.644181  3146 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.632696  3167 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.631278  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:03.648595  3181 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.648890  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.649238  3167 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.649585  1700 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:03.644662  3146 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.651186  3167 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.626315  3173 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.653074  3167 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.652602  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:03.661327  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.662335  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:03.664413  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.603884  2292 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:03.669936  3173 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.672047  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.672228  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.674203  1634 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.675448  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.677500  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.680641  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:03.681604  3179 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.682013  3179 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.682444  3179 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:03.683504  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.680s	user 0.004s	sys 0.013s
W20260709 13:01:03.683861  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.680s	user 0.004s	sys 0.015s
I20260709 13:01:03.682013  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:03.665171  3167 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.702831  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.709810  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.708485  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.706522  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:03.713168  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.713061  1559 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:03.706981  3167 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.640444  3183 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.724205  2995 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.725119  1634 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.726070  3184 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.726580  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.728104  3170 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:03.732335  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080003cfe0 after lost signal to thread 1584
I20260709 13:01:03.771317  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.732924  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.771502  3146 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.720376  3161 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.772598  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.772791  3161 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.773231  3161 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:03.774065  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002c09c0 after lost signal to thread 1659
W20260709 13:01:03.774369  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002c0a20 after lost signal to thread 1660
W20260709 13:01:03.775378  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800110ee0 after lost signal to thread 1733
W20260709 13:01:03.712991  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93713.2us
I20260709 13:01:03.743093  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.776576  1627 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.744395  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.753854  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.772601  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.778367  1634 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
W20260709 13:01:03.779685  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800329420 after lost signal to thread 2321
W20260709 13:01:03.780031  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08003296c0 after lost signal to thread 2322
W20260709 13:01:03.779834  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 77821.5us
I20260709 13:01:03.781126  3177 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.756320  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.756990  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.757246  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.781718  3177 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.782044  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.757122  1457 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 325) took 2154 ms. Trace:
I20260709 13:01:03.782732  1457 rpcz_store.cc:276] 0709 13:01:01.580486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.630559 (+ 50073us) service_pool.cc:224] Handling call
0709 13:01:03.735073 (+2104514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1968695,"reported_tablets":1}
I20260709 13:01:03.761634  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.784013  3165 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.784435  3165 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.709694  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.784837  3165 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.785015  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.734229  3044 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.785786  3158 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.785821  3044 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.786202  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.786417  3044 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.786482  3158 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:03.786935  3158 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.788031  3145 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.788472  3145 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.788842  3145 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.790045  3125 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.790437  3125 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.790791  3125 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.769827  2995 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.792053  2891 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.792554  2891 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.792662  3177 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.792917  2891 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.792102  2995 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.805099  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.806607  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.807356  1553 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:03.808230  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.742758  3179 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.798056  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.798856  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.811342  1555 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.799999  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.801506  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.802230  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.802670  3044 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.803238  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.803871  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.794453  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.810719  1559 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.813061  1554 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.814332  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.818029  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.818132  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.819587  1559 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:03.814661  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.823297  1553 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:03.814350  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.824223  1551 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:03.824932  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.815826  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.826010  3165 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.827662  3179 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.827941  3146 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.828045  1556 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:03.828552  3146 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.830807  3146 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.787307  3158 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:03.838622  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54855.1us
I20260709 13:01:03.844182  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.845161  3183 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.849602  3179 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.850061  3179 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.850425  3179 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.847354  3143 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.854525  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:03.856952  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.858281  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.854472  2891 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.860385  2891 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:03.847596  3181 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.869885  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.871505  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.853233  3165 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:03.877656  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57677.3us
I20260709 13:01:03.851585  3143 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.881121  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.881569  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.882239  1626 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:03.882479  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.883271  3153 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.883634  3153 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.863955  2891 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:01:03.884017  3153 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.862128  3141 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.892932  3141 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:03.893440  3141 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.860283  3163 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:03.864005  3097 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.884999  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.875751  3175 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.899941  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.900873  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:03.871124  3181 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.908552  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:03.917833  1626 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:03.918218  1626 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:03.919227  1626 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.920087  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.920288  1626 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.920890  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.921191  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.922019  3191 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.922399  3191 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.893656  3161 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.922770  3191 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.923456  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.922549  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:03.926205  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.930922  2292 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:03.894925  3044 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.925364  3161 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.933632  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:03.926136  1702 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:03.933213  3165 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.896716  3097 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.906123  2995 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.903980  3175 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:03.940047  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.917260  3181 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.951721  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.925863  3187 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.954632  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:03.955778  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.941918  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.940196  2932 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:03.958026  2932 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.958571  2932 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.958784  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:03.949092  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:03.936796  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76154.5us
I20260709 13:01:03.897931  3167 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.958010  2294 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:03.961198  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.961437  1703 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:03.937579  3165 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.963617  1705 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:03.915905  3125 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.967864  2891 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:03.944204  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.968289  2891 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:03.968641  2891 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.953869  3175 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.969408  3125 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:03.970427  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:03.970733  1558 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:03.971717  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.944414  2995 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:03.973711  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.940116  3097 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:03.979295  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.957154  3187 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.940372  3188 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.940060  3031 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:03.957388  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:03.989470  2891 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:03.975993  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:03.971426  3165 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.997072  1703 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:03.991183  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:03.997359  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:04.000982  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.006507  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.010413  3141 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.012504  3141 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.020120  3175 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:04.040653  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.022235  3167 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.022682  3031 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:04.024147  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 91183.2us
I20260709 13:01:04.026300  1626 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:04.026077  3153 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:03.997587  3187 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.961011  3179 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.052125  3031 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.052732  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.053866  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.053992  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.055114  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.055979  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.056098  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.056205  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.057111  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.051951  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.060107  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:03.980860  3163 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.061116  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.060153  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.062187  2294 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.032994  2891 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.038173  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.021598  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:04.043365  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 98168.4us
I20260709 13:01:04.004156  2995 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.044128  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.066751  2995 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.029043  3172 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:04.039605  3184 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:04.050510  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.050907  3194 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.031432  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.058538  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.064939  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.057111  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.067162  2995 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.069414  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.069414  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.070674  1631 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.040675  3141 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:01:04.074250  2891 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.071480  3172 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.061136  3163 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:04.088654  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.071040  3184 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:04.089170  2932 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.090045  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.077989  3194 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.093338  3194 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:04.093549  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82273.3us
I20260709 13:01:04.047618  3188 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.095758  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.087237  3172 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:04.084930  3163 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.100346  2932 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.094206  3184 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.102764  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.104981  2932 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:04.097829  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.105407  2932 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:04.106442  2932 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.107266  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.135902  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.136031  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.141642  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.142433  1703 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:04.104918  3143 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.144354  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.145774  1706 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.138073  3177 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:04.151013  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 100788us
I20260709 13:01:04.152160  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:04.156255  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53077.1us
I20260709 13:01:04.146900  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.149348  3177 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:04.169014  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.172220  1556 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:04.158128  3125 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:04.176222  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.158131  3153 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.182139  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.162079  3175 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.165901  3197 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.173534  3177 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:04.169281  3188 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:04.193534  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48683us
I20260709 13:01:04.182997  3044 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:01:04.195032  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.195784  3192 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:04.196159  3192 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:04.197086  3192 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.198280  3192 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.178941  3125 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:04.202127  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.202454  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.184918  3159 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.203259  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.203534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.205720  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:04.187373  3193 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:01:04.206630  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.207026  2932 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.192163  3197 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.209581  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.210043  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.203657  3125 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.184868  3191 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:04.205060  3031 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:04.213510  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.206954  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.214857  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.217218  2932 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.219460  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.219892  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.214534  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.206691  3188 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:01:04.231593  1551 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.211644  3197 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.217656  3031 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:04.238670  3158 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.239873  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:04.240090  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.236636  2932 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:04.241875  2932 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:04.242228  2932 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.246944  3158 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.247871  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.249816  3183 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.249541  3145 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.229626  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.260989  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.268182  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.268263  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.271106  2995 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.272403  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.272691  1630 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.273182  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.273450  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.241287  3031 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.275593  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.286209  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.251070  3183 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:04.298015  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.301481  3183 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.291595  3145 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.291546  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.292115  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.292279  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.292721  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.293092  1700 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.292281  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.297045  2294 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:04.298072  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.299906  3163 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:04.323999  3163 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.304051  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.324635  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.325124  1704 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.312515  3181 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.318617  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.326866  1707 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.294972  3191 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.327742  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.334779  3188 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.356961  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.335371  2995 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.336288  3187 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.337227  3181 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.301981  3172 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.339537  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.340893  1707 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:04.345237  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.345855  1708 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.368223  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.346556  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.348626  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.348999  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.351361  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.351854  3031 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.340931  1551 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:04.352780  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.353084  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:04.353176  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67066.5us
I20260709 13:01:04.356189  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.318892  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.358893  3179 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.364573  1702 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.369891  1706 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:04.372059  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.373435  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.391700  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.375030  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.375130  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.375272  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.377245  1633 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.377668  1703 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:04.379498  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.380151  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.380414  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.382262  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.375275  3172 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.433666  3196 raft_consensus.cc:1064] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:01:04.434083  3165 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.436350  3165 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.413699  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:04.436133  3196 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.439687  3181 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.440287  3181 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.397145  3183 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.396262  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.441367  1552 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:04.441072  3177 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.442219  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.443173  3177 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.443275  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.442242  1558 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:04.445640  3183 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.446189  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.447044  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.447199  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.447685  1554 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:04.448436  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.449574  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.397878  3197 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.405019  2297 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.424010  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:04.425673  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64792.2us
I20260709 13:01:04.425886  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.426095  3179 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.458580  1601 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.461787  1554 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:04.426357  3153 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.428342  3171 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:04.431246  3097 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.395985  3187 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.451726  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.452741  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.454630  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.445667  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:04.473831  1558 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:04.445930  3125 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:04.465030  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100447us
I20260709 13:01:04.475021  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:04.469964  2932 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:04.471246  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108374us
I20260709 13:01:04.479434  1628 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.487182  3171 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.490262  3171 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.458469  3192 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.483534  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.502768  1556 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:04.502908  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48402.7us
I20260709 13:01:04.481191  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.512967  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.493837  3170 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.496502  3195 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.481076  3172 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.493643  3187 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.504545  3097 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.504645  2995 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.519974  3187 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.519582  2995 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.520823  2995 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.520823  1700 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:04.505120  1555 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.505580  3188 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:04.507741  3181 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.508186  3192 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:04.524792  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.524660  3192 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.525897  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.508671  3141 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:04.526564  3161 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.526631  3141 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:04.527249  3161 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.506273  3125 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.520848  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.527601  3141 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.528507  3195 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:04.528839  3195 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:04.528905  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.529508  3187 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.517083  3031 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.530619  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.530884  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.531906  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.532444  2932 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:04.535720  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.535081  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:04.529747  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.538276  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.529752  3153 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.539570  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.540354  2932 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.543685  1633 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.545722  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.547919  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.549671  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.549916  1634 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.550109  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.550706  2296 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.551582  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.511334  3194 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.552232  2891 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.555354  2891 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.555758  2891 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.560672  2891 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.562043  2891 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.554015  1630 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.563690  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:04.564718  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.565698  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.537685  3187 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.571127  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.523554  3184 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.537328  3172 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.585490  1627 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.547676  3159 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:04.585111  2995 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:04.566788  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62870.9us
I20260709 13:01:04.549733  3153 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.595868  2995 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.565714  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:04.598223  2295 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.529723  3195 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.572793  3171 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.567806  3177 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:04.603590  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65357.8us
I20260709 13:01:04.604395  3210 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:04.605330  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.606081  3153 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.606680  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.608242  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.613050  3184 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:04.614332  1631 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.614391  3195 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:04.599611  3172 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:04.627354  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:04.567101  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.552125  3099 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:01:04.622797  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.632812  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.635186  3159 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:04.738890  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 167807us
I20260709 13:01:04.743685  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:04.744504  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 112392us
W20260709 13:01:04.747153  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 170233us
W20260709 13:01:04.751003  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 133975us
W20260709 13:01:04.751619  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 204254us
I20260709 13:01:04.606668  3208 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:04.802254  2932 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.607034  3203 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.205s
W20260709 13:01:04.802951  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 267912us
W20260709 13:01:04.804457  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64448.9us
I20260709 13:01:04.806427  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:04.807188  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.602802  3175 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 13:01:04.808104  2932 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.628654  3191 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.809684  2932 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:04.751487  3201 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.268s
I20260709 13:01:04.810351  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.804342  3172 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:04.817754  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.818161  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.824191  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.803579  3184 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:01:04.846423  2932 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.853379  3169 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.854907  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.856412  3159 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.819870  3044 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:04.864312  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46648.8us
I20260709 13:01:04.859475  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:04.861016  3125 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:04.866526  1702 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.845467  3146 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:04.868682  2296 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:04.872368  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:04.871255  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 284169us
I20260709 13:01:04.875815  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.876572  2932 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.876786  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.877672  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.876893  2995 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.816879  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.878839  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.880242  3208 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:04.880267  3222 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:04.880960  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 74985.1us
I20260709 13:01:04.881268  3220 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.859452  3031 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:01:04.882040  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.859331  3145 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.884778  2995 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.881806  3214 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.885442  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.868233  3044 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.902015  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.888995  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.889377  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.870222  3146 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:04.883198  3158 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:04.882640  3194 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.892333  3210 raft_consensus.cc:1064] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:01:04.892526  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:04.893170  3162 raft_consensus.cc:1064] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:01:04.893430  1554 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:04.894658  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.885162  3163 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.897709  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.899305  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.899647  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.902470  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.911595  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.903951  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.881959  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.888111  3206 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:04.910378  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50962.2us
I20260709 13:01:04.912707  3211 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:04.913445  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.912880  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.894022  3184 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.916700  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.915443  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.918455  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:04.914331  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.908541  3044 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.923887  1632 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:04.917225  3217 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:04.911041  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.929134  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.930621  2291 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.930835  3216 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:04.917197  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:04.911485  3158 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.945443  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:04.925909  3165 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:01:04.913926  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 102843us
I20260709 13:01:04.923853  3097 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.946354  1702 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.915253  3181 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.926337  3206 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:04.919190  3191 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:01:04.948997  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.940585  3031 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.952343  1703 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:04.957036  3224 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:04.953088  3146 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:04.934253  3210 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.958515  3197 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.961750  3044 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.962911  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:04.947201  3158 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.920711  3172 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:04.976739  3044 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:04.977450  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:04.977857  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:04.978096  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:04.973723  1706 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.980656  1629 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
W20260709 13:01:05.000020  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46152.4us
I20260709 13:01:04.999714  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:05.014439  1634 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.960258  3165 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:05.020901  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.961158  3171 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.017001  1707 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:04.956317  3206 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.017669  3209 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.017767  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:05.018327  1700 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.019227  3143 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:04.974232  1701 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.035915  3143 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:04.951023  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:05.037132  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 86230.4us
I20260709 13:01:05.037202  3143 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.037964  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.038892  3143 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.039191  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:04.973089  3191 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.961345  3125 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:04.943440  3222 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:01:05.025259  3184 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:05.043901  3125 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.032519  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:05.044317  3184 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:05.044811  3184 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.043987  3189 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:05.044934  3125 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.046226  3125 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.040859  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.042202  3168 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.049209  2288 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.049853  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 304) took 1089 ms. Trace:
I20260709 13:01:05.049722  3194 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:05.050230  2288 rpcz_store.cc:276] 0709 13:01:03.960088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.228228 (+268140us) service_pool.cc:224] Handling call
0709 13:01:05.048863 (+820635us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bff8c 00007f70cb8bfef6 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:05.049036 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.049627 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.049799 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3995648}
I20260709 13:01:05.050719  3194 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.051119  3194 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.049242  3224 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.052012  3194 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.053043  3169 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.053160  3194 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.053444  3169 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.054272  3169 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.057520  1627 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:05.057974  3169 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:05.059505  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.023247  3171 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:04.946743  3220 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:01:05.023989  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61280.4us
I20260709 13:01:05.067237  3221 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.067636  3221 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.068797  3221 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.070087  3221 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.064049  3222 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:05.063906  3163 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:01:05.074381  3222 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.063645  3216 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:05.075016  3163 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.075464  3163 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:05.075724  3216 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.076192  3216 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.042219  3031 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:01:05.064198  3168 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:05.077260  3168 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:05.042289  3191 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.065647  3208 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:05.077682  3168 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.076407  3163 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.077564  3165 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.077689  3183 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.084473  3153 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:05.120909  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.118748  3031 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:05.118373  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47348us
I20260709 13:01:04.972465  3162 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.128698  3179 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:05.129714  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.130473  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.124261  3163 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.132417  3170 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.132959  3177 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.133023  1626 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.124516  3183 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.141675  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.138809  3031 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.136826  3143 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:05.145988  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 216963us
I20260709 13:01:05.122658  3158 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.147688  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.148190  3172 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.128197  3153 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.150445  3207 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:05.150663  3217 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.151306  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:05.153856  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.153440  3204 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:01:05.145709  3161 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.141975  3170 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.123629  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.172188  3229 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.144987  3177 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.151160  1551 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.145529  3194 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.174481  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.146765  3184 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.255710  3213 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.218382  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.170145  3167 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.260849  1700 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.263465  3183 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:05.268959  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 94552.5us
W20260709 13:01:05.268187  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.270028  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:05.270828  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 52520.6us
I20260709 13:01:05.155049  3217 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.271934  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.172981  3207 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.275339  3211 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.268935  3226 raft_consensus.cc:1064] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: attempting to promote NON_VOTER 2579bae7bf1543bf9b10d68e93a6ebbb to VOTER
I20260709 13:01:05.154408  3171 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.280972  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.264678  3220 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:05.274677  3194 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.274654  3167 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.151194  3031 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.274470  3209 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:05.293694  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.279148  3201 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.290403  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:05.155062  3191 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.297339  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:05.280730  3195 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:05.280321  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 162559us
I20260709 13:01:05.286471  2288 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.289235  3181 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:01:05.305693  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:05.306082  3189 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.306521  3189 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.306565  3195 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:05.289127  3213 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.301659  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:05.301687  3184 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:05.172075  3216 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.258912  3153 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.258864  3170 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.318002  3153 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.319181  3158 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:05.319504  1556 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:05.320685  3170 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.312130  1559 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.313963  3168 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.324795  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:05.327788  1630 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:05.328393  3204 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:05.317329  1551 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.340873  2296 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:05.341862  2297 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.290335  3177 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.293608  3229 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:01:05.355233  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 165) took 1083 ms. Trace:
I20260709 13:01:05.307369  3189 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.309207  3220 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.355758  1628 rpcz_store.cc:276] 0709 13:01:04.269483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.274073 (+  4590us) service_pool.cc:224] Handling call
0709 13:01:04.274336 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.480819 (+206483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.352424 (+871605us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54001e8f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:05.352590 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471211776}
I20260709 13:01:05.333884  3184 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:01:05.361901  3230 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:01:05.357878  3222 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.329284  3207 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.400405  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:05.314677  3226 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.368713  3177 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.404345  3158 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:05.298049  3217 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:05.410952  1629 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:05.403187  3189 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.413149  3161 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.405763  3204 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:05.416555  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.416669  3220 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:05.426738  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:05.427157  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:05.428261  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.429908  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.406890  3206 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.432186  1560 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.407698  3158 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:05.435734  1553 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:05.429093  3222 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:05.448933  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.442557  1629 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:05.449867  1633 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.408226  3203 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 13:01:05.457259  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.424889  3204 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.466122  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:05.464262  3217 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.527153  3158 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:05.527833  3231 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:01:05.558883  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.531385  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.472091  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:05.463966  3145 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 13:01:05.623720  3232 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:01:05.524012  3168 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:05.643798  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.623523  3167 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:05.649487  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 189388us
I20260709 13:01:05.651386  2891 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.664556  3167 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:05.665215  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.666792  3167 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:05.620888  3165 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:01:05.643373  3204 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.681025  3167 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:01:05.686201  3167 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.686558  3167 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:05.687112  3208 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.687652  3208 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:05.688172  3163 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.004s
I20260709 13:01:05.688588  3208 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.689801  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.655853  3168 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:01:05.701179  3168 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.701623  3168 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.702435  3168 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:05.693111  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.703526  3168 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:05.674338  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 145715us
I20260709 13:01:05.713116  3146 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.687276  3167 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:05.715061  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 350290us
I20260709 13:01:05.715431  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:05.716974  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:05.717705  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.718596  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:05.706214  3222 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.724491  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:05.730160  1702 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:05.733479  3217 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:05.718700  3208 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.722258  3162 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:01:05.739607  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:05.740481  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.723155  3167 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.742298  3227 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:05.744927  2293 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:05.732354  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:05.745383  3165 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.746414  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.734055  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:05.751468  1556 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:05.750121  1560 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:05.733058  3222 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.733304  3163 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:01:05.749805  3243 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:01:05.756289  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:05.760651  1559 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:05.756536  1558 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.762094  2293 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:05.765307  3165 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.766054  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.768889  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.764848  3162 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.770313  2293 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 LEARNER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:05.771540  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.773716  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.770103  3165 raft_consensus.cc:2764] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:05.770742  3177 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:05.785919  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.787815  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:05.789206  3229 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.773451  3217 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:01:05.791646  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.791424  3159 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.792335  3159 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.772518  3227 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.770885  3162 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:05.777092  3245 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.782346  3222 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:05.799716  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:05.802161  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:05.827041  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:05.798676  3244 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.798513  3243 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.799880  3159 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.801891  3163 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:05.791538  3217 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.834698  3227 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.835458  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:05.836407  3227 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.836091  3217 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.836872  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.837746  3227 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.838356  3212 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.838639  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.838766  3212 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.840847  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.840806  3212 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.841684  3177 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.400511   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:05.842010  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.842342  3177 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.842769  3165 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.845440  3207 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.849325  3217 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:05.852144  3211 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:05.853016  3211 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:05.853396  3211 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.853515  3171 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.852202  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:05.855454  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.855813  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:05.856048  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56410.9us
I20260709 13:01:05.860270  2293 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:05.863869  1554 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:05.866874  3204 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:05.863888  3162 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:05.870411  3162 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.872797  3044 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.851586  3217 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.879484  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.881486  2290 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEARNER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:05.884402  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:05.876339  3169 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:05.899318  3169 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:05.875032  3243 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.906239  1627 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.930184  2932 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:05.884564  3175 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:05.885955  3181 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:05.883919  3179 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:01:05.880160  3158 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:01:05.934630  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.935417  3194 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.883001  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:05.880224  3141 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:01:05.953137  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:05.957906  3229 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:05.961184  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51229.8us
I20260709 13:01:05.966637  2932 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.971174  2296 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:05.943430  3199 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:05.956130  3175 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:05.885097  3227 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:05.964582  3229 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:05.977943  1700 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:05.992240  1552 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.002027  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.002656  1629 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:05.962630  3210 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:01:06.004007  1626 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:06.004998  3195 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.008237  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:06.010357  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46596.1us
I20260709 13:01:06.012609  3204 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:06.014806  3236 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.574s	user 0.006s	sys 0.000s
I20260709 13:01:05.958738  3169 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.019698  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:05.997253  3031 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.021850  1709 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:05.975898  3161 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:01:06.018831  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.016235  3125 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:01:06.026103  1706 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:06.018136  3183 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.032347  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:06.031350  3169 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.036005  3219 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:06.036444  3219 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:06.038483  3229 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.041225  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:06.042719  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:06.038326  3219 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.031675  3206 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.018494  3175 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.049796  1552 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:06.060402  3219 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.045034  3207 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:06.072754  2289 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:06.073041  1701 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:06.074159  1708 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:06.072741  1635 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:01:06.075054  1635 pending_rounds.cc:85] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 3
I20260709 13:01:06.075301  1635 pending_rounds.cc:107] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:01:06.072741  1628 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:06.075579  1635 raft_consensus.cc:2889] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:06.076661  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 178) took 1079 ms. Trace:
I20260709 13:01:06.077030  1633 rpcz_store.cc:276] 0709 13:01:04.997790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.998851 (+  1061us) service_pool.cc:224] Handling call
0709 13:01:04.999222 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.450255 (+451033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.076462 (+626207us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54001419d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:06.076608 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035988864}
I20260709 13:01:06.079804  1704 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:06.062544  3224 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:06.093500  3145 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:06.100661  3172 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:06.104449  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.069551  3211 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.090102  3192 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:01:06.101181  3223 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:01:06.112504  3044 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:06.103856  3189 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:01:06.109212  3171 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:01:06.178542  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.070981  3253 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 13:01:06.194396  3223 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.216008  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:06.217226  3237 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:06.218504  3237 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:06.218904  3237 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:06.220675  3223 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.190541  3158 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.194617  3175 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.239051  3206 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:06.213285  3143 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:06.240376  3206 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.240847  3206 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.243906  3223 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.243811  3158 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.245610  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:06.246057  3158 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:06.264761  1632 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:06.266716  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:06.280661  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:06.281540  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:06.300331  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.358707  3220 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.373221  3220 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.260459  3175 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.395210  3175 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.403522  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.404259  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:06.415733  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 326) took 1124 ms. Trace:
I20260709 13:01:06.375737  3220 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.418237  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.420776  1628 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:06.417160  2297 rpcz_store.cc:276] 0709 13:01:05.291455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.291598 (+   143us) service_pool.cc:224] Handling call
0709 13:01:05.291866 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.342212 (+ 50346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.415342 (+1073130us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b540027f454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:06.415601 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114537728}
I20260709 13:01:06.420157  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.425293  3220 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.429548  3031 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:06.430178  3220 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.430190  3031 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.431221  3031 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:06.440660  3225 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:06.441296  3225 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:06.448195  3225 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.449505  3225 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:06.457252  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.465431  3220 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.426829  3207 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.466737  3207 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.421926  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.418005  3206 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.478744  3031 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.484222  2297 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:06.481902  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:06.427150  3203 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.496439  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:06.497633  3125 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:06.497383  3252 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:06.496387  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:06.493640  3223 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:06.498955  1704 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:06.499025  3223 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.485831  3163 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:06.500671  3163 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:06.501287  3223 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.501523  3163 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.498060  3125 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.502872  3163 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.503113  3125 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.503829  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.510960  3125 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.515306  3206 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.516206  3125 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.470650  3229 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:01:06.488216  3195 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.516804  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:06.517549  3229 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:06.517292  3195 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.517640  1708 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.518009  3229 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:06.518929  3229 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.521337  3229 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.522406  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:06.524258  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:06.525121  3191 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.525331  2296 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:06.492749  3203 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.530416  3203 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.532160  3210 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.532946  3210 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.533422  3207 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.534179  3207 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.534727  3207 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.519356  3195 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.538079  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:06.542649  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.553720  1627 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.553956  1703 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 2: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:06.555692  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:06.556564  2296 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:06.544044  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:06.552057  2294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.023s	sys 0.044s
I20260709 13:01:06.558627  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:06.559333  2294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.023s	sys 0.044s
I20260709 13:01:06.493115  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:06.541273  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.560506  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.560814  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:06.560724  3031 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.562057  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.549547  3216 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:06.548084  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 321) took 1258 ms (client timeout 1612 ms). Trace:
I20260709 13:01:06.564061  1554 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:01:06.560724  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:06.567327  2297 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:06.557511  1701 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:06.570763  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.581492  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.582267  1552 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
W20260709 13:01:06.571653  2292 rpcz_store.cc:269] 0709 13:01:05.289284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.289979 (+   695us) service_pool.cc:224] Handling call
0709 13:01:06.547826 (+1257847us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400350054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:06.547983 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906481152}
I20260709 13:01:06.587360  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:06.542006  1457 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.588768  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.589526  3206 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:06.589959  3206 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.590270  3206 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:06.546090  3212 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.543901  3207 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.598917  1627 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:06.535756  3210 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.592279  3195 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:06.605647  1558 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:06.609190  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 942) took 1153 ms. Trace:
I20260709 13:01:06.610425  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:06.611311  1701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:06.623214  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:06.613188  1455 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.630759  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:06.627238  1455 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:06.632725  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67890.5us
I20260709 13:01:06.632964  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.636153  1455 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 4 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:06.640744  1455 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 3, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:06.614679  3195 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.643316  1627 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:06.645834  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.624718  3198 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.603741  3229 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:06.631873  3125 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:06.638358  1701 raft_consensus.cc:606] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:06.649959  3271 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.657230  3271 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:06.655612  1627 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:06.620955  1700 rpcz_store.cc:276] 0709 13:01:05.455828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.458396 (+  2568us) service_pool.cc:224] Handling call
0709 13:01:06.608998 (+1150602us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54003c0d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:06.609108 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812809472}
I20260709 13:01:06.647117  3195 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.646376  3207 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:06.664968  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:06.659682  1701 raft_consensus.cc:614] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:06.666872  3031 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:06.668967  3031 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:06.669452  3031 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:06.670670  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 103758us
I20260709 13:01:06.671957  1704 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:06.662065  3229 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:06.663285  3125 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:06.665591  3219 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.668504  3207 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:06.655292  3198 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.696434  3199 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:06.695535  1627 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.702472  3158 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:06.729204  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.670639  3213 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.738785  3031 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:06.749979  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107640us
I20260709 13:01:06.704510  3199 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:06.753932  3031 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.686697  3219 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.757934  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.753314  2297 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.761416  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:06.766347  1635 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:06.768779  3229 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.774961  3212 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.748284  3271 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:06.728319  3158 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.799448  3201 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.799700  3203 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:06.800082  3201 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.800115  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:06.800836  3203 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.802831  3203 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.803488  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.804325  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.804441  3189 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:06.804989  1707 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:06.760990  3253 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:06.806741  3206 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.806839  3189 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.813434  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.785202  3212 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.807889  3219 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.771903  3143 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.811021  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.820654  1627 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:06.814303  1709 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:06.772364  3224 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.750129  3213 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.815611  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.829401  1628 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:06.817869  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:06.816277  3253 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.822386  3275 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.835037  3225 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:06.822324  3143 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:06.824621  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:01:06.825421  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.840983  1635 raft_consensus.cc:606] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:06.823347  3169 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.805289  3272 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:06.819100  3097 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.810895  3201 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.761415  3125 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:06.837215  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75886.6us
I20260709 13:01:06.841348  1635 raft_consensus.cc:614] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:06.815632  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.835589  3225 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.861162  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:06.772317  3207 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.826570  3161 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:06.868448  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.835552  3213 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.868438  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:06.843953  3272 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.844108  3208 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:06.870529  3194 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.869894  3146 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.835402  3275 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.871376  3225 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:06.848209  3169 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.885003  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.884974  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.886435  1709 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:06.886813  1555 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:06.875074  3211 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:06.888172  3213 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:06.873306  1701 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:06.888607  3169 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.876158  3272 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.890115  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.896370  1558 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:06.876698  3161 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.866824  3097 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.892805  1554 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.901254  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:06.901904  2932 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:06.902278  2932 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.902629  2932 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:06.884218  3208 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:06.905936  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.907626  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:06.908118  1707 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:06.838762  3224 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.882555  3194 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.824301  3199 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.917610  3206 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.918262  3145 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.917769  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:06.909745  3280 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:01:06.936236  2932 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:06.909597  3189 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:06.909190  3211 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.910913  3213 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:06.944742  3278 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:06.949602  3253 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.949824  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.960052  1556 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:06.764539  3270 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.950661  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.928974  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:06.968292  3275 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.972528  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.973697  1554 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:06.970738  3208 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.953080  3212 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.977954  3213 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.982820  3097 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.983150  3224 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:06.955597  3278 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:06.983992  3278 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.948300  3198 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.990553  3044 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.988694  2932 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.964368  3158 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:06.959124  3211 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:06.946180  3145 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:06.993149  1556 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:06.998164  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.000634  1633 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:06.991211  3044 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.003298  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:07.004788  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:06.971560  1626 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:07.005666  1553 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:07.006670  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:07.008270  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.010402  1560 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:07.012915  2932 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.013563  2932 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.026712  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:07.027806  1554 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:07.028606  2932 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:07.029328  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.030504  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.031126  1555 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:07.023640  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:07.031996  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.032848  1553 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:07.038966  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:07.039562  3253 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:07.039909  3161 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.040292  3253 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.043215  3253 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.039525  3097 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:07.035084  3189 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.043954  3097 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.044060  3189 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.046013  3097 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.039593  1560 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.050289  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:07.051403  1555 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:07.020484  3158 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:07.052912  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.020077  3224 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.055958  3146 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:07.056566  3277 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.013236  3163 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:01:07.057159  3145 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.062788  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.068228  2296 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:07.065454  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.047344  3210 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.076571  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:07.046383  3189 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.067438  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:07.085656  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.086448  1704 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:07.084419  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.087636  1552 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:07.090535  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.075939  3222 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.067312  3277 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:07.093679  3191 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.006s
I20260709 13:01:07.083621  3206 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:07.100935  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.091625  3141 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.107544  1560 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
W20260709 13:01:07.103529  2263 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.718s (SENT)
I20260709 13:01:07.111299  3141 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.068324  3224 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.091782  2292 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.100071  3189 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.119875  1630 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:07.104874  3099 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.095570  3194 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.050328  3237 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.161476  3241 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.154954  3206 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.105412  3222 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.197402  3044 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.198321  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.190270  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:07.287945  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97752.9us
W20260709 13:01:07.212574  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 389906us
I20260709 13:01:07.287796  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.289484  1560 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.180648  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.291424  1706 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 2: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:07.286170  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.292783  1634 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:01:07.292867  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 213999us
I20260709 13:01:07.188512  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.289135  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:07.201083  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.199939  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.300784  1554 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.301849  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.299716  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:07.208868  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.210902  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1874) took 1872 ms. Trace:
I20260709 13:01:07.302511  1631 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:07.278192  3141 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.187629  3223 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:07.186767  3159 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.313458  3177 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:07.313835  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 125396us
I20260709 13:01:07.313944  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:07.173746  3216 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
W20260709 13:01:07.186652  3284 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:07.305052  1628 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.344363  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.162478  3220 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.196143  3211 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.329780  1631 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:07.202600  3143 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.345358  1560 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.201274  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:07.325683  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 222173us
W20260709 13:01:07.348780  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43657.8us
I20260709 13:01:07.168733  3099 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.324976  3290 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:07.287686  3169 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:07.199506  3195 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.534s	user 0.004s	sys 0.007s
I20260709 13:01:07.197934  3224 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.269331  3194 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:07.164444  3189 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.208492  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:07.358225  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 203662us
I20260709 13:01:07.208099  3225 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.364090  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.314771  3278 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:07.319656  1630 rpcz_store.cc:276] 0709 13:01:05.338166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.340102 (+  1936us) service_pool.cc:224] Handling call
0709 13:01:05.340389 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.153454 (+1813065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.210603 (+ 57149us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54001afc54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:07.210830 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4066947456}
W20260709 13:01:07.396834  3235 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:01:07.369324  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:07.188798  3277 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.378187  2297 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:07.383982  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.435117  1702 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:07.384922  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.437098  1708 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:07.377015  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:07.342458  3210 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:07.332546  3177 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.391361  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.381878   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:07.439795  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.395403  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.362171  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:07.407171  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.342716  3175 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:01:07.412077  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.448269  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.450291  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.412732  1556 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.323649  3183 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:07.324082  3206 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.387107  3201 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:07.387017  3169 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.171501  3272 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.430631  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.440809  1555 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:07.445151  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75928.5us
I20260709 13:01:07.459375  1628 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:07.418838  3217 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.444105  1704 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.388052  3290 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.444653  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.445325  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.454137  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.366350  3220 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.443717   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:07.455365  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71095.6us
I20260709 13:01:07.457118  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.411489  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:07.473008  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.457798  3044 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:07.474368  3044 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.474776  3044 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.444958  3031 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.461973  3211 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:07.404690  3195 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.469517  1709 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:07.461710  1706 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:07.466630  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:07.458936  3163 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:07.444483  3177 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.472143  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.457317  3208 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:07.453186  3183 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.412819  3194 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.491101  1704 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:07.448804  3165 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:07.462359  3169 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.468111  3290 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:07.472586  3277 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:07.529330  3206 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.529822  3277 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.516896  3224 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:07.530652  3224 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.519631  1632 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:07.531044  3224 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.493420  1700 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:07.520056  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:07.508510  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.532948  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62155.2us
I20260709 13:01:07.533703  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.534329  1704 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:07.494309  3211 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.510671  3183 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.527107  3044 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.514649  3165 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.529026  3099 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.520709  1626 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:07.493652  3293 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:07.507867  3208 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.533732  1628 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:07.462155  3201 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.543808  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:07.530212  3277 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.544782  3044 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.530813  3206 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:07.496637  3163 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.546017  3165 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.513077  3226 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:07.549999  3031 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:07.471740  3217 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } }
I20260709 13:01:07.511379  3191 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:07.543895  3211 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.544981  3270 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:07.546411  3293 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.549109  3208 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.552503  3163 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.566648  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.569875  1552 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:07.555333  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.571230  3031 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:07.571681  3031 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:07.554738  3226 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.572414  3031 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.562575  3222 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.564745  3293 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:07.555670  3206 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.605639  3206 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.582458  3226 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:07.608683  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:07.611213  1628 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.581630  3143 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:07.616693  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.565174  3191 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.617337  1627 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:07.619490  1628 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:07.566819  3270 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.621338  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.624435  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:07.625407  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 248511us
I20260709 13:01:07.626546  3044 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.627411  3044 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:07.628423  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 183155us
I20260709 13:01:07.628382  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.630115  1628 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:07.632215  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:07.607290  3306 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:07.658586  3044 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.618095  3303 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:07.684412  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59315.2us
I20260709 13:01:07.685220  3195 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.686085  1631 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:07.686388  3293 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.718827  1631 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:07.688421  3298 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.689846  3290 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.690737  3306 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.628863  3141 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.627851  3162 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.623412  3278 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:07.633617  3225 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.617918  3143 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.706916  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.708528  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.711310  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.712023  3277 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.713100  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:07.662585  3272 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.714074  3224 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:07.771369  1704 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:08.008927  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.888589  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:07.740098  3201 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.009181  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:07.738509  2294 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:07.729562  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:08.028051  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:08.028748  1707 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.029650  1708 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:07.955660  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:08.030191  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:08.029858  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:08.031363  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75800.1us
I20260709 13:01:08.031291  1702 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:07.750247  1557 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:08.053980  1703 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:07.749415  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:08.026221  3199 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:08.026255  1556 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:07.757773  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:07.748989  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:08.143862  1558 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:08.144315  1555 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:08.087849  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78763.3us
I20260709 13:01:07.730515  1559 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
W20260709 13:01:08.113700  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 384223us
I20260709 13:01:07.925092  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:08.016875  2292 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.003s
I20260709 13:01:08.087711  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:07.698154  3220 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.705955  3305 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.720436  3203 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.025759  3225 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:08.091274  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.787950  2297 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:08.025770  3237 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.201416  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.107254  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:08.125557  1708 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:08.142860  1629 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.114708  1702 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:07.736778  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.949103  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:08.150107  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:08.191011  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:07.771584  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:08.191984  1634 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:08.222508  2293 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:08.090533  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.014307  3189 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.737555  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.732112  3212 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:08.192478  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.937692  3301 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
W20260709 13:01:08.243336  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 145) took 1412 ms (client timeout 1806 ms). Trace:
I20260709 13:01:08.265451  1629 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:07.744517  3270 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.107322  3308 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:07.828111  3253 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.260424  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.087392  3195 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.295578  2293 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:07.957971  3211 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.868780  3303 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:07.888307  3298 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.107872  3241 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:07.893913  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.719703  3293 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:08.231561  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 460042us
W20260709 13:01:08.261422  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49592.6us
I20260709 13:01:08.242682  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:08.266263  3031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.023s	sys 0.009s
W20260709 13:01:08.266781  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.000s	sys 0.027s
I20260709 13:01:08.267411  1555 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:07.937970  3194 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.107995  3222 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:08.065472  3201 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:08.264003  1559 rpcz_store.cc:269] 0709 13:01:06.787660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.789316 (+  1656us) service_pool.cc:224] Handling call
0709 13:01:08.199560 (+1410244us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54003fb954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:08.199672 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130399488}
I20260709 13:01:08.273241  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:07.927860  3169 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.149092  3219 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:07.735508  3207 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.107630  3278 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:08.107880  3229 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.282822  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.008s	sys 0.015s
I20260709 13:01:08.108233  3097 raft_consensus.cc:476] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:01:07.730157  3159 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:08.198354  3198 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:08.289564  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:07.953158  3217 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:07.719656  3183 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.197976  2292 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:08.019654  3163 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.245352  3220 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.294494  1707 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:08.296129  1633 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.004s	sys 0.014s
I20260709 13:01:08.396704  3143 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:08.392620  3300 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.000s	sys 0.026s
W20260709 13:01:08.432979  1633 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.004s	sys 0.014s
I20260709 13:01:08.259565  3210 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.024856  1700 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:07.745679  3191 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.113204  3141 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.316669  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 188) took 1014 ms. Trace:
W20260709 13:01:08.299481  3223 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.000s
I20260709 13:01:08.243854  3203 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.289146  3310 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:08.300849  3208 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.002s
I20260709 13:01:08.084086  3162 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.298008  3225 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.320498  2296 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
W20260709 13:01:08.312230  3165 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.003s
I20260709 13:01:08.461781  3225 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.461997  3165 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.438894  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:08.273679  3212 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.464193  1630 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:08.463934  3208 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.465157  1627 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:08.465845  3212 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:07.720752  1551 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:08.318589  3302 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.464262  3165 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.467563  3302 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:08.468647  3226 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.845s	user 0.010s	sys 0.000s
I20260709 13:01:08.469066  3226 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.470027  3302 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.470610  3302 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.471311  3226 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.471107  3302 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.325711  3308 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:01:08.307964  3195 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.475868  3171 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.476557  3171 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.477667  3195 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.349885  3290 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.351500  3303 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.478988  3303 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.479374  3162 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.479584  3303 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.481349  3183 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:08.352490  3309 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 13:01:08.352988  3224 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.358013  3211 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.358985  3307 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.483342  3307 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.329785  3311 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:01:08.360584  3306 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.300657  3253 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:08.362378  3177 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.847s	user 0.010s	sys 0.000s
W20260709 13:01:08.374002  2261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.022s	sys 0.023s
W20260709 13:01:08.393375  1635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.006s	sys 0.036s
W20260709 13:01:08.375176  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.010s	sys 0.011s
W20260709 13:01:08.401723  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 160674us
I20260709 13:01:08.403170  2294 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:01:08.403663  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.000s	sys 0.030s
W20260709 13:01:08.490869  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 918401us
W20260709 13:01:08.404271  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161667us
W20260709 13:01:08.406718  3031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.025s	sys 0.010s
I20260709 13:01:08.408823  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:08.493104  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84367us
W20260709 13:01:08.414923  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.009s	sys 0.015s
W20260709 13:01:08.489969  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.011s	sys 0.011s
I20260709 13:01:08.505985  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:08.506980  1552 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:08.425872  1553 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:08.408465  3219 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.508852  3219 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.441151  2297 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:08.509023  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:08.511389  3199 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:08.416561  3194 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:08.511940  3194 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.512001  3199 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:08.512427  3194 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:08.512532  3199 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.514678  3199 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.413829  3278 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:08.516002  3278 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:08.485191  3125 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.443192  3216 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.007s	sys 0.022s
I20260709 13:01:08.429232  3204 raft_consensus.cc:2764] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:08.446980  1634 rpcz_store.cc:276] 0709 13:01:07.302207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.313622 (+ 11415us) service_pool.cc:224] Handling call
0709 13:01:08.316425 (+1002803us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54001432d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:08.316600 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873286912}
I20260709 13:01:08.414426  3161 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:01:08.427688  3159 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.418504  3222 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.419031  3207 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.443675  3210 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.438092  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 337) took 1370 ms. Trace:
I20260709 13:01:08.453480  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1906) took 1246 ms. Trace:
W20260709 13:01:08.440291  3300 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.000s	sys 0.029s
I20260709 13:01:08.414182  3201 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.465899  3208 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.320463  3277 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.245090  3305 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.335983  1703 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:08.472980  3298 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.376502  3241 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.478202  1705 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:08.603348  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:08.604105  1628 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:08.426996  3314 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:01:08.418731  3229 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.428615  3198 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.594578  3195 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.489739  2261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.023s	sys 0.023s
W20260709 13:01:08.489882  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.006s	sys 0.038s
I20260709 13:01:08.523888  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:08.524649  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:08.523581  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:08.528766  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:08.529065  3031 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:08.530409  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:08.489666  3303 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.490185  3306 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.538707  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.538921  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:08.617713  1554 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:08.617951  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 94205.5us
W20260709 13:01:08.551501  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135881us
I20260709 13:01:08.529228  3226 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:08.582249  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.623066  3031 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:08.623411  2296 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:08.623600  3031 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:08.537068  3216 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.007s	sys 0.022s
I20260709 13:01:08.583045  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:08.584438  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.587603  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.593211  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:08.629104  1632 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:01:08.623207  3226 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.629966  3226 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.623809  3195 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.630551  3031 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.632908  3195 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.625036  3311 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:08.636137  3311 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.636497  3311 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.636808  3322 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.628980  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1911) took 1262 ms. Trace:
I20260709 13:01:08.596381  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 1913) took 1140 ms. Trace:
I20260709 13:01:08.536489  3272 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:08.639930  1627 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:08.536449  3318 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.523164  2291 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:08.498229  3177 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.602078  1631 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:08.482152  3183 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.606345  2295 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:08.577391  1633 rpcz_store.cc:276] 0709 13:01:07.206750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.207833 (+  1083us) service_pool.cc:224] Handling call
0709 13:01:07.208130 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.356487 (+148357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.356517 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.386769 (+ 30252us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.524431 (+137662us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:08.437223 (+912792us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.437446 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.449499 (+ 12053us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:08.449588 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.449651 (+    63us) raft_consensus.cc:1685] finished
0709 13:01:08.453246 (+  3595us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.453418 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67734,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":9809,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":27458,"spinlock_wait_cycles":313505152,"thread_start_us":1002488,"threads_started":2,"wal-append.queue_time_us":202420}
I20260709 13:01:08.580098  3212 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:08.448385  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 958) took 1088 ms. Trace:
I20260709 13:01:08.579910  3199 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.538762  3191 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:01:08.581334  3143 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:08.537367  3125 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.592470  3220 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:08.616010  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87327.7us
W20260709 13:01:08.616334  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 86031us
I20260709 13:01:08.550505  2294 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
W20260709 13:01:08.642724  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49586.6us
I20260709 13:01:08.667515  1457 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 205) took 2182 ms. Trace:
I20260709 13:01:08.619012  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:08.620546  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97048.2us
I20260709 13:01:08.753317  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:08.600996  3189 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:08.625459  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100882us
I20260709 13:01:08.754631  2293 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.577247  2292 rpcz_store.cc:276] 0709 13:01:07.062171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.067145 (+  4974us) service_pool.cc:224] Handling call
0709 13:01:08.432653 (+1365508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.490033  3310 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.628301  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:08.641594  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.602383  3208 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.756752  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.643046  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.640053  1630 rpcz_store.cc:276] 0709 13:01:07.441086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.441531 (+   445us) service_pool.cc:224] Handling call
0709 13:01:08.553147 (+1111616us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:08.581730 (+ 28583us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2284981504}
I20260709 13:01:08.625056  3237 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:08.755380  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:08.623015  3169 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.624050  3213 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.625509  3211 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:08.809675  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54150.7us
W20260709 13:01:08.832202  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 203966us
I20260709 13:01:08.659600  3241 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.792783  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:08.833453  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.842892  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:08.639222  1635 rpcz_store.cc:276] 0709 13:01:07.366678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.367446 (+   768us) service_pool.cc:224] Handling call
0709 13:01:07.367871 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.368255 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.368294 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.385917 (+ 17623us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.580775 (+194858us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:07.732609 (+151834us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b48001bc258. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:01:08.612202 (+879593us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.616786 (+  4584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.616860 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.616906 (+    46us) raft_consensus.cc:1685] finished
0709 13:01:08.624841 (+  7935us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.628925 (+  4084us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115234,"prepare.queue_time_us":11565,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":33,"raft.queue_time_us":8379,"raft.run_cpu_time_us":3759,"raft.run_wall_time_us":362490,"spinlock_wait_cycles":13113984,"thread_start_us":1156234,"threads_started":2,"wal-append.queue_time_us":522315}
I20260709 13:01:08.846292  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.846583  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:08.846522  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93782.1us
I20260709 13:01:08.846532  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:08.847363  3031 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:08.846680  1628 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
W20260709 13:01:08.844571  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 217147us
I20260709 13:01:08.847846  3031 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.861478  3031 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.645012  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:08.862918  1626 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:08.863219  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:08.649168  3305 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.657943  3303 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:08.837232  3212 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.865178  3303 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:08.865598  3303 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:08.645321  3229 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:08.869441  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 114129us
I20260709 13:01:08.625423  3306 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.662235  3222 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.647614  3198 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.643456  3141 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.660815  3253 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.667234  3203 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:08.848316  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55628.6us
I20260709 13:01:08.873364  3241 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.832563  3143 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.822736  3199 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.830039  3278 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:08.666353  3163 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.665491  3201 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.659909  3307 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:08.648141  3272 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:08.835867  3298 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:08.663504  3210 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.878216  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:08.836246  3224 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:08.657276  3171 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.661948  3159 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:08.653069  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.637315  3322 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.837075  3183 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.872485  3208 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.886497  3220 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.848932  3195 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.642166  3308 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.231s
W20260709 13:01:08.662653  1551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 134) took 2948 ms (client timeout 1718 ms). Trace:
I20260709 13:01:08.753320  1700 rpcz_store.cc:276] 0709 13:01:07.358388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.376039 (+ 17651us) service_pool.cc:224] Handling call
0709 13:01:08.446490 (+1070451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.836539  3189 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.873767  3315 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.822506  3318 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.760654  3321 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:08.993148  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 236737us
I20260709 13:01:09.095165  2294 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:09.097766  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.102628  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.103518  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:09.000416  3031 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.104333  3241 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.878058  3314 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:09.105780  3314 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:09.106843  3031 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.044238  3310 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.107402  3314 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:09.110141  3322 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.043370  3212 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.110491  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:09.111496  2292 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.111685  2293 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:09.066315  3191 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.112632  3191 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.112988  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.113196  3141 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.113714  2293 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:09.114024  3141 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.115015  1599 leader_election.cc:400] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:09.115357  1599 leader_election.cc:403] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:09.115656  3298 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.116163  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.116439  3272 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.983247  3194 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.008839  3253 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.117799  1634 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:09.118840  3194 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.119347  3253 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:08.930809  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:08.998492  3327 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:09.121400  3327 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.121919  3327 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.123024  3278 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:08.982686  3226 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.116211  3298 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:09.120806  2288 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:09.008467  3222 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.125025  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.125176  3272 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.125458  3222 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.126667  3226 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:09.127259  3207 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.128510  3191 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.129565  3336 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.092566  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:09.083945  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 208329us
I20260709 13:01:08.881752  3213 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:09.131294  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:09.131873  1709 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.890202  3169 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:08.638164  3325 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:09.136611  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.137172  1555 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:09.138866  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.139161  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.139441  1558 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:09.139743  1552 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.127967  3222 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.143090  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:09.140671  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.143544  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.144623  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:09.145385  1554 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:09.083292  3219 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.001s
I20260709 13:01:09.147938  2297 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 2: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:09.149437  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.150127  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:09.146225  1553 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:09.142032  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.150367  1632 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:09.130537  3306 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.142982  3169 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.170680  3169 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.171278  3125 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.152601  1556 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:09.173910  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:08.753916  1457 rpcz_store.cc:276] 0709 13:01:06.485273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.485421 (+   148us) service_pool.cc:224] Handling call
0709 13:01:08.667455 (+2182034us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:06.675968 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:06.676789 (+   821us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:06.676854 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:01:06.676899 (+    45us) tablet.cc:662] Decoding operations
0709 13:01:06.677447 (+   548us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:06.677550 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:06.677582 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:06.677893 (+   311us) tablet.cc:701] Row locks acquired
0709 13:01:06.677950 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:01:06.678149 (+   199us) write_op.cc:270] Start()
0709 13:01:06.678340 (+   191us) write_op.cc:276] Timestamp: P: 1783602066678078 usec, L: 0
0709 13:01:06.678387 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:01:06.750810 (+ 72423us) log.cc:844] Serialized 536 byte log entry
0709 13:01:07.712992 (+962182us) op_driver.cc:464] REPLICATION: finished
0709 13:01:08.001894 (+288902us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b1800287f80. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce961190 00007f70ce95aae7 00007f70ce959dd9 00007f70ce964930
0709 13:01:08.603476 (+601582us) write_op.cc:301] APPLY: starting
0709 13:01:08.611520 (+  8044us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:08.638309 (+ 26789us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:08.650137 (+ 11828us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:08.650505 (+   368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:08.650832 (+   327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:08.650921 (+    89us) write_op.cc:312] APPLY: finished
0709 13:01:08.651240 (+   319us) log.cc:844] Serialized 23 byte log entry
0709 13:01:08.651490 (+   250us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:08.651588 (+    98us) write_op.cc:454] Released schema lock
0709 13:01:08.651656 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:08.652076 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:08.652105 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":825911,"apply.run_cpu_time_us":2447,"apply.run_wall_time_us":70481,"mutex_wait_us":61331,"num_ops":1,"prepare.queue_time_us":51419,"prepare.run_cpu_time_us":83626,"prepare.run_wall_time_us":365502,"raft.queue_time_us":1205854,"raft.run_cpu_time_us":84526,"raft.run_wall_time_us":1287715,"replication_time_us":950973,"spinlock_wait_cycles":5198720,"thread_start_us":1255013,"threads_started":4,"wal-append.queue_time_us":326182}]]}
I20260709 13:01:09.157009  3212 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.159600  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.160087  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:09.161454  2291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 343) took 1401 ms (client timeout 1700 ms). Trace:
I20260709 13:01:09.163918  3225 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.164959  3143 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.136080  3325 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:09.212920  3031 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.164901  3213 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.165970  3272 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:09.146255  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.218746  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.235179  1633 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:09.133921  3321 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.167253  3199 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.084862  3333 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:09.176033  2296 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:09.176100  1560 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:09.130918  3198 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.209784  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.147418  3336 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.140218  3177 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:09.210927  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 342422us
I20260709 13:01:09.116763  3141 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.233546  3031 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.234159  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:09.144060  3290 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.149121  3162 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.143894  3211 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:09.140524  3195 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.236013  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.413690  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.414321  1633 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.414918  1628 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:09.415959  1631 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:09.240226  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.303467  1553 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:09.417785  2294 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:09.413304  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.273052  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.417060  2296 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:09.420778  1558 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:09.429571  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:09.429723  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:09.430581  2294 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:09.437616  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:09.442940  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 570042us
I20260709 13:01:09.448346  1555 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:09.449174  1552 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:09.451234  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.449208  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.452828  2294 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:09.453678  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:09.454499  1553 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:09.452369  1555 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.231778  3191 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:09.438287  1627 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
W20260709 13:01:09.388052  1626 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.002s
I20260709 13:01:09.457656  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.212766  3229 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.464037  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.461163  1631 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:09.311517  3311 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.000s
W20260709 13:01:09.240343  3330 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:01:09.235592  3325 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:09.476912  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.185583  3219 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.483829  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.479025  3031 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.422334  3237 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.307183  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.235064  3315 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:09.483474  1555 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:09.448148  3204 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.494675  3204 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.495982  2288 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:09.496575  3204 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:09.496902  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.497537  2288 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:09.429963  3208 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.241618  3318 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:09.485844  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:09.500576  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.501178  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.447659  3162 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.503247  3321 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.503347  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:09.504261  3318 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:09.504364  3162 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.504779  3318 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:09.505602  3318 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:09.185331  2291 rpcz_store.cc:269] 0709 13:01:07.759574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.827828 (+ 68254us) service_pool.cc:224] Handling call
0709 13:01:09.161248 (+1333420us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540040a954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:09.161388 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222910336}
I20260709 13:01:09.237352  3327 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.520432  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.491290  3183 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.491811  3237 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.470660  1626 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:09.492882  1556 raft_consensus.cc:2368] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3.
I20260709 13:01:09.493796  3219 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:09.092971  1551 rpcz_store.cc:269] 0709 13:01:05.714720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.718047 (+  3327us) service_pool.cc:224] Handling call
0709 13:01:08.662388 (+2944341us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54000e9554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:08.662522 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3690305792}
I20260709 13:01:09.523519  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.447062  3333 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.470826  3191 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.490155  3336 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:09.575801  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 126720us
I20260709 13:01:09.534086  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.425009  3211 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.509188  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:09.509743  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.595883  1632 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:09.597026  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.312462  3303 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.489084  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.598466  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.514384  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.600330  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.601166  1635 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:09.505970  3195 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.516868  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.231575  3212 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.623492  2293 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:09.482177  3161 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.448169  3198 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.525758  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.630579  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.457350  3290 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.521081  1553 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.631932  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.632550  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.633126  1553 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:09.633272  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.533301  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.498355  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.559657  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.552544  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.546926  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.636103  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.526400  3207 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:09.533458  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.560137  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:09.552577  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.555233  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 357) took 1462 ms. Trace:
I20260709 13:01:09.554098  3237 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.480777  3229 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.643584  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.641403  2295 rpcz_store.cc:276] 0709 13:01:08.094450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.289288 (+194838us) service_pool.cc:224] Handling call
0709 13:01:09.555086 (+1265798us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400350cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:09.555182 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431335680}
I20260709 13:01:09.507385  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.521090  2292 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:09.643756  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.644933  1552 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:09.480218  3311 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.602142  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.648794  1555 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:09.649745  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.651017  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.617623  1560 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:09.618561  1633 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:09.622094  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:09.652732  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:09.654402  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:09.523212  3341 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.626819  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.656152  1635 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:09.628170  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.552079  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:09.633893  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.657590  1553 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:09.657968  1551 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.658361  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.658960  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.660861  3241 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:09.518778  3208 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.661477  1551 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:09.556183  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 355) took 1521 ms. Trace:
I20260709 13:01:09.661924  3208 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.662520  3290 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.662366  2290 rpcz_store.cc:276] 0709 13:01:08.034621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.318857 (+284236us) service_pool.cc:224] Handling call
0709 13:01:08.332273 (+ 13416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:08.877622 (+545349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.880256 (+  2634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.902404 (+ 22148us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:08.909440 (+  7036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.555950 (+646510us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54002c8dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:09.555996 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.556125 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231586176}
I20260709 13:01:09.554461  3141 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:09.663571  3212 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.663986  3208 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:09.591162  3191 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.520928  3339 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:09.572376  1709 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:01:09.637528  2296 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.637815  1707 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
W20260709 13:01:09.639901  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 79881.1us
I20260709 13:01:09.559681  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:09.640388  3163 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:09.644526  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.645459  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.647727  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.523602  3315 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:09.636840  1634 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.665294  1631 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:09.534086  3199 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 13:01:09.665817  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:09.560570  3298 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:09.641031  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:09.635109  3161 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.097996  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 980) took 1003 ms. Trace:
I20260709 13:01:09.684392  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.657330  1632 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:10.103850  1558 raft_consensus.cc:2368] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:10.105556  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:09.651650  1555 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:09.714560  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.106431  1552 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:09.735204  1627 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:09.734669  1560 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:09.714435  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.101687  1704 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
W20260709 13:01:10.108803  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 394314us
I20260709 13:01:10.109735  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.107604  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.100512  1633 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:09.668577  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:10.091032  3327 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.108165  3219 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.664017  3141 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.117259  3219 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.117713  3210 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.118211  3210 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.118588  3210 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.117295  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:10.120415  3325 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.000s
I20260709 13:01:10.120227  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:10.119483  3219 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.120872  3325 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.121250  1634 raft_consensus.cc:2368] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3.
I20260709 13:01:10.069162  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.121870  1559 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:10.122958  1701 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:10.126235  1635 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:10.108480  1708 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:10.099861  1703 rpcz_store.cc:276] 0709 13:01:09.095003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.095141 (+   138us) service_pool.cc:224] Handling call
0709 13:01:09.095478 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.095918 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.095959 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.143617 (+ 47658us) log.cc:844] Serialized 26 byte log entry
0709 13:01:09.478248 (+334631us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:09.478303 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.478402 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.097717 (+619315us) raft_consensus.cc:1685] finished
0709 13:01:10.097746 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.097916 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27915,"raft.queue_time_us":129,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10294,"thread_start_us":330867,"threads_started":1,"wal-append.queue_time_us":416478}
I20260709 13:01:10.127693  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.128614  1704 raft_consensus.cc:2368] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:10.129067  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.131253  1632 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:09.735476  2288 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:10.134200  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:10.135353  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.138833  1553 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:10.136130  2294 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:10.129993  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:09.656960  3237 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.140269  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:09.923419  3290 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.142136  1631 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:09.919070  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:10.026057  3198 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.128230  2291 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:10.155668  3161 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:09.959563  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:10.143560  1553 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:09.668534  3125 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.156615  3125 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:10.143577  1705 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:10.156701  3333 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.157056  3125 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:09.735631  3143 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:10.144248  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.141086  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.158073  1704 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:09.717037  3208 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.166934  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:09.684669  3177 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:10.145691  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:09.774883  3212 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:09.685678  1626 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.137257  3163 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:09.949857  3328 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.138538  3141 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:10.123967  3241 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.146728  3162 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:10.193862  3212 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.180151  3161 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:10.166002  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.166707  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.169808  3199 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.167562  3290 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.157069  2292 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.174407  3339 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:10.176503  3189 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.177160  3302 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:10.177804  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 672) took 1655 ms (client timeout 1801 ms). Trace:
I20260709 13:01:10.133949  3314 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:10.184715  2296 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:09.655481  3341 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.138758  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.146862  3327 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.122961  3325 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.164968  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.179884  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:10.204905  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.213833  3338 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.218689  3161 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:10.217864  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.251108  1553 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:10.163422  3237 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:10.228617  2297 rpcz_store.cc:269] 0709 13:01:08.521959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.522252 (+   293us) service_pool.cc:224] Handling call
0709 13:01:10.177624 (+1655372us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540027f454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.177748 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219472128}
I20260709 13:01:10.231746  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.196590  3163 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:10.268428  3163 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.187288  3208 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.271937  1551 raft_consensus.cc:2368] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4.
I20260709 13:01:10.273010  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.273247  3208 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.235527  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.273166  3161 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.274384  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.274832  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:10.275964  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.194170  3328 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.277128  3212 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.278198  3328 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:10.278787  3220 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.279184  3220 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.279582  3220 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.282641  3198 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:10.283164  2292 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:10.243216  3298 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.217015  3199 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.251421  1704 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:10.290020  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.254086  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.255918  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:10.328532  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.255971  3341 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.259212  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 1743) took 1085 ms. Trace:
I20260709 13:01:10.260882  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.263751  1626 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:10.240190  3327 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.264642  1701 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:10.266574  3237 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.340317  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.341073  3237 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:10.341205  1706 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:10.341583  3237 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:10.341837  3199 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.342517  3237 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.268810  3253 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.252406  3338 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.216185  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 349) took 1353 ms. Trace:
I20260709 13:01:10.275597  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.198603  3241 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.244319  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.233814  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.233546  3211 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.327239  3225 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:10.391518  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.329844  1556 rpcz_store.cc:276] 0709 13:01:09.173332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.173669 (+   337us) service_pool.cc:224] Handling call
0709 13:01:10.258972 (+1085303us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540030a7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.259137 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731707136}
I20260709 13:01:10.340142  3327 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.345245  1703 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.344082  2293 rpcz_store.cc:276] 0709 13:01:08.862983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.115930 (+252947us) service_pool.cc:224] Handling call
0709 13:01:10.215973 (+1100043us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.216112 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164105600}
I20260709 13:01:10.345930  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.332648  3208 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.406392  1704 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
W20260709 13:01:10.404604  1702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.037s	sys 0.062s
W20260709 13:01:10.427322  1702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.037s	sys 0.062s
I20260709 13:01:10.449798  1703 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:10.451946  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 892237us
I20260709 13:01:10.454959  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:10.462147  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 359523us
I20260709 13:01:10.410733  3208 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:10.453135  3143 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:10.467372  3198 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.407474  3225 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.464628  3177 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:09.510298   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:10.477278  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.494575  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:10.473410  1630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.002s	sys 0.014s
I20260709 13:01:10.453385  3339 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:10.474633  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:10.501955  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 327831us
I20260709 13:01:10.473682  3162 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:10.488267  3207 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.466949  3314 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:10.504448  3207 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:10.505002  3177 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.495647  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:10.495932  1630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.002s	sys 0.015s
I20260709 13:01:10.483256  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.484071  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1747) took 1898 ms. Trace:
I20260709 13:01:10.481637  3208 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.506186  3314 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.457239  3210 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.497704  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.508121  1629 rpcz_store.cc:276] 0709 13:01:08.585455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.585603 (+   148us) service_pool.cc:224] Handling call
0709 13:01:08.585952 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:08.627421 (+ 41469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.627449 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.627491 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:08.627517 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.483740 (+1856223us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b5400297cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:10.483824 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.483998 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94935296}
I20260709 13:01:10.503268  3189 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.511907  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:10.533805  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.507282  3354 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:10.514374  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.489550  3211 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:10.518790  1675 leader_election.cc:400] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3.
I20260709 13:01:10.537830  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.538144  1675 leader_election.cc:403] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:10.528183  1556 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:10.530658  3352 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:10.527350  3143 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.510195  3290 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.514173  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.535310  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.516681  3302 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:10.497005  3198 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.549031  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.550103  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.550972  3198 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.539130  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.551687  1553 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:10.551813  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.552453  1558 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:10.552650  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.553146  3241 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.553229  1553 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:10.540472  3355 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.554175  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.554436  3341 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3.
I20260709 13:01:10.555212  3311 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.546947  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:10.559461  3191 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.895s	user 0.007s	sys 0.001s
I20260709 13:01:10.518725  3318 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.564122  3318 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.564663  3318 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:10.564970  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 989) took 1470 ms (client timeout 1531 ms). Trace:
W20260709 13:01:10.565366  1559 rpcz_store.cc:269] 0709 13:01:09.094466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.094565 (+    99us) service_pool.cc:224] Handling call
0709 13:01:10.564800 (+1470235us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54003fa054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.564914 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234897664}
I20260709 13:01:10.566563  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1932) took 1052 ms. Trace:
I20260709 13:01:10.566399  3220 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.567042  3327 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.540983  1704 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:10.568151  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1939) took 1042 ms. Trace:
I20260709 13:01:10.568595  3225 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.568786  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.542448  3333 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.569371  3333 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.569532  3336 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.569763  3333 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.569892  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.570062  3336 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.571002  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.516427  3298 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.566984  1628 rpcz_store.cc:276] 0709 13:01:09.513970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.515006 (+  1036us) service_pool.cc:224] Handling call
0709 13:01:09.515357 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.657341 (+141984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.657384 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.657439 (+    55us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:09.657476 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.566359 (+908883us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54003444d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:10.566404 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.566513 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326037760}
I20260709 13:01:10.572009  3298 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.533073  3189 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.545547  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.533301  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.548466  3302 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:10.551062  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.601159  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:10.560047  3339 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.561036  3210 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.561313  3207 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.542047  3211 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.562403  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.563334  1700 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:10.564066  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.619578  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:10.620932  1703 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:10.622156  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.540617  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:10.623585  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83023.5us
I20260709 13:01:10.574213  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.568526  1634 rpcz_store.cc:276] 0709 13:01:09.526065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.122121 (+596056us) service_pool.cc:224] Handling call
0709 13:01:10.122414 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.234685 (+112271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.234712 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.234748 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:10.234775 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.567955 (+333180us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540019c154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:10.567997 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.568098 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257790720}
I20260709 13:01:10.544319  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:10.624943  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50806.8us
I20260709 13:01:10.625216  1556 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
W20260709 13:01:10.508989  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 233078us
I20260709 13:01:10.554209  3355 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.629961  3355 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.540994  3338 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.635566  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:10.583652  3298 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.586541  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.586892  1552 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:10.590873  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:10.591593  3336 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:10.573614  3191 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.648681  1599 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.577672  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.597513  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 1928) took 1090 ms. Trace:
I20260709 13:01:10.649706  1631 rpcz_store.cc:276] 0709 13:01:09.506985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.507110 (+   125us) service_pool.cc:224] Handling call
0709 13:01:10.597104 (+1089994us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54001414d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.597353 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160376576}
I20260709 13:01:10.598415  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:10.650874  3211 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:10.651217  1553 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.608944  3302 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:10.652338  3125 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.652552  3302 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.654567  3125 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.600116  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1741) took 1444 ms. Trace:
I20260709 13:01:10.656096  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.609865  3220 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:10.610155  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:10.668809  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122236us
I20260709 13:01:10.669540  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.658326  1560 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:10.615471  3303 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.595441  3189 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.619287  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:10.625856  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.627694  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:10.672658  1558 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:10.628016  3354 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.628388  3210 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.628772  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.629323  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:10.636238  2290 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:10.594645  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.664885  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.673172  2295 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
W20260709 13:01:10.674103  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48333.7us
I20260709 13:01:10.674961  3357 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.676716  3363 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:10.677402  3241 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:10.678083  2293 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:10.678177  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.678779  3191 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:10.791503  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 196904us
I20260709 13:01:10.791838  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.792791  1674 leader_election.cc:400] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3.
I20260709 13:01:10.771909  1553 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:10.794128  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.778211  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1820) took 1254 ms. Trace:
I20260709 13:01:10.771968  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 360) took 1342 ms. Trace:
I20260709 13:01:10.772742  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:10.796922  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132131us
I20260709 13:01:10.795284  1707 rpcz_store.cc:276] 0709 13:01:09.523791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.552250 (+ 28459us) service_pool.cc:224] Handling call
0709 13:01:10.778002 (+1225752us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.778143 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4884224}
I20260709 13:01:10.686662  1557 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
W20260709 13:01:10.797739  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106743us
I20260709 13:01:10.800220  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.796612  2296 rpcz_store.cc:276] 0709 13:01:09.429095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.429225 (+   130us) service_pool.cc:224] Handling call
0709 13:01:10.771803 (+1342578us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:10.771912 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476791040}
I20260709 13:01:10.656306  1554 rpcz_store.cc:276] 0709 13:01:09.142740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.153218 (+ 10478us) service_pool.cc:224] Handling call
0709 13:01:09.231531 (+ 78313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.626267 (+394736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.626294 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.626724 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.265143 (+638419us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b1800387520. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:10.472272 (+207129us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:10.472324 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.573245 (+100921us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540036e7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:10.573296 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.586480 (+ 13184us) raft_consensus.cc:1685] finished
0709 13:01:10.586500 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.586630 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":648,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9424,"spinlock_wait_cycles":923014912,"thread_start_us":206916,"threads_started":1,"wal-append.queue_time_us":949637}
I20260709 13:01:10.801373  3314 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.805157  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.805578  3314 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.798418  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.806010  3314 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.806592  3364 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:10.806655  3338 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:10.801451  3226 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:10.807288  3338 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.795931  3357 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.807840  3338 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:10.807668  3226 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:10.809792  3226 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:10.811882  3357 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.680374  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.813892  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.683435  3125 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:10.837884  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.798389  3211 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.671974  3311 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:10.842444  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 214205us
I20260709 13:01:10.841032  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.668223  3372 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:10.679425  3336 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:10.801746  3191 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:10.846879  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.847188  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.849041  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:10.818539  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 304407us
I20260709 13:01:10.793203  3318 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.833753  3327 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.815521  3354 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:10.734381  3162 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.850373  3363 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.838881  3226 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:01:10.851076  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:10.846104  3321 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:10.852053  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.853438  2263 leader_election.cc:400] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:10.825234  3210 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.837617  3341 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:10.856830  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.843899  3163 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:01:10.864588  2262 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:10.656315  3333 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.864037  3354 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.847973  3311 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:10.864992  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70437.5us
I20260709 13:01:10.815215  3314 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.875069  1703 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:10.880762  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.873100  3363 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:10.918059  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71002.3us
I20260709 13:01:10.875319  2263 leader_election.cc:403] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:10.860033  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:10.901551  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.844561  3143 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:01:10.864109  3360 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:10.902441  2262 leader_election.cc:403] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:10.846261  3211 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:10.874342  3321 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.880620  3208 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.845808  3355 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:10.884968  3341 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.941881  3229 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:10.941109  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.910614  3362 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:10.940402  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.941037  1704 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:10.938558  3363 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.866050  3372 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:01:10.901590  3125 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:10.981593  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.985591  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.910943  3311 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.985286  3339 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:10.939157  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 79215.8us
W20260709 13:01:10.994345  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132867us
I20260709 13:01:10.983630  3211 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.999966  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:10.881510  3210 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.986326  3241 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.011272  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:10.985728  3321 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:10.984918  3373 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.010742  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:11.015522  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132421us
W20260709 13:01:10.939826  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 96743.2us
I20260709 13:01:11.010285  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.011246  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.012689  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:11.013837  1704 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.026369  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.013259  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:10.983943  3336 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.993661  3341 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:10.996663  3327 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:11.001720  2291 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:10.917966  3207 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:01:11.022436  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.022599  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:11.091869  3356 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.557s	user 0.008s	sys 0.002s
I20260709 13:01:10.999272  3362 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:11.133890  3362 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.135190  3362 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.145038  3318 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.152995  3314 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:11.154706  3381 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.355s	user 0.001s	sys 0.001s
I20260709 13:01:11.017407  3339 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.157750  3339 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.161104  3338 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.026588  1634 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:11.171536  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.030071  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:11.172850  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.173396  2297 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:11.174160  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.028293  1631 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:11.028827  2263 leader_election.cc:400] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4.
I20260709 13:01:11.175012  2263 leader_election.cc:403] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:01:11.175477  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 146760us
I20260709 13:01:11.027189  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.076015  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.177624  2293 raft_consensus.cc:2368] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:11.062530  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.178201  1552 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:11.076237  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.178349  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.077435  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.029088  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.080619  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.079138  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.081717  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.027385  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:11.081250  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46027us
I20260709 13:01:11.179864  2296 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.082276  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.026103  3386 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.180634  3386 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:11.181030  3386 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.087450  3387 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:11.090131  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.184257  3387 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.184950  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.185503  1558 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.186268  2296 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:11.186393  3387 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.014073  3378 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:11.091070  3383 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:01:11.191634  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.187644  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.191916  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.047540  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.192994  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.193559  1627 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:11.193977  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.195348  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.194026  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.196189  1553 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:11.095685  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.096614  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.197016  1629 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:11.197212  1555 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:11.197381  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.098297  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.014725  3241 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:11.030910  3303 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.202651  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.104130  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:11.104717  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 74414.7us
I20260709 13:01:11.203344  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:11.203640  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 99885.4us
I20260709 13:01:11.204097  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.091190  3357 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:11.107818  2294 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:11.010977  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.082235  3336 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.219746  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.220470  3336 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.221848  3229 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:11.223184  3229 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.223841  3396 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.223917  3237 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.224278  3177 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.080132  3191 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:11.090098  3379 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:01:11.230552  1628 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:11.231803  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.109676  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.111104  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.111706  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.109405  3189 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:11.114423  3376 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:01:11.098865  3225 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.115906  3390 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:01:11.116801  3374 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:01:11.118543  3355 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.121999  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.122035  3208 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.110270  3226 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:11.246798  3226 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.247185  3354 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.248814  3226 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.249270  2295 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:11.243196  3386 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.127475  3210 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.253012  3321 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.253896  3210 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.254103  3386 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.165066  3253 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.254568  3253 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:11.254659  2292 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:11.254935  3321 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.255345  3253 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.255725  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:11.170250  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 348057us
I20260709 13:01:11.257356  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.179945  1704 raft_consensus.cc:2368] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.258000  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.258503  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.259099  1704 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:11.259917  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.080974  3143 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.187408  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.188577  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.189625  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.262476  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.262701  2296 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.031944  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.197788  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.197813  1556 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:11.266556  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 234668us
I20260709 13:01:11.198985  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.267700  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.208449  1709 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.268064  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.208993  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.203362  3241 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.211089  1702 raft_consensus.cc:2368] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6.
I20260709 13:01:11.214212  3338 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:11.213954  3311 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.203065  3303 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:11.207960  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 114115us
W20260709 13:01:11.217133  3380 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.417s	user 0.001s	sys 0.001s
I20260709 13:01:11.217353  3339 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.294787  3360 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.295255  3360 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:11.295449  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.295083  1457 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:11.179194  1630 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3. Re-sending same reply.
I20260709 13:01:11.296146  1709 raft_consensus.cc:2368] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.296908  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.297114  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.296757  1457 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:11.218606  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.297883  1709 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.297894  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.298641  1630 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.228258  1558 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:11.301015  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.233121  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.229570  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.236491  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.238477  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:11.302078  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69048.4us
I20260709 13:01:11.239012  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.240595  3357 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:11.310578  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1012) took 1077 ms. Trace:
I20260709 13:01:11.296236  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:11.311342  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.311872  2294 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.241196  3189 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:11.241662  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 175) took 1713 ms (client timeout 1986 ms). Trace:
I20260709 13:01:11.245081  3373 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.191408  3377 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4.
I20260709 13:01:11.132525  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.279799  3385 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.165174  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.258710  1706 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.297180  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 356) took 1136 ms. Trace:
I20260709 13:01:11.260933  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.328948  1704 raft_consensus.cc:2368] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.012365  3325 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:11.272225  1552 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.274219  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.330876  1627 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.279436  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.224756  3177 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.284004  2260 leader_election.cc:400] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.314671  3390 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.333683  2260 leader_election.cc:403] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.280340  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.334859  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.284271  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.229368  3394 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:11.260192  3333 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:11.273003  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 173) took 1745 ms. Trace:
I20260709 13:01:11.291623  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 296) took 5440 ms. Trace:
I20260709 13:01:11.225948  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.302724  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.302846  1628 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.339375  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.339723  1525 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.303398  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:11.336562  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 114096us
I20260709 13:01:11.341691  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:11.342741  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.343250  1632 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:11.345065  1632 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:11.346098  1632 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:11.346606  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62735.1us
I20260709 13:01:11.347911  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:11.289726  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:11.313782  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:11.315294  1633 raft_consensus.cc:2368] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:11.310948  1560 rpcz_store.cc:276] 0709 13:01:10.233118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.233269 (+   151us) service_pool.cc:224] Handling call
0709 13:01:11.310448 (+1077179us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b540036ecd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:11.310528 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703317248}
I20260709 13:01:11.315869  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:11.317108  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98576.2us
I20260709 13:01:11.324786  3229 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.272305  3143 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.320624  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.284435  3396 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.294976  3339 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:11.279172  3378 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.281397  3303 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.325225  3385 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:11.256505  3253 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:11.326695  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.300024  3386 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.336964  2292 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:11.261363  3211 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.329366  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
W20260709 13:01:11.335963  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51727.7us
I20260709 13:01:11.234376  3363 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.260267  3191 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.351955  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:11.353286  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:11.484690  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 131454us
W20260709 13:01:11.314766  1705 rpcz_store.cc:269] 0709 13:01:09.528216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.285242 (+757026us) service_pool.cc:224] Handling call
0709 13:01:11.241507 (+956265us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:11.241609 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106365824}
I20260709 13:01:11.353437  1601 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.325882  3226 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:11.403198  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 82618.9us
I20260709 13:01:11.336910  2289 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:11.453787  1706 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:11.279022  3355 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.328630  2291 rpcz_store.cc:276] 0709 13:01:10.160825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.167600 (+  6775us) service_pool.cc:224] Handling call
0709 13:01:10.590711 (+423111us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:11.297005 (+706294us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54005a0fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:11.297125 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825573504}
I20260709 13:01:11.456974  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.496587  2293 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:11.496678  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 144787us
I20260709 13:01:11.329402  3354 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.337261  3325 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.337986  1455 rpcz_store.cc:276] 0709 13:01:05.850875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.851032 (+   157us) service_pool.cc:224] Handling call
0709 13:01:11.291547 (+5440515us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:07.057205 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:07.057755 (+   550us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:07.057784 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:01:07.057801 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:07.058386 (+   585us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:07.058429 (+    43us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:07.058456 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:01:07.059301 (+   845us) tablet.cc:701] Row locks acquired
0709 13:01:07.059318 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:01:07.059454 (+   136us) write_op.cc:270] Start()
0709 13:01:07.059561 (+   107us) write_op.cc:276] Timestamp: P: 1783602067059416 usec, L: 0
0709 13:01:07.059580 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:01:07.084443 (+ 24863us) log.cc:844] Serialized 1701 byte log entry
0709 13:01:08.623391 (+1538948us) op_driver.cc:464] REPLICATION: finished
0709 13:01:08.845498 (+222107us) write_op.cc:301] APPLY: starting
0709 13:01:08.870397 (+ 24899us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:08.932199 (+ 61802us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:08.932310 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:08.933002 (+   692us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:08.933193 (+   191us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:01:08.933246 (+    53us) write_op.cc:312] APPLY: finished
0709 13:01:08.933495 (+   249us) log.cc:844] Serialized 41 byte log entry
0709 13:01:08.933695 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:08.933819 (+   124us) write_op.cc:454] Released schema lock
0709 13:01:08.933889 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:08.934234 (+   345us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:08.934267 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58612,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":141881,"apply.run_cpu_time_us":2446,"apply.run_wall_time_us":88716,"num_ops":4,"prepare.queue_time_us":384265,"prepare.run_cpu_time_us":3720,"prepare.run_wall_time_us":35046,"replication_time_us":1550896,"thread_start_us":185616,"threads_started":1}]]}
W20260709 13:01:11.468175  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 171) took 1334 ms (client timeout 1559 ms). Trace:
I20260709 13:01:11.472381  1627 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:11.474653  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:11.331586  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.380308  3161 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.257385  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.456291  2290 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:11.402642  3143 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.296056  3360 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.339680  3125 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.403614  3396 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:11.452136  3339 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.402472  3373 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.531669  3211 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.531898  3373 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:11.532305  3373 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.533172  3363 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.451458  3393 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.354547  3327 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.415748  3321 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.458912  3390 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.485910  1601 leader_election.cc:403] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.488242  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.472532  3385 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.491753  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.495148  1628 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:11.497624  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 157965us
I20260709 13:01:11.498297  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.546128  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.503597  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:11.501487  1557 rpcz_store.cc:269] 0709 13:01:10.133621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.133996 (+   375us) service_pool.cc:224] Handling call
0709 13:01:11.467933 (+1333937us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54002f9254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:11.468096 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616857856}
I20260709 13:01:11.406817  3394 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.515869  3403 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.515843  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.489086  3362 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:11.517513  1703 raft_consensus.cc:2368] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
W20260709 13:01:11.550329  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 58666.2us
I20260709 13:01:11.513111  3364 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:11.523550  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.525403  1554 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:11.479672  3386 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.526882  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.563169  1552 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.527978  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.573978  1551 raft_consensus.cc:2368] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.574100  3386 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.537688  3393 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:11.525907  3396 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.574672  3393 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.574509  3339 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.545024  3315 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.536677  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.340106  1701 rpcz_store.cc:276] 0709 13:01:09.527779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.340384 (+812605us) service_pool.cc:224] Handling call
0709 13:01:10.790234 (+449850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.790443 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.790474 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.790523 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.790574 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.790638 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.250326 (+459688us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54003dca50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:11.269371 (+ 19045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006808832}
I20260709 13:01:11.537602  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.517068  3161 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:11.541563  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53402.1us
I20260709 13:01:11.544520  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.591414  1598 leader_election.cc:400] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.591892  1598 leader_election.cc:403] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.592816  1598 leader_election.cc:400] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:11.593073  1598 leader_election.cc:403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.544631  3325 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:11.546994  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 206160us
W20260709 13:01:11.547441  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43934.7us
I20260709 13:01:11.594846  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.548856  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.550465  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.548508  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:11.550575  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.557034   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:11.596153  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47728.8us
W20260709 13:01:11.596381  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45870.9us
I20260709 13:01:11.540473  3125 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.570750  3225 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:11.537902  3341 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:11.572331  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.573256  1552 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:11.576503  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.620687  2294 raft_consensus.cc:2376] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3. Re-sending same reply.
I20260709 13:01:11.580211  3162 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.584154  1555 raft_consensus.cc:2376] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2. Re-sending same reply.
I20260709 13:01:11.585783  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:11.597198  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.597441  1524 leader_election.cc:400] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:11.626623  1524 leader_election.cc:403] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.598793  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.599814  1703 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.603753  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.606163  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.627810  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.628808  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.629693  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.565080  2293 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:11.608232  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.611013  3177 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.611665  3339 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.614125  3225 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.615161  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.615615  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.562367  3363 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.561962  3365 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.643591  3311 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.611696  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.622097  3387 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4.
I20260709 13:01:11.621618  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.685571  2294 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:11.686614  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.687333  2294 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.536164  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:11.631049  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.590245  3161 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.591147  3364 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:11.588622  3362 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:11.640355  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.555756  3229 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:11.738281  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97980.9us
I20260709 13:01:11.739687  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.748901  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:11.754273  3377 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:11.754617  3333 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:11.754806  3377 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:11.755146  3333 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:11.755736  3377 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.756363  3333 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.759565  3364 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:11.763441  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 164686us
I20260709 13:01:11.769855  2294 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:11.778894  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.742038  3229 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.780817  3229 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.742823  3378 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.791981  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:11.651672  1709 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:11.631026  1598 leader_election.cc:400] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6.
I20260709 13:01:11.622520  3390 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:11.661828  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64500.1us
I20260709 13:01:11.801622  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:11.645682  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48546.7us
I20260709 13:01:11.750999  3327 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:11.625026  3325 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:11.632323  3162 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:11.665679  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62034.3us
I20260709 13:01:11.759743  3225 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.820374  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.648840  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:11.631953  3386 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:11.668799  1705 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:11.648551  3339 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:11.661309  3404 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.625542  3400 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.640400  3355 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.580355  3403 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.664381  3253 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.664526  3125 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:11.651935  1703 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:11.668560  3315 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.648013  3354 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:11.683089  3303 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.700857  3394 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.701114  3341 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.681519  3226 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.701700  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.681926  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:11.719465  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.570799  3373 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.813992  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.762360  3241 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.821374  1702 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.884274  3357 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:11.888749  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74922.3us
I20260709 13:01:11.830003  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:11.892112  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62210.5us
I20260709 13:01:11.893018  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:11.835162  3404 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.894457  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.811478  3363 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:11.837391  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 72832us
I20260709 13:01:11.822789  3162 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.896080  3198 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.896137  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.897111  3198 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:11.897585  3198 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.898090  3198 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.900141  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.829190  3387 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.829090  3339 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.901213  3377 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.901321  2295 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:11.901564  3339 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:11.900173  3315 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.897625  3355 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.832846  3408 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:11.833247  3325 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:11.904326  2290 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:11.904688  3325 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.906278  2290 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:11.906891  3315 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:11.904341  3408 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:11.830981  3143 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.910095  3341 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:11.911188  1556 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:11.913142  2297 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:11.910373  3143 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:11.818089  3362 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.829207  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:11.890398  3241 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.942198  2260 leader_election.cc:400] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.920660  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:11.946282  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.942457  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:11.922644  2293 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:11.941745  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47391.9us
I20260709 13:01:11.947588  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:11.941790  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 174579us
I20260709 13:01:11.948765  2260 leader_election.cc:403] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.947966  3143 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.948865  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:11.954010  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.954401  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:11.955658  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.956537  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.958786  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.959031  1552 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:11.958848  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:11.963320  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.963581  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.966162  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:11.967600  3416 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.968184  3416 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:11.969386  3416 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:11.970925  3416 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.972976  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:11.976976  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:11.982148  1552 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:11.941166  3408 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:11.954125  3363 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.953415  3354 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:11.984278  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:12.001711  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:12.006645  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:12.007299  1552 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:11.956874  3357 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:11.959651  3377 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.959565  3315 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.975070  3143 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.960400  3241 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.972450  3384 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.972404  3298 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.972584  3189 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.960007  3333 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:11.972249  3400 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.107313  3400 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.107955  3400 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.975411  3220 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.051952  3357 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:12.056779  1555 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:12.057943  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56322.4us
I20260709 13:01:12.118359  1601 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:12.048617  3315 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.058704  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
W20260709 13:01:12.063577  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90675.6us
I20260709 13:01:11.983420  3303 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:11.984301  3237 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:12.072190  1552 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:11.959550  3355 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:01:11.985483  3225 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.018544  3191 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:12.071368  3387 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.092267  1559 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:12.092119  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:12.092679  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 462607us
I20260709 13:01:12.148379  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:12.093895  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:12.094450  1551 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:12.095110  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:12.152377  1558 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:12.106249  3229 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:12.106923  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.095801  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:12.058027  3327 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:12.119364  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:12.058107  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 177) took 2528 ms (client timeout 1812 ms). Trace:
I20260709 13:01:12.133499  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:12.142799  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.154707  1559 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:12.157717  2263 leader_election.cc:400] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3.
I20260709 13:01:12.157150  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:12.191192  1708 rpcz_store.cc:269] 0709 13:01:09.529551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.129425 (+599874us) service_pool.cc:224] Handling call
0709 13:01:12.057896 (+1928471us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001342d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:12.058039 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514371968}
I20260709 13:01:12.162590  2290 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:12.193166  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:12.193379  2297 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:12.194229  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:12.195466  1702 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:12.195711  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:12.196414  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:12.196964  1702 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.196604  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.170097  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:12.197144  2297 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:12.157620  2291 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:12.198618  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:12.172711  3405 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.161567  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:12.194006  3378 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.199968  1708 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:12.199692  3405 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.201342  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.203610  1708 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:12.204082  3405 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.204562  1702 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:12.204811  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:12.205470  2295 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:12.205819  1702 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:12.206852  3333 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:12.212414  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47235.5us
I20260709 13:01:12.127249  3315 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:12.148805  3396 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.006s
I20260709 13:01:12.136278  3415 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:12.216228  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1759) took 1136 ms. Trace:
I20260709 13:01:12.137668  3325 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.194401  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:12.191618  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:12.224756  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76476.3us
I20260709 13:01:12.197232  3229 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.211838  3404 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.141124  3189 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:12.230299  1633 rpcz_store.cc:276] 0709 13:01:11.079955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.082053 (+  2098us) service_pool.cc:224] Handling call
0709 13:01:12.216027 (+1133974us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:12.216166 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309808512}
I20260709 13:01:12.217494  1705 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.141353  3298 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:12.149928  3387 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:12.239982  1557 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.226819  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:12.247300  3426 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:12.129899  3384 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 13:01:12.252705  2290 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:12.225718  3403 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.225602  3237 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:01:12.276640  1558 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:12.260843  3220 raft_consensus.cc:2764] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:12.282290  1552 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:12.262408  3387 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.275100  1554 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:12.253309  3357 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.239283  3415 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.236162  3396 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.279103  3373 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.304240  2296 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:12.303357  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:12.312111  3385 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 13:01:12.265089  3189 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.332345  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:12.333552  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:12.333966  1634 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.268584  3298 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:12.299693  3412 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 13:01:12.316601  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1016) took 1065 ms. Trace:
I20260709 13:01:12.318255  3371 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.314972  1557 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:12.327260  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:12.347990  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.324869  3430 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:12.328291  1627 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:12.336023  1524 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.801s (SENT)
I20260709 13:01:12.336421  1633 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:12.349598  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:12.350010  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:12.350806  1633 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:12.351346  1627 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:12.335386  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:12.352105  1633 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:12.352193  1634 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:12.337963  1631 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:12.339857  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:12.313793  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 381) took 1143 ms. Trace:
I20260709 13:01:12.319823  3191 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.346156  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.360922  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:12.287549  3225 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.348400  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.364508  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:12.359115  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 145861us
I20260709 13:01:12.362085  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:12.328624  3405 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.385306  1703 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:12.392041  1627 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:12.414161  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:12.361984  3404 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:12.356061  3430 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:12.337013  3385 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.423628  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:12.424880  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1834) took 1866 ms. Trace:
I20260709 13:01:12.428388  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:12.428587  1631 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:12.429334  1633 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:12.428471  1707 rpcz_store.cc:276] 0709 13:01:10.558687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.402864 (+844177us) service_pool.cc:224] Handling call
0709 13:01:11.467913 (+ 65049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.468249 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.468276 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.468719 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.679354 (+210635us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:11.679396 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.679729 (+   333us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.424613 (+744884us) raft_consensus.cc:1685] finished
0709 13:01:12.424643 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.424809 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":458,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2896,"spinlock_wait_cycles":22400,"thread_start_us":203255,"threads_started":1,"wal-append.queue_time_us":347256}
I20260709 13:01:12.430684  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:12.395097  1628 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:12.368597  2291 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.369839  3405 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:12.441226  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45024.7us
I20260709 13:01:12.444165  1525 leader_election.cc:400] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
W20260709 13:01:12.435515  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 316255us
I20260709 13:01:12.447907  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.383309  3371 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.449103  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:12.449105  1630 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:12.449265  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.415889  3373 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:01:12.444917  3423 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:01:12.356317  3162 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.433723  3404 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:12.355880  2292 rpcz_store.cc:276] 0709 13:01:11.168455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.255451 (+ 86996us) service_pool.cc:224] Handling call
0709 13:01:12.296466 (+1041015us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b5400350cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:12.311896 (+ 15430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168806656}
W20260709 13:01:12.455207  3390 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.000s
I20260709 13:01:12.420306  3426 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:12.457906  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:12.454876  3325 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.383734  3225 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.370797  3191 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:12.447230  3321 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.901s	user 0.008s	sys 0.004s
I20260709 13:01:12.485014  3321 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.450489  3405 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.451897  3237 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:01:12.467562  3434 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:01:12.455001  3372 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:12.485203  3177 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:12.481660  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.508632  1635 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:12.458213  3422 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:12.485041  3411 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.784s	user 0.008s	sys 0.003s
W20260709 13:01:12.512980  3161 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.003s
I20260709 13:01:12.495292  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:12.513103  2297 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:12.485328  3393 consensus_meta.cc:287] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.910s	user 0.008s	sys 0.007s
I20260709 13:01:12.514321  3298 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.514243  3393 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.518464  1634 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:12.519304  1627 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.477835  3415 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.499861  3396 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.500156  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:12.495932  3363 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:01:12.479756  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:12.496215  3321 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:01:12.496395  3143 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:01:12.502446  3385 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.504230  3371 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.504352  3372 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.549590  3372 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:12.343880  1701 rpcz_store.cc:276] 0709 13:01:11.250989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.854489 (+603500us) service_pool.cc:224] Handling call
0709 13:01:12.020061 (+165572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.070260 (+ 50199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:12.210391 (+140131us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.316272 (+105881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:12.316326 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:12.316365 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.316431 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.316547 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48985,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":128196,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9253,"thread_start_us":80454,"threads_started":1,"wal-append.queue_time_us":126848}
W20260709 13:01:12.505306  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1763) took 1421 ms (client timeout 1645 ms). Trace:
W20260709 13:01:12.505584  3339 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.002s
I20260709 13:01:12.484198  1704 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.509016  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:12.485833  3430 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:12.512496  3365 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.002s
I20260709 13:01:12.470448  3390 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.520572  3315 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.519897  3325 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:12.555353  3298 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.525738  3191 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.508224  3177 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.564683  3177 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:12.566079  3315 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.526679  3162 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:12.528344  3198 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.630s	user 0.004s	sys 0.005s
I20260709 13:01:12.568440  3198 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.570133  1629 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:12.528815  3241 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:12.550802  1626 rpcz_store.cc:269] 0709 13:01:11.083934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.191397 (+107463us) service_pool.cc:224] Handling call
0709 13:01:12.504702 (+1313305us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400343ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:12.505235 (+   533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941488896}
I20260709 13:01:12.554685  3387 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.572255  3241 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.559017  3339 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.572849  3241 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:12.573472  1702 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:12.574506  3339 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.516991  3386 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.575574  3386 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.530658  3225 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.480404  3404 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.461685  3431 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
W20260709 13:01:12.535923  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54355.9us
I20260709 13:01:12.536427  3336 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.536294  3426 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.537189  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:12.548168  3393 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:12.553732  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44781.8us
W20260709 13:01:12.577657  3424 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:01:12.557202  3365 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.573983  3241 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.513464  3161 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.571358  3198 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.580688  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.609256  3225 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:01:12.614035  3365 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.614415  3365 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:12.614537  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:12.614816  3365 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.616014  3162 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:12.616173  3371 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:12.616448  3211 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.617116  3211 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:12.620430  1700 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:12.576810  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:12.598243  3336 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:12.606540  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:12.552416  2289 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:12.622242  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:12.629663  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.634248  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:12.665607  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.628720  3336 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:12.676450  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 219384us
I20260709 13:01:12.633702  3431 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.676069  3211 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:12.635229  3225 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:12.685781  3225 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:12.636407  3415 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:12.689080  3211 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:12.665529  3387 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.676450  3442 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:12.635528  3339 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:01:12.687181  3386 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.715740  3415 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:12.716689  3415 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.688745  3225 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:12.721567  3400 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
W20260709 13:01:12.754251  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 316296us
I20260709 13:01:12.754745  3405 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.757589  3405 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.757889  3161 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.762188  3372 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.769168  3211 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.818068  3372 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.786700  3431 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.751271  3400 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:12.832604  3141 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:12.840941  3141 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.818799  3431 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.846705  3416 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:12.850778  3360 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:12.853933  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 362067us
I20260709 13:01:12.855155  3241 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.859359  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:12.843199  3141 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.865274  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:12.866423  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:12.870319  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:12.871344  3357 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:12.871541  3452 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:12.871837  3357 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.871407  3405 raft_consensus.cc:2764] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:01:12.872233  3357 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:12.855333  3177 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.836773  3365 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.882541  3177 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.893879  3365 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.894188  3311 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:12.894709  3311 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.896064  3290 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:12.896449  3290 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:12.896631  3311 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:12.897222  3290 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.884054  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:12.898389  3290 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:12.898483  2289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 385) took 1345 ms (client timeout 1649 ms). Trace:
W20260709 13:01:12.899116  2289 rpcz_store.cc:269] 0709 13:01:11.553098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.555694 (+  2596us) service_pool.cc:224] Handling call
0709 13:01:12.479640 (+923946us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:12.898400 (+418760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39850240}
I20260709 13:01:12.899834  3431 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.884424  3400 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.838622  3443 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:12.901862  3443 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:12.902303  3443 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:12.905304  3336 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.907297  3336 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:12.905303  3357 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.879341  3405 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:12.845717  3161 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:01:12.874805  3404 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:12.880290  3452 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.917136  3405 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.917780  3452 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.919245  3405 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:12.912673  3421 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.762s	user 0.003s	sys 0.001s
I20260709 13:01:12.919826  3452 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.903301  3443 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:12.921069  3404 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.921271  3443 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:12.921530  3404 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:12.922233  3357 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.928296  3404 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.930143  3404 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.930438  3357 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.931005  3357 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:12.933422  3357 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:12.959683  3189 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.000404  3189 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.000993  3189 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.003618  3189 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.022599  2292 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:13.026384  3357 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:13.027443  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.027277  3357 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.028010  1559 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.030000  3357 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.036810  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:13.038367  3226 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:13.038878  3226 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.041966  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 382) took 1869 ms. Trace:
I20260709 13:01:13.042220  2296 rpcz_store.cc:276] 0709 13:01:11.172464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.265646 (+ 93182us) service_pool.cc:224] Handling call
0709 13:01:12.303926 (+1038280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.326129 (+ 22203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.041813 (+715684us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400350cd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:13.041916 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387266432}
I20260709 13:01:13.044528  3226 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.046391  3226 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.081463  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.086527  1553 raft_consensus.cc:2368] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
W20260709 13:01:13.083907  2260 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.775s (SENT)
I20260709 13:01:13.092739  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.093750  1559 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.095546  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.097138  1601 leader_election.cc:400] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3.
I20260709 13:01:13.098726  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.099613  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.101112  3384 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.101241  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.103328  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.105054  1553 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.107209  1559 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:13.108812  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.109113  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.110415  3384 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:13.110526  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.111074  3384 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.112300  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.112476  3384 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.113479  1559 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:13.113498  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 178) took 1585 ms. Trace:
I20260709 13:01:13.114678  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:13.114640  1556 rpcz_store.cc:276] 0709 13:01:11.535685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.535954 (+   269us) service_pool.cc:224] Handling call
0709 13:01:13.112986 (+1577032us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540036f1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:13.113277 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512471936}
I20260709 13:01:13.116750  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.124549  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.124740  1559 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:13.126271  1559 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.131809  1560 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:13.134636  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.136606  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.137602  1553 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:13.138059  3315 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.138873  3315 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.139209  3290 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.139382  3400 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.140436  3400 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.140375  3315 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.143112  3400 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.145256  3357 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.146669  3357 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:13.146811  3448 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.147128  3357 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:13.147830  3448 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.150557  1631 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:13.150723  3448 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.151261  3357 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.152974  1631 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.153431  3357 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.155400  1631 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:13.158185  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.159515  3189 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.163002  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.164436  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.164983  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.165303  3400 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:13.166388  3189 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.167979  3426 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.167841  3189 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.168676  3189 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:13.168632  3426 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.165314  1631 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:13.167812  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.170672  3426 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.173647  3426 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.173846  3373 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.176476  1551 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:13.170672  3189 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.177942  3189 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.179695  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.180264  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.180351  3311 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.181620  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.182519  1628 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.183162  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.183621  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.184808  3373 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.187712  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.188910  3373 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:13.189368  3373 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.189760  3373 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:13.198572  3448 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:13.199211  3448 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.200464  3448 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.172482  1629 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:13.202171  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.203727  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.225778  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.207108  3311 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:13.210754  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.232378  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.233130  1629 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.233207  3272 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.234035  3272 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.234947  3430 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:13.236110  3443 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:13.236897  3443 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.251726  3443 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.216938  3226 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:13.217561  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.217831  3392 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.218087  3386 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:13.217888  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 200) took 1745 ms (client timeout 1775 ms). Trace:
I20260709 13:01:13.220569  1634 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:13.200941  1635 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:13.266896  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.267652  3386 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.268131  3386 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:13.263190  1706 rpcz_store.cc:269] 0709 13:01:11.472070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.499443 (+ 27373us) service_pool.cc:224] Handling call
0709 13:01:13.217442 (+1717999us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54001333d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:13.217648 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661852928}
I20260709 13:01:13.269167  3386 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.269766  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.270857  3386 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.271358  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.271975  1634 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:13.272080  3226 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.272974  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.228307  1553 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:13.251905  3298 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.180474  1631 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:13.255481  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.258615  3272 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.264547  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.259203  1551 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:13.275133  3298 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:13.278525  3220 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:01:13.284253  3298 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:13.285843  1628 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:13.286298  1626 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:13.288218  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.293208  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.293587  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.294404  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.294783  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.295562  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.296804  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.297967  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.297601  3298 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.300390  3226 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.298550  1635 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.337500  1702 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.340458  1635 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:13.342574  1631 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:13.343880  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.344313  1631 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.344224  3272 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.345184  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.345921  3339 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:13.299453  3298 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:13.346308  3339 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:13.347479  3339 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.348757  3339 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.350194  3392 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:13.354557  1629 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:13.354810  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.355652  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:13.361583  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.358467  3392 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:13.372377  3392 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.372763  3392 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.377043  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.377758  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.378051  1554 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:13.378784  1633 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:13.380455  1557 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:13.382411  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.383486  3315 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.384498  1601 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.384790  1706 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.385563  3315 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.391687  1628 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:13.412071  3315 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:01:13.417066  1633 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:13.416169  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.421299  1627 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:13.415750  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.425880  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.426972  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.427139  1629 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:13.428792  3452 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:13.437937  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 61039.1us
I20260709 13:01:13.439862  3422 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.440390  3422 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:13.441737  3422 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:13.443013  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.443387  3422 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.446465  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.446992  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1003) took 2810 ms. Trace:
I20260709 13:01:13.456686  1706 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:13.458314  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.459266  1706 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:13.460391  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.464679  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.465077  1709 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:13.466413  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1014) took 2235 ms. Trace:
I20260709 13:01:13.447409  1704 rpcz_store.cc:276] 0709 13:01:10.636184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.329503 (+693319us) service_pool.cc:224] Handling call
0709 13:01:11.716446 (+386943us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:11.716650 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.552879 (+836229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.446598 (+893719us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b5400132ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:13.446821 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749533824}
I20260709 13:01:13.466774  1709 rpcz_store.cc:276] 0709 13:01:11.231055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.832844 (+601789us) service_pool.cc:224] Handling call
0709 13:01:13.464629 (+1631785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.465482 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.466210 (+   728us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54001cbbd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:13.466342 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3751868800}
W20260709 13:01:13.463536  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1844) took 1989 ms (client timeout 1795 ms). Trace:
I20260709 13:01:13.472353  3430 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.479892  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.484659  3439 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:13.486867  3439 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.487304  3439 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:13.487923  3450 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.488351  1702 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:13.488591  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.487957  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.489658  3450 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.491227  3450 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:13.490036  3430 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.492991  3443 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:13.493073  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.493535  3443 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:13.493985  3430 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:13.494457  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.494722  3443 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.495436  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.497022  1705 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:13.497687  3430 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:13.477540  1700 rpcz_store.cc:269] 0709 13:01:11.474175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.544298 (+ 70123us) service_pool.cc:224] Handling call
0709 13:01:13.463126 (+1918828us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:13.463360 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348045184}
I20260709 13:01:13.497040  1702 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:13.501470  3314 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.501482  3167 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.503024  3314 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:13.508270  3443 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:13.508015  3430 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:13.491689  3450 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.512185  3450 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.519006  3364 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.520601  3439 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.521580  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 700) took 1292 ms. Trace:
I20260709 13:01:13.522212  3439 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.522121  2295 rpcz_store.cc:276] 0709 13:01:12.228629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.228768 (+   139us) service_pool.cc:224] Handling call
0709 13:01:12.229100 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.355990 (+126890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:12.356023 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:12.356066 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:12.356097 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.521016 (+1164919us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:13.521089 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.521311 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291381504}
I20260709 13:01:13.495970  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.529244  1632 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.529421  1552 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.529637  3314 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.530493  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1979) took 1197 ms. Trace:
I20260709 13:01:13.530846  1632 rpcz_store.cc:276] 0709 13:01:12.332660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.332768 (+   108us) service_pool.cc:224] Handling call
0709 13:01:12.333020 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.529733 (+1196713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.530248 (+   515us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540024f254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:13.530431 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750168832}
I20260709 13:01:13.531242  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.532341  3448 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.533516  3448 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:13.533944  3448 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.534406  3448 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:13.547516  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51630.3us
I20260709 13:01:13.549996  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.553288  3436 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:13.553836  3436 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.555032  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.560410  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.560930  1631 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:01:13.561124  3417 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.604s	user 0.003s	sys 0.000s
I20260709 13:01:13.561295  1632 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:13.564736  3384 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:13.559078  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132180us
I20260709 13:01:13.565348  3384 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.566512  3426 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:13.568673  3477 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.569120  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.569275  3477 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.570544  3161 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.571987  3477 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.555003  3436 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.573052  3477 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.573298  3315 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.574188  3436 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.575150  1702 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.569274  3426 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:13.576272  3426 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.577315  3426 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.577874  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.573239  1523 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:13.573638  3477 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.580168  1523 leader_election.cc:403] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:13.555193  3476 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:13.581630  3373 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.579632  3384 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.555858  3452 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:13.585005  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:13.585609  1556 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:13.582024  3477 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.570943  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.589406  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.589924  3373 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.590243  1702 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:13.591174  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.592301  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.579574  3426 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.591553  3373 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.599658  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.598097  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.601112  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.593238  3477 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:13.596006  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.631896  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.632591  1706 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:13.602983  3163 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:13.646368  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.596966  3384 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:01:13.649724  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:13.652628  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.652356  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.653820  3485 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.654470  3485 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:13.658478  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.660352  1635 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:01:13.660888  1635 pending_rounds.cc:85] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 2
I20260709 13:01:13.661168  1635 pending_rounds.cc:107] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:01:13.661477  1635 raft_consensus.cc:2889] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:13.663417  3161 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.666136  3384 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.666574  3384 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.665642  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:13.667870  1635 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:13.679963  1704 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:13.603055  3373 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.648126  3476 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:13.688273  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.602571  3452 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.693490  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.643218  3448 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.700205  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:13.702720  3477 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:13.703047  3373 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.667446  3384 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.709156  3384 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.713457  3448 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.715247  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:13.715863  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:13.707041  3450 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:13.718726  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.721494  3480 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:13.723316  3373 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:13.723783  3480 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.723973  3373 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.726022  3373 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:13.726536  3373 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.727023  3373 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.728361  3422 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.730499  3314 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.705353  3436 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:01:13.731124  3314 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.733388  3314 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:13.735289  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.736302  1551 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:13.739259  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 207) took 2265 ms. Trace:
I20260709 13:01:13.739671  1708 rpcz_store.cc:276] 0709 13:01:11.473708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.204737 (+731029us) service_pool.cc:224] Handling call
0709 13:01:12.205208 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.342439 (+137231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:12.342483 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:12.342536 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:12.342580 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.738849 (+1396269us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001cb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:13.738929 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.739109 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314970496}
I20260709 13:01:13.741787  3303 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.741601  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:13.742271  3303 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:13.743510  3189 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.750121  3338 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.743397  3303 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.750562  3338 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:13.750984  3303 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.760556  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:13.767455  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:13.768373  1634 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:13.774927  3384 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:13.776783  3384 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:13.783033  3384 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.751673  3338 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:13.792745  3338 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.803982  1700 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:13.804332  3422 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.804883  3422 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.805418  3373 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.806285  3476 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.809805  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 1970) took 1486 ms. Trace:
I20260709 13:01:13.810225  1630 rpcz_store.cc:276] 0709 13:01:12.323601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.323714 (+   113us) service_pool.cc:224] Handling call
0709 13:01:12.323969 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.450000 (+126031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.809285 (+1359285us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000cc6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:13.809611 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287303424}
I20260709 13:01:13.812919  3422 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.814417  3422 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.815069  3422 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.817816  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.824563  3422 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.827116  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.818702  3373 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.830554  3493 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.831640  3493 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.833920  3493 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:13.834637  3373 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:13.836953  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.839454  3404 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.839093  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.840092  3404 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.840937  1707 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:13.841487  3325 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:13.843789  3220 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:13.843951  1524 leader_election.cc:400] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 2. Current term is 3.
I20260709 13:01:13.847561  3392 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.848097  3373 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.844462  3220 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:13.848098  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:13.845291  3404 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.845233  3303 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.852970  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.857617  3493 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.861577  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:13.863030  3493 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:13.851265  3220 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.863391  3303 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.869546  3384 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:13.871629  3303 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:13.869035  1627 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:13.881606  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:13.882529  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:13.883172  1627 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:13.881511  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.884806  3484 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:13.882581  3472 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.410s	user 0.002s	sys 0.000s
I20260709 13:01:13.885216  3484 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.886431  3484 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.887200  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1053) took 1008 ms. Trace:
I20260709 13:01:13.887557  1559 rpcz_store.cc:276] 0709 13:01:12.879063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.126990 (+247927us) service_pool.cc:224] Handling call
0709 13:01:13.127432 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.128017 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.128061 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.128671 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.279048 (+150377us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:13.279073 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.279153 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.881938 (+602785us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:13.886874 (+  4936us) raft_consensus.cc:1685] finished
0709 13:01:13.886898 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.887127 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":211,"spinlock_wait_cycles":4742912,"thread_start_us":147384,"threads_started":1,"wal-append.queue_time_us":258993}
I20260709 13:01:13.890331  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:13.890976  2292 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.898115  1701 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.904479  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 214) took 1101 ms. Trace:
I20260709 13:01:13.896927  1707 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:13.916113  1701 rpcz_store.cc:276] 0709 13:01:12.803049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.805748 (+  2699us) service_pool.cc:224] Handling call
0709 13:01:12.840098 (+ 34350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.904104 (+1064006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.904416 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.956037  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:13.956801  2294 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.959920  3298 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:13.960824  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:13.960606  3298 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:13.969000  2288 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
W20260709 13:01:14.036507  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.969864  2296 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:14.037320  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.015918  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.108446  1627 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:14.083240  2294 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:13.427939  3357 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.110008  3357 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:14.107250  2297 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:13.997527  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.111872  3207 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.116158  3207 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.119710  3207 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.117446  3303 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.116183  3452 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:14.120911  3303 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.121960  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.122605  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:14.122705  2288 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:14.122961  3303 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.121161  3450 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:14.124008  1707 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:14.124068  3392 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.124931  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.111129  3357 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.127489  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.075949  2292 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:14.115296  3290 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:14.132323  3364 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.132488  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:14.133369  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.132499  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.133306  3364 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:14.135447  3364 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.136097  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.137117  3386 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:14.137373  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.128829  2297 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:14.015659  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.142616  3372 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.132812  3290 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.144116  3436 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.073374  3484 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.144430  3372 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:14.147491  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 279505us
I20260709 13:01:14.141317  3480 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.147188  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.146284  3436 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:14.150815  3436 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.151744  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.152712  2290 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 2: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:14.153965  3303 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.154505  3303 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.154836  3290 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.155326  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.156214  3392 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:14.156585  3392 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.156623  3303 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.157034  3392 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.158595  3364 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.147933  3372 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.160069  3364 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:14.162202  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.162745  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.153468  2289 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:14.166628  3373 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.167207  3373 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.168331  3426 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:14.168475  3189 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.168802  3426 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:14.169296  3426 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.169075  3189 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.169795  3373 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.171113  3189 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.166965  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.167980  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.171272  3372 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.178301  1457 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 206) took 3936 ms. Trace:
I20260709 13:01:14.191454  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.179082  3450 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:14.189688  3493 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.179955  3373 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:14.192212  1629 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:14.192246  1631 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:14.192574  3161 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:14.192968  3161 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.180578  3303 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:14.193823  3303 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.193845  3161 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.194607  3290 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.195014  3290 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.195250  3161 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.195328  3290 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.195636  3303 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.181136  3333 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.181391  3225 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.196501  3225 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.198690  3225 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:14.185039  3298 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.186162  3480 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:14.196491  3333 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.203198  3333 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.188177  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.204061  3325 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.191145  1457 rpcz_store.cc:276] 0709 13:01:10.241472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.241633 (+   161us) service_pool.cc:224] Handling call
0709 13:01:14.178013 (+3936380us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:11.586647 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:11.683629 (+ 96982us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:11.691625 (+  7996us) write_op.cc:435] Acquired schema lock
0709 13:01:11.696411 (+  4786us) tablet.cc:662] Decoding operations
0709 13:01:11.780853 (+ 84442us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:11.780932 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:11.780974 (+    42us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:01:11.781313 (+   339us) tablet.cc:701] Row locks acquired
0709 13:01:11.781337 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:01:11.781504 (+   167us) write_op.cc:270] Start()
0709 13:01:11.781640 (+   136us) write_op.cc:276] Timestamp: P: 1783602071781456 usec, L: 0
0709 13:01:11.781670 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:01:11.823023 (+ 41353us) log.cc:844] Serialized 1045 byte log entry
0709 13:01:12.575841 (+752818us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:12.868670 (+292829us) op_driver.cc:464] REPLICATION: finished
0709 13:01:12.911094 (+ 42424us) write_op.cc:301] APPLY: starting
0709 13:01:12.911185 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:12.911307 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:12.911332 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:12.911748 (+   416us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:12.911940 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:01:12.911985 (+    45us) write_op.cc:312] APPLY: finished
0709 13:01:12.915912 (+  3927us) log.cc:844] Serialized 29 byte log entry
0709 13:01:12.916324 (+   412us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.916411 (+    87us) write_op.cc:454] Released schema lock
0709 13:01:12.916454 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:12.916786 (+   332us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.916815 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":828538,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":42107,"apply.run_cpu_time_us":1992,"apply.run_wall_time_us":5665,"num_ops":2,"prepare.queue_time_us":244192,"prepare.run_cpu_time_us":63646,"prepare.run_wall_time_us":533595,"raft.queue_time_us":125069,"raft.run_cpu_time_us":1672,"raft.run_wall_time_us":74086,"replication_time_us":1076741,"spinlock_wait_cycles":16535424,"thread_start_us":519190,"threads_started":4,"wal-append.queue_time_us":424137,"wal-append.run_cpu_time_us":27885,"wal-append.run_wall_time_us":1686476}]]}
I20260709 13:01:14.204591  3325 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.188339  3484 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.205840  3430 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:14.188413  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:14.208921  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:14.208899  3392 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.210502  3315 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.211104  3392 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.210677  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:14.211783  3484 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:14.212462  3430 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.212991  3477 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:14.213364  3477 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:14.214179  3477 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.215621  3477 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:14.215801  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.546s (SENT) [suppressed 34 similar messages]
I20260709 13:01:14.218845  3315 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.189635  3452 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:14.189875  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:14.188951  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.193362  1705 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:14.232472  1628 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.184010  3189 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.201381  3480 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.185719  3426 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.192356  3493 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.207273  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.219499  3325 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.221395  3430 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:14.224488  2297 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.225131  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.226455  3191 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:14.226132  2288 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:14.227252  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.227280  2294 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.213394  3290 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.228796  1552 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:14.229010  1554 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:14.229324  2291 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:14.176672  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:14.230005  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.230598  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.231025  1557 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:14.230975  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.232595  1705 pending_rounds.cc:85] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 1
W20260709 13:01:14.198781  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.244540  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 67961.5us
I20260709 13:01:14.233727  3189 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.236454  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.234724  3480 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.241501  1559 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:14.243103  1555 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:14.244632  1705 pending_rounds.cc:107] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:14.233610  3452 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.239044  3392 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.247746  1705 raft_consensus.cc:2889] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:14.246066  3161 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.257933  3189 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.258121  3426 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.259896  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1795) took 1073 ms. Trace:
I20260709 13:01:14.250870  3477 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:14.264586  1626 rpcz_store.cc:276] 0709 13:01:13.178795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.179864 (+  1069us) service_pool.cc:224] Handling call
0709 13:01:14.252122 (+1072258us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b54000de3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:14.252208 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15914368}
I20260709 13:01:14.279605  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:14.280424  2297 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:14.280917  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.252923  2292 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:14.261842  3161 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.285002  2294 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:14.269433  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.291177  1628 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:14.275969  3226 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:01:14.256312  3325 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:01:14.295368  3493 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.295521  3226 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:14.266613  3398 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:01:14.264917  3220 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:01:14.297833  1555 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:14.286592  3161 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.295931  3226 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.294293  3450 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:01:14.316443  3226 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.317729  3226 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.300979  3452 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.296052  3167 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:14.308461  3428 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:14.320043  3428 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.316459  3450 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.321162  3450 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.322099  3450 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.323078  3372 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.324522  3450 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.309958  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:14.311013  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.313239  3480 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:14.338719  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122973us
I20260709 13:01:14.380328  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:14.380997  2294 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.332918  3167 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.391985  3220 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:01:14.407860  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:14.406222  3167 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.406917  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:14.398140  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.402204  3298 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.397410  3430 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.396481  3189 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:14.419621  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.428417  2294 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
W20260709 13:01:14.420715  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 171325us
W20260709 13:01:14.432762  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.434232  1630 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:01:14.435138  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.435864  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.443158  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.444109  3213 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.444937  1558 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:14.444692  3213 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.447276  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.421532  3333 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.447876  3298 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.448138  3167 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.448205  3333 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.450122  3298 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.450501  3333 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.452193  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.452787  1555 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:14.453595  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.454577  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.455190  1555 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:14.456346  1552 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:14.456768  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.457574  1552 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:14.459823  1525 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.460675  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:14.463300  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43820.5us
I20260709 13:01:14.451856  3189 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.466207  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.469684  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.458649  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.471534  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.472632  1560 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:14.474996  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.476119  1559 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:14.478762  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.479218  1556 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:14.479883  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.480893  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:14.481120  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.482988  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.458655  3452 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:14.485220  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:14.489959  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.491850  1705 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:14.492981  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.455524  3161 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.422019  3486 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:01:14.472823  3484 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:14.468153  3428 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.473412  3493 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:14.503511  1556 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:14.503841  1703 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:14.508512  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.437999  3430 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:14.450141  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc040 after lost signal to thread 434
I20260709 13:01:14.506031  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.509859  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:14.511169  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.501413  3477 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:14.500742  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.514994  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:14.520109  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.520576  1556 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:14.506057  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.522495  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.508181  3298 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.523921  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.525652  1704 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:14.526134  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:14.526714  3485 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.525738  3362 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.527921  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:14.530102  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60488us
I20260709 13:01:14.514024  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.534240  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.535166  3298 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.562261  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.562271  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.563055  3298 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:14.536151  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:14.536360  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.564453  1556 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:14.538203  3213 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.539121  3161 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:14.567407  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.563104  1701 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:14.570896  3372 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:14.571408  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.571494  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.519851  3338 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 13:01:14.540396  3500 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:01:14.541296  3507 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:14.547385  3189 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.557782  1704 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:14.554569  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:14.560665  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:14.569098  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:14.569762  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.572577  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.610423  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.611064  1700 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:14.612020  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.612329  3448 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:14.612437  1630 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:14.612632  1634 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:14.613376  3191 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.613575  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.613633  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.614826  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.615388  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.616145  1559 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:14.616827  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.617708  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.563560  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.618569  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:14.620143  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65654.8us
I20260709 13:01:14.621413  1551 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:14.622507  3430 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.574674  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.613910  3191 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:14.607964  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:14.626561  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51971.5us
I20260709 13:01:14.630951  1557 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:14.631879  3493 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:14.632297  3493 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:14.632706  3493 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.633734  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:14.634471  2291 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:14.636462  1554 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:14.636662  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.637221  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.637176  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:14.637794  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.637998  1553 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:14.638365  1558 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:14.638713  3372 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.639088  3372 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.639405  3372 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.612756  3448 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:14.629522  3404 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:14.554121  3362 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.637315  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.603870  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.652714  1560 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.640249  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.639727  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.653501  1553 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:14.645617  1706 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:14.651043  1631 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:14.661201  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.644773  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.645479  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.663730  1555 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:14.576669  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.631332  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.668503  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.669129  1558 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:14.673230  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:14.629880  3476 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.669086  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:14.549948  3477 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:14.675210  1635 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.680441  1632 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:14.672967  3213 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.698007  3213 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.699376  1707 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:14.684085  3452 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:01:14.681725  3191 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.640182  3448 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.572741  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.690277  1702 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:14.693013  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.662590  3362 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.572767  3480 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.695871  3333 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.697021  3508 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:14.697532  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.640766  3404 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.568192  3436 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:14.694972  3476 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:14.717674  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.710165  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 187478us
W20260709 13:01:14.573437  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080018eec0 after lost signal to thread 1492
I20260709 13:01:14.704816  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.707864  3430 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:14.721905  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.728976  1560 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:14.733489  3477 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.734983  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.713768  3213 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:14.749506  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.715647  3333 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:14.758622  3333 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.763751  3404 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.765058  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.765384  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:14.768720  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.769730  3237 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:01:14.773953  1458 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 3, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:14.775524  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.776734  3298 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:14.777124  3298 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:14.777519  3298 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.717676  3508 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:14.722196  3436 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:14.798076  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:14.804867  2263 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:14.806686  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 87717us
I20260709 13:01:14.754957  3398 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:14.797057  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003cac00 after lost signal to thread 1584
W20260709 13:01:14.828291  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 153796us
I20260709 13:01:14.821928  3333 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.756843  3430 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.829680  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.806690  3513 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.796878  3167 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:01:14.794514  3226 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:14.834154  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.834323  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.811395  3398 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.809958  3503 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:14.796734  3364 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:01:14.837217  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.828377  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.828586  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.844429  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:14.853026  1560 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.847486  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:14.855666  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.832926  3493 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.827627  3338 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:01:14.879455  3430 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.879823  3338 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:14.880234  3338 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.880288  3508 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.881249  3338 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.882500  3338 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.882843  3290 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.882898  3436 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.884025  3333 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.884773  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.885366  1559 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:14.886600  1559 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.839735  3476 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.889665  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:14.890393  1633 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:14.890655  1630 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:14.854899  3503 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.891742  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.892100  3398 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.894683  3398 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.842717  3513 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.868405  3167 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:14.868651  3486 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.896973  3486 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.897426  3513 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:14.892382  1631 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:14.892563  3503 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.899230  3486 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.900719  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:14.904362  1560 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:14.884542  3333 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.909068  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:14.901413  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:14.910780  3339 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:14.911175  3339 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:14.911993  3333 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:14.913247  1705 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.912532  3333 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:14.915226  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49747.8us
W20260709 13:01:14.863745  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080037e640 after lost signal to thread 1659
W20260709 13:01:14.916056  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003a85c0 after lost signal to thread 1660
W20260709 13:01:14.917088  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4c80 after lost signal to thread 1733
W20260709 13:01:14.917375  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4c20 after lost signal to thread 1734
I20260709 13:01:14.917966  3486 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:14.918553  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08000a9ea0 after lost signal to thread 2321
I20260709 13:01:14.920300  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:14.923704  1629 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:14.920744  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.940891  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:14.927698  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080021ada0 after lost signal to thread 3141
I20260709 13:01:14.917357  3448 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.954030  3448 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:14.912037  3339 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.955552  3339 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.968847  3362 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:14.970224  1700 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:01:14.972220  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52045.5us
I20260709 13:01:14.975189  1631 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:14.976495  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135043us
I20260709 13:01:14.982100  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.977350  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.971385  3290 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.992846  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.981086  3398 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.951470  3503 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:14.995118  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:14.995801  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:14.999533  2294 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:15.000545  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132103us
I20260709 13:01:14.982661  3522 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:14.990602  3226 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.015197  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.016252  3398 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.978852  3432 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:01:15.026103  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:15.013758  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:15.015882  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75125us
I20260709 13:01:15.012058  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.016700  3523 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:14.990741  3364 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:01:15.033829  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.016796  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:15.025669  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
W20260709 13:01:15.017068  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 169629us
I20260709 13:01:15.023747  3503 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.043824  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.038493  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.040614  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.043278  1704 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:15.014247  3161 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.014554  3522 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.009280  3386 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:15.009187  3315 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:14.996909  3290 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.015184  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.014731  3527 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.031867  3523 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:15.062422  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.048085  3333 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.049892  3226 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.050087  3398 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:15.054333  3503 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.062503  3436 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.061367  3513 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.066515  3537 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.065047  3523 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.053361  3386 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.072177  3226 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:15.086299  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45769us
I20260709 13:01:15.073700  3527 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.088238  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:15.056255  2288 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:15.055375  3372 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.054898  3315 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.080456  3339 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.108794  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:15.110013  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.097963  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.098712  3226 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:15.103152  3524 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:15.104032  3536 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.106081  3526 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:15.106691  3521 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 13:01:15.097527  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:15.145395  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48079.9us
I20260709 13:01:15.146170  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.147269  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:15.115453  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.135602  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77945.1us
I20260709 13:01:15.136921  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.100852  3428 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:15.094363  3362 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.150403  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.157560  3527 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.159860  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:15.156067  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 106995us
W20260709 13:01:15.164925  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 126526us
I20260709 13:01:15.172140  2294 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.172312  2292 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:15.173851  2289 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:15.178158  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.179553  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.099428  3333 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:01:15.154074  3537 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.143739  3477 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:15.154974  3428 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:15.153332  3535 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.205314  3535 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.205729  2292 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:15.204959  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.160681  3503 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.208936  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.209585  1708 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:15.210237  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.210789  2297 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.211019  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.212062  3540 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:15.213274  3539 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:01:15.181291  3527 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.222409  2297 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:15.207558  3535 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.198117  3537 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:15.218361  3428 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.148703  3536 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:15.181071  3398 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.249598  3522 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.195884  3298 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.197494  3526 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:15.278892  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.219645  3477 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.286811  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.259295  3537 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.265761  3542 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:15.295130  3540 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:01:15.295240  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.293152  3404 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.295091  3542 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.277504  3290 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.304494  2288 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.310593  1706 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:15.314718  3452 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.332268  3452 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.333516  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.334443  1627 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:01:15.353655  3549 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:15.331489  3315 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.348172  1708 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:15.358990  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.360421  3527 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.360452  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.361301  3523 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.361676  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.362818  3503 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.363189  3503 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.363497  3503 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.364435  3541 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.364813  3541 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.365118  3541 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.365787  3493 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.369951  1709 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:15.370344  1709 pending_rounds.cc:85] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 1
I20260709 13:01:15.370558  1709 pending_rounds.cc:107] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:15.370766  1709 raft_consensus.cc:2889] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.372300  3536 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.372414  1707 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:01:15.372926  1707 pending_rounds.cc:85] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 2
I20260709 13:01:15.373157  1707 pending_rounds.cc:107] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:01:15.373415  1707 raft_consensus.cc:2889] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.372344  3503 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.374949  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.376400  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.375478  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.375635  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.356853  1708 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:15.380290  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:15.373286  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:15.339202  3333 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.338371  3426 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:01:15.340310  3542 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.395735  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.378185  2296 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:15.346261  3372 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.363441  3523 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.405786  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.417393  1708 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:15.405385  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 239372us
I20260709 13:01:15.408238  2288 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:15.429955  1708 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.408867  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.411059  3422 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:15.432301  3422 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:15.433475  3422 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.435050  3422 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.413226  3386 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.389947  3333 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.414935  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.418437  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.419140  1555 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:15.425366  1700 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:15.426134  2295 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:15.426648  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.384375  3493 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.445601  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.449491  1703 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:15.369954  1701 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:15.414041  3546 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.344687  3404 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.385578  3484 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.450685  3545 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.457427  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.400158  3325 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.468479  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.470115  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.461481  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.456369  1708 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:15.450898  3534 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 13:01:15.447959  3333 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.445218  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.456542  3311 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.442538  3338 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:01:15.468297  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.474922  3493 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.503504  3553 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:01:15.504428  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.505235  3523 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:15.506682  3526 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.511940  3551 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.521806  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.519641  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.467125  3430 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:01:15.436832  3339 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:01:15.472579  3522 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.474291  2291 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.542138  3311 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.476408  3538 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:15.555589  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87372us
W20260709 13:01:15.556137  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51789.2us
I20260709 13:01:15.559602  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.575312  2294 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:15.523684  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.474313  3325 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:15.559324  2292 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:15.561071  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:15.541627  3298 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.561004  3372 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.580318  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.556921  3523 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:15.536442  3508 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:15.586939  3553 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.589067  3553 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.589399  3553 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.590408  3325 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.591197  1705 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:15.592105  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.593255  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.594300  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.595862  3452 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.596576  2291 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:15.597903  2291 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.597941  3553 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.599037  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.599653  1701 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:15.600610  3553 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.600780  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.601615  1706 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:15.602474  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.569993  3526 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.463339  3546 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.571882  3486 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.571230  3513 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:15.614177  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:15.612744  3436 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.614811  1706 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.615403  3436 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.615932  3339 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:15.616946  3339 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.617276  3339 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:15.617439  3436 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.618090  3339 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.619392  3339 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.621379  3311 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.622963  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.615327  3560 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:15.625994  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.606166  3523 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.573987  3522 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.632968  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:15.633366  3525 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.812s	user 0.001s	sys 0.003s
I20260709 13:01:15.633597  3522 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.633672  1628 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
W20260709 13:01:15.635419  3516 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.959s	user 0.002s	sys 0.002s
I20260709 13:01:15.588351  3540 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.636179  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.650619  1629 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:15.637432  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.650830  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:15.651127  2297 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.639128  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:15.640285  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.653393  1552 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:15.654124  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.642489  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.643461  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.644425  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.645321  3560 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:15.647434  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.656558  1557 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:15.648162  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:15.648413  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.654553  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.657734  1554 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:15.658133  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.611629  3513 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:15.663560  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.664219  1560 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:15.624490  1558 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:15.624120  3503 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.663571  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.669900  3577 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.670362  3577 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:15.670629  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.670792  3577 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.672222  3364 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.672626  3364 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:15.681099  3515 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.003s	user 0.002s	sys 0.000s
I20260709 13:01:15.663163  3484 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:01:15.682981  3484 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:15.683135  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:15.683389  3484 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:15.683840  3484 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.684981  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.685832  2288 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:15.687906  3550 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
W20260709 13:01:15.688395  3558 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:01:15.689888  3480 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.690143  3493 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.683702  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.705586  1629 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.707140  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.708038  1634 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:15.711584  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:15.721001  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:15.722227  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.699294  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.612179  3226 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.720566  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.726457  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.727556  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2013) took 1311 ms. Trace:
I20260709 13:01:15.723047  1706 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:15.727909  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.728909  1628 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:15.727936  1626 rpcz_store.cc:276] 0709 13:01:14.415956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.416105 (+   149us) service_pool.cc:224] Handling call
0709 13:01:14.433281 (+ 17176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.433623 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.433651 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:14.443342 (+  9691us) log.cc:844] Serialized 26 byte log entry
0709 13:01:14.674134 (+230792us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:14.674174 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.674233 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:15.680488 (+1006255us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:15.696537 (+ 16049us) raft_consensus.cc:1685] finished
0709 13:01:15.696556 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.727502 (+ 30946us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3897,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":25451,"spinlock_wait_cycles":83859200,"thread_start_us":183394,"threads_started":1,"wal-append.queue_time_us":202120}
I20260709 13:01:15.732270  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:15.733284  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.732141  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.735018  1629 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:15.736193  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.740689  3538 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:15.742017  3538 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:15.746516  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 75966.2us
I20260709 13:01:15.747509  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.748569  3486 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:15.752633  3364 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.754115  3364 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.757591  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.757532  3584 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.777509  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.779248  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.663954  3452 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:01:15.734088  3503 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:15.784062  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:15.781414  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:15.788174  2296 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.790484  2297 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:15.765688  3485 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.666152  3540 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.792073  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:15.764430  3298 raft_consensus.cc:2764] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:15.778674  3338 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:15.805222  3568 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:15.805759  3568 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.726388  3428 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.804203  3485 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.811136  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.825052  2296 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:15.826522  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.827739  3522 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.828291  3522 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.830379  3428 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.805214  3338 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.831315  3338 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.837060  3485 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:15.838514  3485 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.842603  3220 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.843627  3536 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:15.844306  3536 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:15.793303  3404 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.850976  3536 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.830384  3522 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.854805  3546 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.856583  3477 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.858646  3325 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.859138  3325 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:15.871351  3527 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 13:01:15.799651  3436 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.878515  2290 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:15.880049  3398 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.880995  3325 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.899714  3325 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.895471  3338 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.907215  3338 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.908788  2288 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:15.912963  3485 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.917573  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.918144  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:15.882118  3226 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:15.827327  2296 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:15.893464  1552 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:15.785899  3545 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:15.828223  3530 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.898305  3398 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:15.918256  1705 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:15.818325  3568 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:15.919567  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.891996  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.967842  1707 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:15.926115  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.940104  1632 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:15.823659  3167 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:15.801462  3584 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:01:15.972812  3575 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.890311  3333 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.925403  3384 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:15.867571  3484 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.802414  3513 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.919938  3527 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.951880  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:15.954962  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.948845  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:15.958151  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:15.954018  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:15.948632  3191 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:15.957043  3529 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:15.961302  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.047209  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
W20260709 13:01:15.968354  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76449.2us
I20260709 13:01:15.972251  3167 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:15.978852  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 324337us
I20260709 13:01:15.821702  3541 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.962874  3485 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.052156  3167 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.052663  3485 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.054545  3485 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.055153  2294 raft_consensus.cc:2368] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
W20260709 13:01:15.989748  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 232217us
I20260709 13:01:15.991132  1635 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:01:16.056561  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:16.056998  3422 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 13:01:16.058112  3541 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.058527  3333 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.060575  3540 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.046260  3290 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:16.062397  3290 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.062776  3290 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.068621  3398 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:15.816656  3560 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:16.074378  3542 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.000s
I20260709 13:01:16.074810  3542 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.075935  1627 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:16.077522  3493 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:16.078034  3493 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:16.079422  3290 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.080822  3577 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.083174  3577 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.084596  3523 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.086961  3523 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.088570  3290 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:15.965161  3537 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.004s
I20260709 13:01:16.090691  3537 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.092679  3537 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:16.033664  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84922.7us
I20260709 13:01:15.923223  3436 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.051059  1701 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:16.087625  2291 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:16.094893  3436 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.026680  1709 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:15.971576  3226 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:15.976261  3568 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.048534  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:15.969084  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.096633  3568 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:15.979821  3384 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.097513  3384 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.098754  3568 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:16.087452  3513 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.103729  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.104836  3430 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:16.105459  3430 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.105844  3430 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.107821  3436 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.108799  3436 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:16.109141  3436 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:16.109958  3436 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.089154  3484 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.111136  3436 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.078998  3493 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.112311  3493 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.107376  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.114569  3311 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:16.114712  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.115087  3311 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.115526  3191 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:16.115967  3191 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.116382  3191 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:16.115548  3311 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.122241  3448 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.122788  3448 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.125154  3448 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.134013  3311 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.136561  3311 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.140596  3448 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.162395  3448 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.165627  3448 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.142285  3191 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:16.166450  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 115877us
I20260709 13:01:16.168296  3191 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.167701  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.168560  3422 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.171975  3452 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:16.001009  3538 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.147145  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:16.168345  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 111839us
I20260709 13:01:16.173658  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.173456  3538 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.146194  3542 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.171231  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.176137  3542 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:16.176545  3542 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.176412  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:16.177490  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:16.177995  1627 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:16.178180  3568 raft_consensus.cc:2764] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:01:16.178884  3540 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:16.179165  3583 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:16.179484  3540 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:16.179611  3583 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:16.180073  3583 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.180604  3540 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.184016  3542 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.184056  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:16.181006  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:16.214951  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 476806us
W20260709 13:01:16.213774  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 431794us
I20260709 13:01:16.197299  3538 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.236897  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:16.127092  3503 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:16.259509  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78582.5us
W20260709 13:01:16.260195  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 211759us
I20260709 13:01:16.261070  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:16.263094  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.005s	sys 0.029s
W20260709 13:01:16.263466  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.005s	sys 0.029s
W20260709 13:01:16.263775  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 535603us
I20260709 13:01:16.264421  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.265228  1703 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.266778  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.268340  1707 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:16.268783  1635 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:16.270092  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.270756  1707 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.274700  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.275493  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:16.275782  1700 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:16.276975  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.277391  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.277560  1557 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:16.284080  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:16.264485  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.139460  3384 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:16.288240  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.289016  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.289964  1559 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:16.226276  3311 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:16.300266  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:16.303508  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63779.8us
I20260709 13:01:16.300434  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.304304  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:16.307108  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:16.302107  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.302565  1630 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:16.304301  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.311036  1559 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.247617  3448 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:16.312448  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.317415  1700 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.312204  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.318920  3576 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:16.320384  3422 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:16.320604  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.316481  3484 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:16.321513  3484 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.269279  3325 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:16.321873  3484 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.322027  3325 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.311805  3311 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.306774  3476 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:16.323843  3503 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:16.315678  3591 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:16.315055  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.316020  1705 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:01:16.325198  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.325742  1555 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:16.301366  1556 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "334958c5aee349629d63e898f13ba95b"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:01:16.315898  3571 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:16.326750  1556 raft_consensus.cc:606] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:16.327149  1556 raft_consensus.cc:614] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:16.323939  3325 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.268440  3485 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.273077  3575 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:16.300671  3213 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:16.332465  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:16.325235  1705 pending_rounds.cc:85] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 3
I20260709 13:01:16.363915  1705 pending_rounds.cc:107] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:01:16.364257  1705 raft_consensus.cc:2889] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.325809  1706 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:16.333323  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.349539  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:16.369079  1554 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.288301  3290 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:16.358752  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 441220us
I20260709 13:01:16.371615  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.381796  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.382944  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.308100  3450 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:16.378229  1706 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:16.384600  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.317298  3448 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:16.385426  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58223.4us
W20260709 13:01:16.404721  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100492us
I20260709 13:01:16.303097  3480 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:01:16.386937  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.333294  3571 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.405856  1709 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.401615  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.324513  3503 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.407059  3589 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.410794  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.368216  3452 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.411489  1552 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:16.409029  1554 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:16.412806  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44829.7us
I20260709 13:01:16.371486  3189 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.419183  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.424772  1707 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.422358  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:16.408339  3448 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.413484  1700 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:16.435292  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.440970  2296 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.443491  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:16.444890  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:16.445752  2296 raft_consensus.cc:2368] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
W20260709 13:01:16.437801  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.019s	sys 0.039s
W20260709 13:01:16.447703  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.019s	sys 0.039s
W20260709 13:01:16.448109  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 656095us
I20260709 13:01:16.408943  3571 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.379719  3485 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.410907  3422 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:16.413141  3476 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:16.472118  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.472783  1556 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:16.473120  1707 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:16.411280  3503 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.476286  1558 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:16.478533  2297 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.022s	sys 0.013s
W20260709 13:01:16.478956  2297 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.022s	sys 0.013s
I20260709 13:01:16.406158  3450 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.412818  3364 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:16.503106  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.506951  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.508057  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.434597  3333 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:01:16.516611  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:16.521639  2294 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:16.448798  3191 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:01:16.419739  3452 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.496800  3384 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.527135  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.425158  3576 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:01:16.465495  3560 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:01:16.434772  3189 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.507488  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:16.513602  3588 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:16.480214  3422 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.528040  3550 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:16.538869  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132039us
I20260709 13:01:16.538705  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.516248  3493 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:01:16.539199  3325 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:16.521677  3364 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.508958  3583 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.550733  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.558676  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:16.560158  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:16.557045  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 104569us
I20260709 13:01:16.548660  3584 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:16.440424  3589 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.542652  3430 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.573463  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:16.588546  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.589668  1709 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:16.588222  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.590464  1558 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.546275  3339 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:01:16.604578  1558 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.606398  1558 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:16.610267  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.563736  3384 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:16.569291  3436 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:16.577755  3432 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:16.566810  3584 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:16.551283  3550 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.567003  3325 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:01:16.548660  3476 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.551400  3588 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.633908  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.660487  1709 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:16.644507  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:16.635474  3384 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.619709  3191 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:01:16.665802  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.669173  1709 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:16.636540  3432 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:16.656219  3594 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:01:16.672101  3325 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:01:16.672506  3601 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:01:16.672791  3587 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 13:01:16.673493  3602 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:16.662376  3526 raft_consensus.cc:2764] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:01:16.667680  3384 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.669569  3542 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.688381  1703 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:01:16.674306  3485 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.675450  3432 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:16.670727  3450 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.677326  3325 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:16.675580  3584 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.696641  3325 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.697765  3325 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:16.700227  3364 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:16.701632  1554 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:16.701807  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.703179  1704 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:16.690976  3609 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.704370  3609 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.705888  3503 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.714538  3583 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.714775  3477 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.715373  3477 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.690220  3537 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:16.714618  1708 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:16.740464  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.716786  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.743268  1704 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:16.749081  1705 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:16.749603  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:16.758172  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 407) took 1117 ms. Trace:
I20260709 13:01:16.730023  3430 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.798432  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:16.790982  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 260767us
I20260709 13:01:16.729441  3311 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.731403  3588 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.749071  3189 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.721858  3589 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:16.786551  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 373100us
I20260709 13:01:16.740453  3477 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.749203  3550 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.804874  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.808120  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:16.808056  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:16.825929  1552 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.796584  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:16.826488  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 264632us
I20260709 13:01:16.808467  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:16.750902  3537 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:16.796845  3325 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.797716  3542 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.717068  3448 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.801008  3583 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.788666  3484 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.860781  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:16.801213  3485 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:16.866609  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:16.933401  1707 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.806994  3503 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.806532  3609 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.943208  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:16.868606  1627 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:16.805887  3311 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:16.924140  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:16.829452  3226 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.945075  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:16.857707  3430 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:16.959906  3226 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.960775  3448 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:16.863978  3523 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.973305  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:16.945320  3538 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:16.976491  3538 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.976869  3538 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:16.942566  3189 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.977921  3189 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.970346  3311 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:16.973104  3430 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:16.982421  3430 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:16.857376  3191 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.106s
W20260709 13:01:17.016995  3571 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.003s
I20260709 13:01:16.971024  3609 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:16.869051  3290 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:16.915263  3585 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:01:17.042681  1552 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.977044  3484 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:16.870489  3325 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:16.942178  3583 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.028645  3432 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:17.014482  3596 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.633s	user 0.002s	sys 0.000s
I20260709 13:01:16.954388  3485 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.198487  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:16.975386  3523 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.017467  3571 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.210634  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:17.205904  3538 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.211807  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.213898  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:17.196898  3364 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.217109  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.220901  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.219568  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:17.221567  1707 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.222270  3513 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.218819  3364 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.215636  3608 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:01:17.222791  3513 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.219031  3607 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:17.223802  3607 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.222437  3617 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.224172  3364 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.221655  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.206537  3619 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.059310  3384 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.226377  1635 raft_consensus.cc:2368] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:17.231017  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.231720  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:17.233083  1708 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.223587  3538 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.026090  3537 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.255445  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.212939  3609 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.241873  1708 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:17.245966  1556 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.266198  1553 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.253834  1560 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:17.260478  1704 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.261282  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.226931  3617 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.232388  3311 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.254485  3436 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:17.253989  2291 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:17.236056  3613 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.290894  3613 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.294407  3613 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.296382  3311 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.267493  3189 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:17.272609  3609 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.274302  3538 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:17.310590  1704 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:17.315065  1707 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:17.272464  3537 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.274781  3226 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.324958  3325 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:17.325829  3615 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:01:17.326454  3538 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.235847  3619 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.320996  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:16.791594  2297 rpcz_store.cc:276] 0709 13:01:15.641212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.798171 (+156959us) service_pool.cc:224] Handling call
0709 13:01:15.875489 (+ 77318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.875740 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:16.498922 (+623182us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.757721 (+258799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:16.757796 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:16.757854 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.757942 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.758094 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224950,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":835,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":1660,"thread_start_us":815166,"threads_started":2}
I20260709 13:01:17.344297  1704 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:17.347774  1553 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:17.331161  3436 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.334586  3613 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.349926  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.352919  1707 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:17.364563  2295 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:17.333178  3311 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.377848  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:17.378830  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:17.349824  3624 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.352970  3583 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.349391  3617 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.400815  3364 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:17.401358  3364 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.403412  3364 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.404088  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1088) took 1141 ms. Trace:
I20260709 13:01:17.404325  3384 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.404480  1701 rpcz_store.cc:276] 0709 13:01:16.262929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.269292 (+  6363us) service_pool.cc:224] Handling call
0709 13:01:16.318800 (+ 49508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.439591 (+120791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.447205 (+  7614us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.496719 (+ 49514us) log.cc:844] Serialized 26 byte log entry
0709 13:01:16.805367 (+308648us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:16.825334 (+ 19967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.842434 (+ 17100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.387956 (+545522us) raft_consensus.cc:1685] finished
0709 13:01:17.387985 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.404020 (+ 16035us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109540,"raft.queue_time_us":136381,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12639,"thread_start_us":243958,"threads_started":1,"wal-append.queue_time_us":661923}
I20260709 13:01:17.406240  3583 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.370635  3613 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.376374  3290 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:17.411238  3290 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.413512  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.414003  3513 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.414156  3290 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.415851  3484 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:17.416226  3484 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:17.416868  3619 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.417021  3484 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.417976  3484 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:17.391270  3623 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:17.328639  3607 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.388193  3589 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:17.388550  3550 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.396931  3432 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.397142  3625 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:17.397552  3551 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:17.397621  3523 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.397850  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1111) took 1089 ms. Trace:
I20260709 13:01:17.398334  3436 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.421954  3624 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:17.418255  3538 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.386479  1631 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:17.419577  3311 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:17.423086  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 201494us
I20260709 13:01:17.406997  3583 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.452531  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.453629  1554 raft_consensus.cc:2410] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:17.433099  1632 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:01:17.422389  2290 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.869s	user 0.006s	sys 0.005s
I20260709 13:01:17.437114  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:17.454788  2290 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:17.455412  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 422) took 1033 ms. Trace:
I20260709 13:01:17.455754  2290 rpcz_store.cc:276] 0709 13:01:16.421494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.424484 (+  2990us) service_pool.cc:224] Handling call
0709 13:01:17.455344 (+1030860us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:17.456946  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.457708  2290 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:17.458662  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.459280  2290 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:17.385167  3585 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.384979  3325 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.460100  3325 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.460062  3585 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.462287  3325 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.438318  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:17.462812  3585 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.440006  2295 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:17.464041  1630 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:17.441532  1560 rpcz_store.cc:276] 0709 13:01:16.308645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.392702 (+ 84057us) service_pool.cc:224] Handling call
0709 13:01:17.397631 (+1004929us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:17.397753 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1830802688}
I20260709 13:01:17.439445  2291 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:17.432822  3485 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.445911  3634 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.429493  3339 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.449501  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.450907  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:17.453778  1631 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:17.463165  1628 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:17.469449  1559 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:17.464239  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:17.470392  2295 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:17.399581  3612 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:01:17.473156  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:17.473721  1558 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:17.443518  3568 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.444033  3589 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:17.448462  3550 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:17.451737  3432 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.484421  3432 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.487389  1524 leader_election.cc:400] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:17.487948  3432 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.441179  3625 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.489174  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.477172  3311 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.489876  1556 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:17.490363  3625 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.490425  3311 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.490983  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.492635  3311 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.465517  3613 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.451737  3619 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:17.492769  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.491689  1556 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:17.520629  1559 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:17.521453  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.496057  1628 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:17.522081  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.522502  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.496840  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.498822  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.474852  3634 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.476572  3339 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.509109  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1893) took 1039 ms. Trace:
I20260709 13:01:17.511245  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.512188  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.513388  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.493428  3568 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.542790  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.518342  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.544384  1632 raft_consensus.cc:2410] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:17.519646  1633 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.496243  1558 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:17.495853  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.523399  1553 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:17.523484  1630 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:17.545547  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.524528  2295 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:17.525117  3624 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.533360  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.546363  1560 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:17.534152  1700 rpcz_store.cc:276] 0709 13:01:16.470238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.503664 (+ 33426us) service_pool.cc:224] Handling call
0709 13:01:16.504016 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.524700 (+ 20684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.524745 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.525593 (+   848us) log.cc:844] Serialized 26 byte log entry
0709 13:01:16.788074 (+262481us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:16.788129 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.788221 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.508831 (+720610us) raft_consensus.cc:1685] finished
0709 13:01:17.508861 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.509021 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20290,"raft.queue_time_us":20808,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7391,"spinlock_wait_cycles":665344,"thread_start_us":262101,"threads_started":1,"wal-append.queue_time_us":521384}
I20260709 13:01:17.493182  3609 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.536206  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.541697  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.527405  1556 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:17.493615  3589 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.533387  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:17.552336  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.552907  1558 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:17.553574  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.553668  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:17.536934  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:17.555748  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 59985.4us
I20260709 13:01:17.556022  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.556304  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.550350  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.557138  1556 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 1.
I20260709 13:01:17.537639  3619 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.551669  3430 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.570817  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.572511  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.556612  1524 leader_election.cc:400] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:17.573503  1524 leader_election.cc:403] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:17.555737  1559 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:17.579577  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe"
dest_uuid: "444b494bb53842b59408a7230884398a"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:35924
I20260709 13:01:17.580297  1559 raft_consensus.cc:606] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:17.580737  1559 raft_consensus.cc:614] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:17.558095  1556 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:17.541476  3633 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.558626  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.559237  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.559392  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.526453  3550 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.535756  3625 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:17.541682  3364 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:01:17.561611  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.589883  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.591099  1708 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:17.562683  3568 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:17.548794  2291 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:17.550160  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.592803  1633 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:17.567991  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.568661  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:17.594173  1707 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:17.594816  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.595408  2297 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:17.583216  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.596118  1700 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:17.596163  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.597328  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.597805  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:17.560328  3634 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.584723  1558 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 1.
I20260709 13:01:17.623828  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.567230  3617 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.555050  3486 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:17.592567  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.591339  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.570482  3167 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:17.640897  3636 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:01:17.641166  3167 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:17.642046  3167 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:17.591637  3580 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.241s	user 0.003s	sys 0.000s
I20260709 13:01:17.635700  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.584916  3585 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:17.589438  3430 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.584379  3513 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:17.637290  1707 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:17.586992  3480 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:17.584486  3226 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:01:17.640134  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.599140  2290 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:17.633261  3436 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:17.584232  3609 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
W20260709 13:01:17.653013  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68409.7us
I20260709 13:01:17.599519  3633 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.592530  3625 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.706386  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.657332  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.660109  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.716193  1628 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:17.717226  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:17.588814  2288 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:17.648828  2292 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:17.742048  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 718) took 1306 ms. Trace:
I20260709 13:01:17.742636  2288 rpcz_store.cc:276] 0709 13:01:16.435738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.439974 (+  4236us) service_pool.cc:224] Handling call
0709 13:01:17.403473 (+963499us) spinlock_profiling.cc:243] Waited 8.12 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.511089 (+107616us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfe9c 00007f70cb8bfa5b 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.741959 (+230870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62756480}
W20260709 13:01:17.746311  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 180008us
W20260709 13:01:17.687436  3598 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.143s	user 0.002s	sys 0.002s
I20260709 13:01:17.716473  3290 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:17.750455  3290 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.751521  3290 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.719331  3436 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.759629  3589 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.686599  3619 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.731382  3339 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:17.761348  3436 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.761461  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.649506  3617 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.762472  1630 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:17.766024  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.766194  3617 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.766686  1553 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:17.770382  3571 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.772508  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.745078  3430 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.784369  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1101) took 1507 ms. Trace:
I20260709 13:01:17.785071  1557 rpcz_store.cc:276] 0709 13:01:16.277353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.301847 (+ 24494us) service_pool.cc:224] Handling call
0709 13:01:17.784090 (+1482243us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54001ee1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:17.784281 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500138240}
I20260709 13:01:17.792521  3625 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:17.793304  3616 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.845s	user 0.002s	sys 0.000s
I20260709 13:01:17.794173  3583 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.801446  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.804489  3452 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.815483  3584 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.834353  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 719) took 1398 ms. Trace:
I20260709 13:01:17.787006  3633 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.834826  2292 rpcz_store.cc:276] 0709 13:01:16.436193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.452237 (+ 16044us) service_pool.cc:224] Handling call
0709 13:01:17.391333 (+939096us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.513236 (+121903us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfe9c 00007f70cb8bfa5b 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.834068 (+320832us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54005e1854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:17.834266 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167578112}
I20260709 13:01:17.820762  3191 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.838900  3191 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.843838  3226 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:17.844405  3226 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:17.845449  3226 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.846992  3484 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.848548  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 131365us
I20260709 13:01:17.848835  3338 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:17.849463  3338 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.849915  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:17.851099  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:17.847689  3484 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.852270  3613 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:17.852969  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104897us
I20260709 13:01:17.853691  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.854489  3364 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:17.854333  3513 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:17.854986  3364 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:17.855422  3364 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:17.849891  3338 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:17.847021  3226 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:17.860337  3589 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:17.855497  3513 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:17.861549  3513 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:17.864491  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.868311  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:17.865754  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.869371  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.870324  3625 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:17.870837  3625 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:17.852344  3526 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:17.871615  3526 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.871014  3513 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:17.872088  3526 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:17.871688  3625 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.872991  3513 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:17.866392  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.867061  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.866873  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:17.876158  1707 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:17.867600  3646 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.876959  3646 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:17.880071  3553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.010s	sys 0.017s
W20260709 13:01:17.880447  3553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.010s	sys 0.017s
W20260709 13:01:17.883035  3606 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.035s	user 0.000s	sys 0.002s
I20260709 13:01:17.883965  3553 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:17.884404  3553 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:17.885190  3553 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:17.886271  3614 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.909s	user 0.001s	sys 0.001s
I20260709 13:01:17.886417  3553 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.890163  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1102) took 1612 ms. Trace:
I20260709 13:01:17.890620  1555 rpcz_store.cc:276] 0709 13:01:16.277591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.438745 (+161154us) service_pool.cc:224] Handling call
0709 13:01:16.439129 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.517346 (+ 78217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.517389 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.523716 (+  6327us) log.cc:844] Serialized 26 byte log entry
0709 13:01:16.803149 (+279433us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:16.856668 (+ 53519us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.856887 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.404128 (+547241us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b540036c4d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:17.404171 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.885690 (+481519us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:17.889861 (+  4171us) raft_consensus.cc:1685] finished
0709 13:01:17.889888 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.890056 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53908,"prepare.run_cpu_time_us":7307,"prepare.run_wall_time_us":71527,"raft.queue_time_us":256,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":60354,"spinlock_wait_cycles":185343616,"thread_start_us":257565,"threads_started":1,"wal-append.queue_time_us":439151}
I20260709 13:01:17.894258  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.894730  3422 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.852968  3613 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.896806  3613 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:17.897446  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.898156  2289 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:17.898633  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 423) took 1476 ms. Trace:
I20260709 13:01:17.898952  2289 rpcz_store.cc:276] 0709 13:01:16.421747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.424648 (+  2901us) service_pool.cc:224] Handling call
0709 13:01:17.391859 (+967211us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.897304 (+505445us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfe9c 00007f70cb8bfa5b 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:17.898571 (+  1267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112288384}
I20260709 13:01:17.900213  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.875605  1700 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:17.903374  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.904812  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:17.905512  3537 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:17.905879  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.924041  1632 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:17.909224  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:17.909415  3527 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.910090  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.928683  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.910924  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:17.912833  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.935403  1551 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:17.937309  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.938866  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.940068  3527 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.913064  1700 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:17.941787  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.913719  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:17.895105  2294 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:17.942981  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:17.929857  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76233.1us
I20260709 13:01:17.942981  1557 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:17.944664  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.939543  1632 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:17.945156  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1834) took 1711 ms. Trace:
I20260709 13:01:17.864575  3191 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.930644  3568 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:17.931386  2288 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:17.931975  3537 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:17.945545  1627 rpcz_store.cc:276] 0709 13:01:16.234160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.234304 (+   144us) service_pool.cc:224] Handling call
0709 13:01:16.234650 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.944079 (+709429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.944912 (+1000833us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b540019c654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:17.945098 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1447784832}
W20260709 13:01:17.955793  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.875594  3484 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:17.972473  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.973475  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.973794  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.974479  1627 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:17.974989  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:17.976934  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.930083  3631 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:17.978941  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:17.981319  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.879045  3646 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:17.984794  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:17.989180  1700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41b83284fac147339ac4124c2eedee11"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:17.989744  1700 raft_consensus.cc:606] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:01:17.990063  1700 raft_consensus.cc:614] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:17.962553  3568 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.997602  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:17.998236  3632 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:17.972929  3550 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:18.004432  1700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:18.014863  1700 raft_consensus.cc:606] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.015334  1700 raft_consensus.cc:614] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:18.002727  3484 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:18.005779  3568 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:18.002612  3617 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 13:01:18.039127  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 145919us
W20260709 13:01:18.041240  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.043546  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 185774us
I20260709 13:01:18.045747  1634 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:18.046581  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:18.049744  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:18.050555  1554 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:18.012995  3364 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:18.060881  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 137698us
I20260709 13:01:18.024099  3631 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:18.027390  3432 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:18.065848  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62536.3us
I20260709 13:01:18.068367  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.024976  3484 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.070010  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:18.074044  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:18.076409  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:18.078976  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "8b2c3934407d422495728f84374c6b53"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:18.079588  1708 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Received request to transfer leadership to TS 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.077183  3553 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:18.080722  3372 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:18.081398  3372 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:18.082263  3372 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.080380  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:18.093199  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 247) took 1616 ms. Trace:
I20260709 13:01:18.080788  3553 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.037477  3523 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:01:18.093711  1703 rpcz_store.cc:276] 0709 13:01:16.476946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.671283 (+194337us) service_pool.cc:224] Handling call
0709 13:01:16.671560 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.690011 (+ 18451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.092920 (+1402909us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b54003c1254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:18.093123 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38161792}
I20260709 13:01:18.102324  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:18.068894  3432 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:18.104815  3633 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:18.095722  1601 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.109282  1700 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:18.105479   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:18.113559  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.104104  3553 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:18.127504  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.077596  3191 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:18.113986  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50492.7us
I20260709 13:01:18.057458  3613 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.120416  3364 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.120994  3537 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.116025  3527 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:18.107717  3611 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.131s	user 0.000s	sys 0.002s
I20260709 13:01:18.099272  3632 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:18.107575  3654 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:18.110103  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:18.105991  3653 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:18.129685  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:18.131062  3625 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:18.137228  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.125120  3634 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:18.133618  3578 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.153752  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.144637  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:18.125257  3633 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:18.115048  3432 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:18.132465  3311 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:01:18.165201  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:18.148947  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.171535  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.178169  2294 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:18.181063  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:18.182176  3526 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.160555  3633 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:18.150028  3617 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:01:18.164098  3625 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:18.166643  3661 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.214892  1560 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:18.221036  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:18.158835  3578 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:18.200084  3650 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.002s
I20260709 13:01:18.226776  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:18.230952  1553 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:18.202615  3537 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.199400  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.192981  3668 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.202203  3634 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:18.221594  3523 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:18.238950  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.244424  2295 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:18.239282  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67826.8us
I20260709 13:01:18.245319  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:18.193485  3364 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:18.249989  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.315021  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 252) took 1108 ms. Trace:
I20260709 13:01:18.235601  3537 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.431564  3537 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:18.432016  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:18.238521  3667 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:01:18.345191  1553 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:18.241359  2292 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:18.246172  3527 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.316697  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:18.248873  3655 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:01:18.239549  3668 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.243448  3632 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:18.369801  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 119885us
I20260709 13:01:18.224738  3661 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.468420  3613 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.470323  3661 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:18.471050  3661 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.369266  3167 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.472131  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.472051  3167 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.473047  2289 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:18.377489  3426 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:18.462011  3668 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:18.316215  3663 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.426697  1635 rpcz_store.cc:276] 0709 13:01:17.206783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.206999 (+   216us) service_pool.cc:224] Handling call
0709 13:01:18.314764 (+1107765us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b54000dd9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:18.314930 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9969920}
I20260709 13:01:18.449831  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.479722  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:18.340698  3339 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.241859  3666 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.134s
I20260709 13:01:18.317477  3364 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.340718  3526 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:18.378341  3658 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:01:18.494589  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.369760  3325 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.495919  3325 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.462589  2291 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
W20260709 13:01:18.467020  3671 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:01:18.511371  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:18.511973  1635 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:18.493570  3663 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.512898  3663 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:18.467118  3632 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.527668  1635 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:18.529212  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.455579  2288 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
W20260709 13:01:18.495932  3338 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.007s
I20260709 13:01:18.503921  3339 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.459064  3527 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.471683  3661 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:18.624402  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.594908  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 148785us
I20260709 13:01:18.653932  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.587700  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.527647  3669 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:18.723837  1629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.007s
W20260709 13:01:18.726678  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.731343  1629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.005s	sys 0.007s
I20260709 13:01:18.725433  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
W20260709 13:01:18.655370  3568 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.000s
W20260709 13:01:18.540864  3662 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
W20260709 13:01:18.629242  3658 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:01:18.664726  3553 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.526s	user 0.002s	sys 0.003s
I20260709 13:01:18.744709  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.568567  3613 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.740769  2295 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:18.687258  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.757522  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:18.661223  3290 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.907s	user 0.007s	sys 0.002s
I20260709 13:01:18.626541  3668 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.684866  3666 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.711925  3493 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:18.751749  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.725000  3646 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:18.655730  2289 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:18.716312  3167 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.721625  3325 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.686918  3364 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.756014  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.783097  2297 raft_consensus.cc:2376] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5. Re-sending same reply.
W20260709 13:01:18.778956  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 116420us
I20260709 13:01:18.688115  3633 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.741829  3669 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.787586  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.793197  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.594493  3338 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.686401  3537 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.796955  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.799170  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.799749  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.760944  3613 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.802016  3339 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.802716  3527 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:18.803020  2292 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:18.809324  3669 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.803645  3668 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.804010  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.810278  3613 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.810516  1635 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:18.811494  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.813182  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.754262  3553 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.778108  3646 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.808229  3364 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:18.804972  3663 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.816102  3668 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:18.816171  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.829559  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.830930  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.751863  3568 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.779465  3493 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.821408  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.824405  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:18.843279  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:18.843827  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 398165us
I20260709 13:01:18.769253  3290 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.827272  3527 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:18.827867  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.794296  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1106) took 1241 ms. Trace:
I20260709 13:01:18.836463  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.840315  1635 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:18.774547  3666 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.820528  3167 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:18.852686  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:18.849431  3476 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:18.856400  2289 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:18.854312  1701 rpcz_store.cc:276] 0709 13:01:17.552750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.562243 (+  9493us) service_pool.cc:224] Handling call
0709 13:01:17.562523 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.588690 (+ 26167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.592040 (+  3350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.594173 (+  2133us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:17.594227 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.780552 (+1186325us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:18.782814 (+  2262us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.794229 (+ 11415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6816128}
I20260709 13:01:18.854007  3339 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.830168  3550 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:01:18.860345  1632 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:18.860661  3668 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:18.849628  3526 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.874868  3675 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:01:18.852502  3529 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:18.850831  3325 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.876183  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:18.877948  3527 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:18.842720  3338 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:18.877717  3526 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.874678  2290 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:01:18.952831  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71159.1us
I20260709 13:01:18.953686  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:18.879045  3476 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.955014  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.877206  3678 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:18.953745  3526 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:18.877769  3661 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:18.877229  3537 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:18.962244  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:18.969017  1703 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:18.969110  1632 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:01:18.963428  1707 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:18.963414  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:18.882089  3527 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:18.875969  3633 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:18.876179  3339 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.964516  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:18.953699  3550 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:18.970777  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:18.971599  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:18.973753  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1843) took 1586 ms. Trace:
I20260709 13:01:18.974105  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.980026  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:18.883523  3325 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.960376  3678 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:18.875945  3167 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:18.982115  3339 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:18.983332  3657 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:18.987720  3338 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.988500  3527 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:18.989450  3550 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:18.974365  3529 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.990186  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:18.991925  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:18.993050  3680 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:01:18.996654  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.998003  3583 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.966005  3691 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:19.016345  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.005999  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.007730  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.017481  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:19.012085  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.982520  1629 rpcz_store.cc:276] 0709 13:01:17.387624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.387825 (+   201us) service_pool.cc:224] Handling call
0709 13:01:17.388205 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:17.388587 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.388628 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.138751 (+750123us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b2c00046b18. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:01:18.139176 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.158269 (+ 19093us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:18.731833 (+573564us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.961707 (+229874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.961780 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:18.961817 (+    37us) raft_consensus.cc:1685] finished
0709 13:01:18.970453 (+  8636us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.973694 (+  3241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22744,"prepare.queue_time_us":17759,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":38333,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":6859,"spinlock_wait_cycles":62000512,"thread_start_us":798427,"threads_started":3,"wal-append.queue_time_us":82314}
I20260709 13:01:19.018577  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:18.995072  3484 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.019958  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.004165  2292 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:19.008755  3678 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:19.020584  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:18.987157  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:19.030000  1631 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:18.993086  3578 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.009794  3575 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:19.014627  3626 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.041105  3432 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.870s	user 0.005s	sys 0.005s
I20260709 13:01:19.037329  3691 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:01:19.015852  3529 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.041664  3432 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.042119  3529 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.012877  3657 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.028858  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:19.035439  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1121) took 1798 ms. Trace:
I20260709 13:01:19.037245  3568 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.037505  3550 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.038700  3476 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.039893  3575 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.031646  3290 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.016304  3338 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.019788  3675 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:01:19.031420  3692 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:19.058012  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84619.5us
I20260709 13:01:19.138106  1552 rpcz_store.cc:276] 0709 13:01:17.236870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.237032 (+   162us) service_pool.cc:224] Handling call
0709 13:01:17.270585 (+ 33553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.337044 (+ 66459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.337075 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.337117 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:17.337151 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.035153 (+1698002us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b540036bfd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:19.035204 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.035363 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152419584}
I20260709 13:01:19.065119  3613 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.143827  3613 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.145587  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.146360  2293 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:19.146620  3613 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:19.146733  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 725) took 1766 ms (client timeout 1921 ms). Trace:
W20260709 13:01:19.147171  2293 rpcz_store.cc:269] 0709 13:01:17.380667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.395601 (+ 14934us) service_pool.cc:224] Handling call
0709 13:01:19.145439 (+1749838us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:19.146674 (+  1235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57894656}
I20260709 13:01:19.148289  3568 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.149130  3664 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.010s	user 0.016s	sys 0.013s
I20260709 13:01:19.047886  3167 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.037313  3578 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:19.150319  3578 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.043718  3432 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.151440  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1858) took 1239 ms. Trace:
I20260709 13:01:19.153000  3663 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.151830  1627 rpcz_store.cc:276] 0709 13:01:17.911960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.975673 (+ 63713us) service_pool.cc:224] Handling call
0709 13:01:17.976161 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:17.986392 (+ 10231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.986424 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:18.004676 (+ 18252us) log.cc:844] Serialized 52 byte log entry
0709 13:01:18.133028 (+128352us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:18.153321 (+ 20293us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.188193 (+ 34872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.188298 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.148527 (+960229us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:19.151169 (+  2642us) raft_consensus.cc:1685] finished
0709 13:01:19.151249 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.151377 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36238,"prepare.queue_time_us":15272,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":720,"raft.queue_time_us":25171,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"spinlock_wait_cycles":27346176,"thread_start_us":152787,"threads_started":2,"wal-append.queue_time_us":129763}
W20260709 13:01:19.119540  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111898us
I20260709 13:01:19.148793  3568 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:19.126824  3693 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:01:19.022253  3537 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:19.154650  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.164242  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:19.166193  1524 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.024099  3553 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.049714  3364 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.051865  3646 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.138729  3657 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:19.161983  3666 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.022470  3583 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.175226  2292 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:19.188271  3339 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.189055  3575 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.165324  3167 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.192437  3339 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.200953  3338 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.070482  3493 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.169989  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:19.170291  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 141490us
I20260709 13:01:19.205713  1525 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.042152  3691 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:19.161958  3537 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.169320  3696 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.169529  3476 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.172765  3678 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.173522  3692 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:19.041132  3626 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.174396  3646 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:19.176600  3583 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.180441  3526 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.181843  3325 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.169633  3697 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:19.140873  3675 consensus_queue.cc:1243] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:19.070883  3484 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:19.221546  3665 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.084s	user 0.007s	sys 0.000s
I20260709 13:01:19.221653  3526 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.222445  3325 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.222922  3646 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:19.201826  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 217470us
I20260709 13:01:19.211918  3692 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.219151  3697 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.157935  3613 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.216238  3699 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:19.218604  3626 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.225016  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.209684  3493 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.208406  3537 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:19.210136  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.226711  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.231555  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2052) took 1259 ms. Trace:
I20260709 13:01:19.232035  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.208916  3691 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:19.235328  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.235896  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:19.211179  3696 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:19.236683  3656 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.090s	user 0.005s	sys 0.004s
I20260709 13:01:19.236815  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:19.232077  3692 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.241729  3697 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.223850  3646 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.242250  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:19.242692  3678 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.244926  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.249140  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:19.258847  3493 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:19.284389  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78723.6us
I20260709 13:01:19.261485  3633 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:19.261209  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51138.2us
I20260709 13:01:19.263118  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.285835  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.259821  1633 rpcz_store.cc:276] 0709 13:01:17.972586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.986298 (+ 13712us) service_pool.cc:224] Handling call
0709 13:01:17.986632 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:18.003087 (+ 16455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.003118 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:18.022793 (+ 19675us) log.cc:844] Serialized 52 byte log entry
0709 13:01:18.064604 (+ 41811us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:18.064659 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.064749 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.221008 (+1156259us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:19.231255 (+ 10247us) raft_consensus.cc:1685] finished
0709 13:01:19.231276 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.231495 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14536,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":7757696,"thread_start_us":6085,"threads_started":1,"wal-append.queue_time_us":81627}
I20260709 13:01:19.271052  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:19.152364  3578 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.264149  3551 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:19.267093  3699 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.285899  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:19.266004  3666 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:19.212456  3529 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.264981  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:19.269196  3619 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.272037  2289 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.287829  3493 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.316969  1632 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:19.316850  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2051) took 1792 ms (client timeout 1764 ms). Trace:
I20260709 13:01:19.301608  3633 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.330344  3699 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.352757  3619 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.358846  3699 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:19.359131  2291 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:19.321476  3578 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:19.331554  3691 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.331614  3657 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:19.320633  3622 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 2.004s	user 0.005s	sys 0.006s
I20260709 13:01:19.296212  3701 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:19.351604  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.371630  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1138) took 1554 ms. Trace:
I20260709 13:01:19.296094  3669 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:19.351433  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:19.323680  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114518us
I20260709 13:01:19.264928  3613 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:19.352358  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 188176us
I20260709 13:01:19.288317  2292 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.356047  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:19.354869  1628 rpcz_store.cc:269] 0709 13:01:17.524661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.533132 (+  8471us) service_pool.cc:224] Handling call
0709 13:01:19.316664 (+1783532us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54000ccbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:19.316786 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390138624}
I20260709 13:01:19.296170  3550 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:19.361686  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.372093  1556 rpcz_store.cc:276] 0709 13:01:17.817384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.820006 (+  2622us) service_pool.cc:224] Handling call
0709 13:01:17.822131 (+  2125us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:17.844074 (+ 21943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.844128 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:17.845538 (+  1410us) log.cc:844] Serialized 590 byte log entry
0709 13:01:18.027958 (+182420us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:18.028003 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.028076 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.236016 (+1207940us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:19.371323 (+135307us) raft_consensus.cc:1685] finished
0709 13:01:19.371343 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.371572 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23429,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":5588224,"thread_start_us":181483,"threads_started":1,"wal-append.queue_time_us":277081}
I20260709 13:01:19.380085  1627 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:19.320524  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:19.363739  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98836.7us
I20260709 13:01:19.381179  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:19.381361  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:19.268370  3372 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:01:19.302124  3551 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:19.387212  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2050) took 1863 ms (client timeout 2211 ms). Trace:
W20260709 13:01:19.387702  1630 rpcz_store.cc:269] 0709 13:01:17.524301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.546022 (+ 21721us) service_pool.cc:224] Handling call
0709 13:01:19.387070 (+1841048us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002972d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:19.387157 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435199744}
I20260709 13:01:19.388907  3551 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.389241  3527 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.389488  2297 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:19.390590  3657 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.390677  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1853) took 1483 ms. Trace:
I20260709 13:01:19.371028  3448 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.391178  1634 rpcz_store.cc:276] 0709 13:01:17.907973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.975612 (+ 67639us) service_pool.cc:224] Handling call
0709 13:01:17.975881 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.053438 (+ 77557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.390480 (+1337042us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b540019c154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:19.390621 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160260352}
I20260709 13:01:19.401024  3568 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.402804  3633 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.404050  3696 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.404678  3696 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:19.409560  3476 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:19.412123  3703 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:01:19.412322  3448 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:19.412789  3448 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:19.413033  3689 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 13:01:19.414444  3666 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:19.414858  3666 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:19.417702  3430 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.421022  3339 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.421587  3339 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.422410  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 433) took 1256 ms. Trace:
I20260709 13:01:19.422955  2290 rpcz_store.cc:276] 0709 13:01:18.166172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.170872 (+  4700us) service_pool.cc:224] Handling call
0709 13:01:18.234842 (+ 63970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.956694 (+721852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.422020 (+465326us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:19.422308 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439229696}
I20260709 13:01:19.384915  3691 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.392102  3678 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:19.424193  3678 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.424549  3678 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:19.425755  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.426172  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.428906  3339 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.430471  3657 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.436111  3613 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.397718  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.434602  3484 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:19.439208   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:19.443202  3493 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.371635  3529 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.454733  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:19.461231  3698 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.461742  3698 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:19.464807  3526 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.465101  3696 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:19.467931  3493 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.436115  3657 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.465883  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:19.478921  3626 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:19.459203  3619 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:19.492360  3583 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:19.465442  3432 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:19.492853  3583 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:19.415623  3666 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.493661  3583 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.494784  3583 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.459566  3668 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:19.497990  2292 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:19.497071  3607 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:19.498258  3668 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:19.474396  2289 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:19.497306  1455 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 3, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:19.505930  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 287) took 8894 ms. Trace:
I20260709 13:01:19.465304  3526 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.499850  3668 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:19.528687  1455 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 3, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.501595  3432 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:19.536978  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 251132us
W20260709 13:01:19.543525  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 237958us
I20260709 13:01:19.545194  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.534240  2294 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:19.548038  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.551692  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.552148  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.553030  1556 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:19.546733  1455 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:19.557413  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.504709  3448 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.558782  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.559333  1557 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.561745  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.562500  1553 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:19.567824  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.568349  1558 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.521981  1458 rpcz_store.cc:276] 0709 13:01:10.604309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.604501 (+   192us) service_pool.cc:224] Handling call
0709 13:01:19.498725 (+8894224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:15.307069 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:15.373437 (+ 66368us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:15.373485 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:01:15.373497 (+    12us) tablet.cc:662] Decoding operations
0709 13:01:15.373900 (+   403us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:15.373985 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:15.374011 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:15.374296 (+   285us) tablet.cc:701] Row locks acquired
0709 13:01:15.374337 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:01:15.374491 (+   154us) write_op.cc:270] Start()
0709 13:01:15.374629 (+   138us) write_op.cc:276] Timestamp: P: 1783602075374434 usec, L: 0
0709 13:01:15.374665 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:01:15.376803 (+  2138us) log.cc:844] Serialized 541 byte log entry
0709 13:01:15.792955 (+416152us) op_driver.cc:464] REPLICATION: finished
0709 13:01:15.908392 (+115437us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9dfe28 00007f70d241a581 00007f70d2419d58 00007f70d241e9f1 00007f70d241e9a9 00007f70d241e939 00007f70d241e901 00007f70d241db2d 000000000038f354 000000000038f199
0709 13:01:16.452323 (+543931us) write_op.cc:301] APPLY: starting
0709 13:01:16.479907 (+ 27584us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:16.518032 (+ 38125us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:16.527865 (+  9833us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:16.566218 (+ 38353us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:16.576063 (+  9845us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:16.576126 (+    63us) write_op.cc:312] APPLY: finished
0709 13:01:16.647143 (+ 71017us) log.cc:844] Serialized 23 byte log entry
0709 13:01:16.664382 (+ 17239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:16.669163 (+  4781us) write_op.cc:454] Released schema lock
0709 13:01:16.675904 (+  6741us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:16.692455 (+ 16551us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:16.692487 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":681946,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":608820,"apply.run_cpu_time_us":3087,"apply.run_wall_time_us":273715,"num_ops":1,"prepare.queue_time_us":188724,"prepare.run_cpu_time_us":40965,"prepare.run_wall_time_us":626401,"raft.queue_time_us":178474,"raft.run_cpu_time_us":11053,"raft.run_wall_time_us":241018,"replication_time_us":401808,"spinlock_wait_cycles":3174144,"thread_start_us":496783,"threads_started":4,"wal-append.queue_time_us":182832,"wal-append.run_cpu_time_us":15966,"wal-append.run_wall_time_us":2124168}]]}
I20260709 13:01:19.544502  3657 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:19.573166  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 191825us
I20260709 13:01:19.573333  1553 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:19.575495  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:19.577431  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.573386  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.579346  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:19.579774  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.580466  1632 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:19.580940  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.581514  1634 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:19.584098  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.584892  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.585916  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.585162  1703 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:19.591446  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.591734  1705 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.592062  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.596521  1628 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:19.596597  1704 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:19.596892  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.597393  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.597470  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.592700  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.592998  1702 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.599411  1635 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:19.593837  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:19.599961  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.595170  1558 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:19.605232  2291 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:19.530341  3551 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.606340  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.607723  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.608170  3551 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:19.608677  3339 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.609180  3339 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.611022  3339 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.574558  3613 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.611735  3613 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.612301  1701 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:19.592267  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.613229  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.613641  1709 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:19.614284  1455 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 2, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:19.614995  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.616050  1455 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.617486  1626 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:19.614339  1705 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:19.617422  1455 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.618306  1557 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:19.620597  1455 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 3, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.625419  1455 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.627980  3668 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:19.631258  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51989.1us
I20260709 13:01:19.535440  3575 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:01:19.641530  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.638610  1705 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.640004  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.542066  3626 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:01:19.635743  3568 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:19.646541  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:19.649237  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.649716  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.650177  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:19.650923  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:19.654884  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.656219  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.657095  1560 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:19.653734  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.657801  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.658376  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.655624  1553 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:19.658855  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:19.658126  1705 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:19.641083  3691 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.554608  3699 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.613595  3613 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.661859  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:19.640964  3702 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.665935  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.640529  3669 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:19.667757  1555 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.650355  3551 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:19.674503  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.652073  3696 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.690760  1558 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:19.675174  1555 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:19.672856  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:19.685299  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.689664  1560 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:19.661150  1551 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:19.701718  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.702399  1558 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:19.703303  1705 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:19.703438  1702 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.644448  3553 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:19.704557  1556 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:19.706324  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.703197  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:19.711210  1558 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:19.711973  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.713379  1705 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:19.713901  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.654179  3670 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:19.714823  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.685415  3613 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.670971  2292 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:01:19.709615  1553 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:19.714733  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.652688  3692 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.657478  3710 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
W20260709 13:01:19.764851  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.729064  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:19.816869  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 118902us
I20260709 13:01:19.721244  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:19.888748  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.890324  1703 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:19.890818  1560 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:19.890918  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.891306  1552 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:19.798370  1556 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:19.891619  1704 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:19.687098  3551 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:19.887562  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.696331  3669 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.944298  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.966114  1554 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:01:19.966534  1554 pending_rounds.cc:85] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 2
I20260709 13:01:19.966601  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.966863  1554 pending_rounds.cc:107] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:01:19.967231  1554 raft_consensus.cc:2889] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:19.887468  1705 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:19.893195  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:19.795395  3537 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.004s
I20260709 13:01:19.796759  3526 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:19.978937  1703 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:01:19.979322  1703 pending_rounds.cc:85] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 2
I20260709 13:01:19.979566  1703 pending_rounds.cc:107] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:01:19.979820  1703 raft_consensus.cc:2889] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:19.982244  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.982961  1704 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 1.
I20260709 13:01:19.665417  3432 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:19.976246  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:19.887496  3550 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:19.990278  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.012130  1559 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:19.992400  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:20.000512  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.013422  1554 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:20.005188  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.794s (SENT) [suppressed 9 similar messages]
I20260709 13:01:19.892088  2292 pending_rounds.cc:85] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 6
I20260709 13:01:19.714545  3484 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.816825  3589 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:19.704306  3696 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:19.792017  3713 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.032835  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.046300  1552 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:20.038317  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.033565  1704 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:20.046834  1556 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:20.047613  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:19.689049  3702 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.049455  3687 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.050406  1701 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:20.051159  3687 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.970273  3551 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:20.014712  3537 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.056273  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.056624  2296 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:20.057366  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.057636  3537 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.057964  1552 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:20.049893  3702 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.061527  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.038002  3484 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:20.061849  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:20.063719  3484 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.064000  3448 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.025559  2292 pending_rounds.cc:107] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:01:20.064774  2292 raft_consensus.cc:2889] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:20.066175  3607 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.000s
I20260709 13:01:20.004125  3339 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.791380  3678 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.067180  3339 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.043597  1703 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.068243  1704 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:20.069358  3339 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.069826  3678 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.037050  3710 consensus_queue.cc:1243] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:20.044602  1554 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:20.066395  3551 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.081868  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.083755  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:19.970562  3716 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:19.784968  3714 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.092451  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.094446  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:20.094446  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:19.964227  3613 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.056921  3699 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.105995  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.130151  1706 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:20.130836  2295 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:20.131882  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:20.133183  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.134325  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.106870  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.138441  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:20.142043  3583 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.001s
I20260709 13:01:20.142889  1555 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:19.901744  3646 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:19.790293  3691 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:20.126292  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.151580  1552 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:20.145030  1705 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:20.153175  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.154280  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.154808  1705 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:20.155619  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.135494  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:20.147854  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.094095  3527 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.101147  3678 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.107585  3537 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:20.171350  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 166195us
I20260709 13:01:20.152128  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.175114  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.179426  1554 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:20.181622  3696 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.182266  3725 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.182838  3725 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.078446  3589 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.185400  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:20.077920  3713 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.188748  3613 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.190471  3692 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.165807  3714 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.191574  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:20.191969  3669 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.165416  3550 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.145991  3712 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:20.194355  3527 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.194808  3583 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.146345  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.196099  1556 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:20.211843  1556 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:20.212261  1700 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:20.196796  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.213886  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.215147  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.144240  3716 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:20.199085  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:20.245313  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46290.9us
I20260709 13:01:20.246413  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.199354  1554 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:20.199647  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.248423  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:20.199941  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 101021us
I20260709 13:01:20.202344  3720 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:20.183708  3448 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:01:20.168390  3646 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:20.206776  1552 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:20.201526  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 209184us
I20260709 13:01:20.184573  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.201588  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:20.249536  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:20.270179  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55468us
I20260709 13:01:20.270452  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.212575  1556 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.270975  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:20.271590  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.248435  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:20.272547  1554 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:20.168056  3691 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.264057  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:20.276212  3527 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.276161  3669 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:20.268196  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:20.278039  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.278642  3537 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:20.279347  3537 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:20.287195  3714 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.286520  3537 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.289039  3725 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.278173  3691 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.271749  1556 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.294204  3589 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.279964  2295 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.203210  3696 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.284283  3364 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:20.284339  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82822.6us
I20260709 13:01:20.284693  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.296429  3713 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.296597  3364 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:20.297096  3713 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:20.297366  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:20.297871  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.251204  3720 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:20.298555  3702 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.299044  3720 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.297590  3364 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.285145  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:20.299861  3702 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.300760  3339 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:20.300957  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.301877  3722 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.302294  3716 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.302850  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.303544  3613 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.305408  3698 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:20.305596  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.306356  3698 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.309816  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.310623  3698 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:20.241695  3712 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.311023  3698 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.311451  3698 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:20.312721  3712 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:20.314481  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:20.314671  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.318046  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:20.322314  2295 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
W20260709 13:01:20.326419  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55537.8us
I20260709 13:01:20.320322  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.327721  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:20.326365  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.328495  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.251364  3692 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.332285  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:20.416354  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:20.360059  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.256760  3706 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:20.433264  1705 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:20.434367  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:20.436877  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 109240us
I20260709 13:01:20.270967  3553 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.438395  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:20.441489  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:20.442278  1703 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:20.455668  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.327255  3613 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:20.327939  3696 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:20.274462  3721 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:20.478735  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.415714  3583 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:20.326337  3713 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:20.469394  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 205413us
I20260709 13:01:20.469581  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:20.496097  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 348314us
I20260709 13:01:20.439225  3311 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:20.415535  3699 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:20.499789  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.440567  3619 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.435497  3706 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:20.497694  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:20.504998  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.327173  3712 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.155s
W20260709 13:01:20.509371  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.327029  3432 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.170s
W20260709 13:01:20.516018  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46541.4us
I20260709 13:01:20.479171  3436 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.484454  3613 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:20.524380  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:20.527583  1703 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:20.486570  3663 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.487416  3692 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:20.488216  3696 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.463336  3673 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:01:20.546133  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:20.548843  3730 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.551383  3696 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:20.578467  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.769940  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1862) took 1194 ms. Trace:
W20260709 13:01:20.770579  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.548976  3613 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:20.515538  3706 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.477941  3724 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:01:20.546078  3699 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:20.600636  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76363.1us
W20260709 13:01:20.589404  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.778756  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.002s	sys 0.014s
I20260709 13:01:20.611248  3436 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
W20260709 13:01:20.815958  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 455972us
I20260709 13:01:20.646129  3607 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:20.793104  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 247058us
I20260709 13:01:20.602689  3290 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:01:20.770395  1559 rpcz_store.cc:276] 0709 13:01:19.575519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.577060 (+  1541us) service_pool.cc:224] Handling call
0709 13:01:19.577300 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.020012 (+442712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.769640 (+749628us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:20.769875 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999435136}
W20260709 13:01:20.809115  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.003s	sys 0.014s
W20260709 13:01:20.824060  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 675653us
I20260709 13:01:20.514918  3646 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:20.792510  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.817600  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:20.771726  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 232367us
I20260709 13:01:20.769601  3730 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.800724  3691 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.794222  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 457) took 1142 ms. Trace:
I20260709 13:01:20.850090  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:20.770685  3738 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:20.856436  3738 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.855865  2289 rpcz_store.cc:276] 0709 13:01:19.651899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.653706 (+  1807us) service_pool.cc:224] Handling call
0709 13:01:20.181788 (+528082us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:20.182053 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.182318 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:20.511707 (+329389us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.614850 (+103143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.616054 (+  1204us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:20.621268 (+  5214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.646175 (+ 24907us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.785886 (+139711us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127118,"raft.queue_time_us":3407,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4731,"spinlock_wait_cycles":45754752,"thread_start_us":395760,"threads_started":2,"wal-append.queue_time_us":132183}
I20260709 13:01:20.858170  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.858652  3738 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.858791  2289 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:20.808492  3527 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:20.864521  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:20.806368  3311 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:20.820648  3607 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.866600  3432 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.867921  3750 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.876960  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.850277  3730 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.868854  3436 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.869443  3725 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:20.885566  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:20.810948  3705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.020s	sys 0.011s
I20260709 13:01:20.823618  3737 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:20.872643  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.873260  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:20.868718  3732 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:01:20.876441  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 1871) took 1206 ms. Trace:
I20260709 13:01:20.828116  3699 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.891088  1555 rpcz_store.cc:276] 0709 13:01:19.670590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.710371 (+ 39781us) service_pool.cc:224] Handling call
0709 13:01:20.876296 (+1165925us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:20.876388 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980423552}
I20260709 13:01:20.877381  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.862382  3583 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:20.896035  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.893167  3725 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:20.897236  3725 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.898787  3737 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:20.865012  3619 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:20.899706  3737 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.900903  3619 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:20.901300  3619 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:20.902083  3619 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.903369  3619 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.879179  1631 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:20.881446  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.883636  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.904553  1559 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:20.884878  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:20.870599  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:20.874253  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:20.905505  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:20.871196  3738 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.886834  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.873821  3607 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:20.912402  1627 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:20.890056  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:20.890450  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.913853  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.892283  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.914703  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:20.855890  3691 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:20.893172  3752 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:20.893043  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:20.893518  3731 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.344s	user 0.000s	sys 0.000s
I20260709 13:01:20.905508  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.917254  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.917691  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:20.888854  3705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.020s	sys 0.011s
I20260709 13:01:20.922960  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:20.871361  3527 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:20.924675  1555 raft_consensus.cc:2376] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6. Re-sending same reply.
I20260709 13:01:20.921706  1559 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:20.926795  3750 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:20.928433  3722 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.935963  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.932518  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:20.932497  1560 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:20.940289  1555 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:20.912782  3167 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:20.918671  3550 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.940441  1559 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:20.925104  2289 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:20.943181  1555 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:20.943502  1559 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.934667  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.918340  3339 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:01:20.945135  1552 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:20.965515  3526 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:20.966995  3526 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:20.945616  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:20.967367  3526 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:20.968160  3526 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:20.937839  3705 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.931846  3724 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:01:20.920066  3663 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:01:20.972209  2289 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:20.974466  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.975020  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:20.978160  3527 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:20.978765  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:20.979382  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:20.969559  3526 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.982059  1632 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:20.918802  3716 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:01:20.989197  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:20.918928  3738 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.000931  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1134) took 1417 ms. Trace:
I20260709 13:01:20.997889  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:20.968421  3550 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.001400  1707 rpcz_store.cc:276] 0709 13:01:19.583680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.583819 (+   139us) service_pool.cc:224] Handling call
0709 13:01:19.584093 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.888905 (+304812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.888956 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.889457 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.245007 (+355550us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.245046 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.245108 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.000741 (+755633us) raft_consensus.cc:1685] finished
0709 13:01:21.000761 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.000876 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214163,"raft.queue_time_us":212401,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":467608,"threads_started":2,"wal-append.queue_time_us":493651}
W20260709 13:01:20.983364  3749 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:20.987478  2289 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:20.980937  3705 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:21.016822  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 99700.1us
I20260709 13:01:21.020866  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:21.034087  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:21.101959  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 258683us
I20260709 13:01:21.012144  3550 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:21.017817  3743 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:01:21.013321  3476 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:21.000489  3167 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 13:01:20.970233  3311 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.106438  3669 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.103642  3737 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.158350  3675 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.170459  3675 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.121435  3663 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:21.124177  3750 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:01:21.217476  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.128172  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 742) took 1007 ms. Trace:
I20260709 13:01:21.129204  3191 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.233067  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:21.231359  3675 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.231338  3741 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.339s	user 0.001s	sys 0.001s
W20260709 13:01:21.255607  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:21.245101  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.266427  3745 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:21.209740  3663 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:21.227082  3750 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:21.281176  3754 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:01:21.287056  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:21.287940  1707 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:21.281613  3776 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:21.282464  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 284656us
I20260709 13:01:21.232533  3725 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.302387  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:21.325934  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 380367us
I20260709 13:01:21.159794  3669 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.330732  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:21.286146  3720 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.987s	user 0.013s	sys 0.000s
I20260709 13:01:21.244575  3765 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:21.325991  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 190095us
I20260709 13:01:21.354348  3675 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:21.354976  3675 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.355509  3675 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:21.359984  1707 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:21.326347  3550 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.360213  3699 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.182519  3737 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.288908  3691 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.231541  2289 rpcz_store.cc:276] 0709 13:01:20.120529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.986984 (+866455us) service_pool.cc:224] Handling call
0709 13:01:20.987269 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.101353 (+114084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.122961 (+ 21608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.289036  3663 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.375105  3760 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.244836  3191 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:21.382186  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.300395  3553 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.393834  3737 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.395449  3742 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
I20260709 13:01:21.424414  3550 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:21.426560  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 405758us
I20260709 13:01:21.427727  3758 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:21.428301  3758 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:21.429574  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:21.429296  3758 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.430334  1707 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:21.430426  3745 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.430734  3758 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:21.430925  3745 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:21.432039  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.434075  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.434751  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:21.435942  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108925us
I20260709 13:01:21.437405  3753 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:21.334065  3725 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.446107  3705 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.447618  2289 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:21.402004  3698 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.090s	user 0.005s	sys 0.007s
W20260709 13:01:21.476027  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 113863us
I20260709 13:01:21.372359  3720 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.485247  1703 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:21.490262  1558 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:01:21.490728  1558 pending_rounds.cc:85] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 2
I20260709 13:01:21.490981  1558 pending_rounds.cc:107] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:01:21.422283  3691 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.489162  3720 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:21.491300  1558 raft_consensus.cc:2889] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:21.493954  3759 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:01:21.497013  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1154) took 1805 ms. Trace:
I20260709 13:01:21.500659  1558 rpcz_store.cc:276] 0709 13:01:19.692079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.489345 (+1797266us) service_pool.cc:224] Handling call
0709 13:01:21.489778 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.496687 (+  6909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.496939 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.401544  3760 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.448565  3737 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:21.511111  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.478924  3646 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.536909  1709 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:21.493242  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 464) took 1337 ms. Trace:
I20260709 13:01:21.441969  3553 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:21.506814  3706 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.002s
I20260709 13:01:21.538309  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1915) took 1528 ms. Trace:
I20260709 13:01:21.548655  1709 rpcz_store.cc:276] 0709 13:01:20.010295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.010555 (+   260us) service_pool.cc:224] Handling call
0709 13:01:21.531312 (+1520757us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:21.531512 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.537983 (+  6471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.538238 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12904704}
I20260709 13:01:21.503609  3725 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.545710  3775 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:01:21.546767  3774 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.483209  3698 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:21.502743  3738 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.533982  3737 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.492121  3750 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.509522  3691 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.540665  3646 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.542410  2289 rpcz_store.cc:276] 0709 13:01:20.144762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.446866 (+1302104us) service_pool.cc:224] Handling call
0709 13:01:21.447245 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.448064 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.481849 (+ 33785us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:21.603806  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.588860  3745 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.534490  3772 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:01:21.554085  3706 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.600805  3760 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.627861  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 321597us
I20260709 13:01:21.630722  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:21.633081  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:21.635752  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1918) took 1590 ms. Trace:
I20260709 13:01:21.636139  1704 rpcz_store.cc:276] 0709 13:01:20.045303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.047397 (+  2094us) service_pool.cc:224] Handling call
0709 13:01:21.635596 (+1588199us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:21.635700 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45649152}
I20260709 13:01:21.637676  1554 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:21.637869  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:21.588518  3774 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.646466  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:21.647372  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.635094  3191 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.634292  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:21.640964  3745 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:21.661309  3772 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:21.668817  3766 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:01:21.671568  1558 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:21.657584  3725 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.674952  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:21.678196  1704 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:21.668891  3772 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.686174  3777 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.686930  3777 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.690405  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:21.691505  2288 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:21.696137  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 1869) took 2037 ms (client timeout 1757 ms). Trace:
W20260709 13:01:21.696736  1557 rpcz_store.cc:269] 0709 13:01:19.658804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.695851 (+2037047us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:21.696020 (+   169us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:21.697860  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.698637  1557 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
W20260709 13:01:21.698971  3696 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.894s	user 0.010s	sys 0.000s
I20260709 13:01:21.705343  3737 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:21.667503  3191 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:01:21.670230  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:21.707870  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:21.713866  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:21.714668  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.716218  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.716938  1557 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:21.717947  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:21.720254  3755 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.356s	user 0.007s	sys 0.010s
W20260709 13:01:21.721616  3613 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.905s	user 0.010s	sys 0.000s
I20260709 13:01:21.722335  2294 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:20.274994  3710 consensus_queue.cc:1175] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 4 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:21.723913  1557 raft_consensus.cc:2410] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:21.728441  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1132) took 2145 ms. Trace:
I20260709 13:01:21.728823  1708 rpcz_store.cc:276] 0709 13:01:19.583204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.583301 (+    97us) service_pool.cc:224] Handling call
0709 13:01:19.583557 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.615279 (+ 31722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.615323 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.615359 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:19.615398 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.728202 (+2112804us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b5400165654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:21.728265 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.728388 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72564992}
I20260709 13:01:21.675053  3782 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.732223  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:21.681599  3553 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:21.738024  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 148974us
W20260709 13:01:21.739326  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.734669  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.741043  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:21.743606  1557 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:21.728279  3526 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.004s
I20260709 13:01:21.742657  1599 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:21.677331  3669 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.745414  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 140979us
I20260709 13:01:21.707828  3691 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.753338  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:21.754179  3553 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:21.754544  3553 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:21.754866  3553 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.756613  2289 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:21.757300  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 467) took 1610 ms. Trace:
I20260709 13:01:21.757635  2289 rpcz_store.cc:276] 0709 13:01:20.147362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.745535 (+1598173us) service_pool.cc:224] Handling call
0709 13:01:21.756360 (+ 10825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.757032 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.757243 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.731612  3613 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.738488  3430 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:21.762977  3619 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.859s	user 0.012s	sys 0.000s
W20260709 13:01:21.764580  3730 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.876s	user 0.007s	sys 0.004s
I20260709 13:01:21.765331  3782 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.766080  3778 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:21.767356  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 741) took 1685 ms (client timeout 1918 ms). Trace:
I20260709 13:01:21.699514  3696 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.767524  3675 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.762027  3613 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:21.683674  3725 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.774205  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:21.745577  3607 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.001s
I20260709 13:01:21.734966  3673 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:21.777796  2295 rpcz_store.cc:269] 0709 13:01:20.082025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.135247 (+ 53222us) service_pool.cc:224] Handling call
0709 13:01:21.767177 (+1631930us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540040b5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:21.767294 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281472256}
I20260709 13:01:21.780191  1557 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:21.711993  3772 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.782689  3706 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.744181  2288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 733) took 1664 ms (client timeout 1608 ms). Trace:
I20260709 13:01:21.775043  3745 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.783329  3785 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:21.801105  3785 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.801769  3785 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:21.801417  2288 rpcz_store.cc:269] 0709 13:01:20.079829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.088269 (+  8440us) service_pool.cc:224] Handling call
0709 13:01:21.690169 (+1601900us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:21.737855 (+ 47686us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34440576}
I20260709 13:01:21.759701  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 468) took 1612 ms. Trace:
I20260709 13:01:21.793321  3774 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.787032  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:21.809270  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:21.775696  3669 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.797446  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:21.798285  2295 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:21.794202  3430 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:21.783092  3754 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:01:21.815496  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:21.795145  3526 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:21.798784  3696 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.795205  3673 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:21.795948  3607 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.814893  3613 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:21.778213  3730 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.821983  3526 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.783690  3778 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.823433  3720 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:21.819667  3669 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.824472  3673 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:21.826337  1631 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:21.835201  1628 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:21.809636  3745 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.828132  3613 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.836915  1635 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:21.824376  3550 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:01:21.839870  3715 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:21.840021  3787 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.832683  3526 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.845994  3782 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.808467  2289 rpcz_store.cc:276] 0709 13:01:20.147682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.758447 (+1610765us) service_pool.cc:224] Handling call
0709 13:01:21.759175 (+   728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.759366 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.759403 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.759451 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:21.759485 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.759541 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.759650 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.849251  2289 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:21.850240  3715 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.850638  3715 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.842770  3613 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.853067  3787 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.863507  3745 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.863834  3668 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.863642  3754 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:01:21.842967  3720 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:01:21.875753  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 452) took 2636 ms. Trace:
I20260709 13:01:21.876370  2296 rpcz_store.cc:276] 0709 13:01:19.233042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.376977 (+143935us) service_pool.cc:224] Handling call
0709 13:01:19.398002 (+ 21025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.057015 (+659013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.869437 (+1812422us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:21.869619 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515462784}
W20260709 13:01:21.881265  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.885233  3698 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:21.886935  3663 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.893369  1556 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:01:21.893827  1556 pending_rounds.cc:85] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 2
I20260709 13:01:21.894125  1556 pending_rounds.cc:107] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:01:21.894414  1556 raft_consensus.cc:2889] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:21.895293  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1163) took 1700 ms. Trace:
I20260709 13:01:21.895680  1556 rpcz_store.cc:276] 0709 13:01:20.195264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.195594 (+   330us) service_pool.cc:224] Handling call
0709 13:01:20.195965 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.887957 (+1691992us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540010aa20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e6d87 00007f70ce9e7470 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:01:21.894976 (+  7019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.895225 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3209546496}
I20260709 13:01:21.897610  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:21.898351  1556 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:21.871408  3785 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.906020  3782 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:21.906570  3782 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:21.906886  3785 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:21.907007  3787 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.878574  3666 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:21.879472  3725 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.887951  3675 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.929225  3673 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:21.892994  2289 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:21.935885  3673 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.888844  3772 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:21.964174  3782 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:21.965754  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.966730  3673 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:21.967705  3607 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.974740  3778 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.975980  3673 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:21.967352  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 342058us
I20260709 13:01:21.972909  3669 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:21.985380  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:21.992156  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:21.993528  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.005913  2292 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:21.985055  3730 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:21.997543  3778 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.000941  3669 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:22.014343  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 191773us
I20260709 13:01:21.997855  3793 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:22.019402  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:22.008674  3782 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.008087  3780 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.013415  3663 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.002794  3553 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.026696  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:22.031126  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.031872  2291 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
W20260709 13:01:22.032385  2291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 455) took 2384 ms (client timeout 2466 ms). Trace:
I20260709 13:01:22.032853  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:22.033747  1552 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:22.029745  3780 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.026507  3722 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:01:22.036936  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:22.037740  3493 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:01:22.038185  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.038780  1555 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:22.032845  2291 rpcz_store.cc:269] 0709 13:01:19.647552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.652978 (+  5426us) service_pool.cc:224] Handling call
0709 13:01:22.030949 (+2377971us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:22.032311 (+  1362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33799040}
I20260709 13:01:22.040508  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:22.041042  1557 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.041103  1629 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:22.041867  3613 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.044147  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.045496  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.035218  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 465) took 1888 ms. Trace:
I20260709 13:01:22.022821  3698 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:22.047966  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2090) took 1158 ms. Trace:
I20260709 13:01:22.048095  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:22.048353  1628 rpcz_store.cc:276] 0709 13:01:20.889931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.890039 (+   108us) service_pool.cc:224] Handling call
0709 13:01:20.890327 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.838135 (+947808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.047718 (+209583us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400140ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:22.047912 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172329344}
I20260709 13:01:22.049258  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:22.050241  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:22.050515  3711 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:22.051402  3711 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.051637  3607 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:22.023722  3782 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.053318  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1874) took 1080 ms. Trace:
I20260709 13:01:22.049331  3493 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:22.054680  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2080) took 1178 ms. Trace:
I20260709 13:01:22.058374  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.020426  3778 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.057369  1631 rpcz_store.cc:276] 0709 13:01:20.972930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.974740 (+  1810us) service_pool.cc:224] Handling call
0709 13:01:22.053168 (+1078428us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54001e8554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:22.053262 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822656256}
I20260709 13:01:22.046463  2294 rpcz_store.cc:276] 0709 13:01:20.146741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.737783 (+1591042us) service_pool.cc:224] Handling call
0709 13:01:21.760190 (+ 22407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:21.770696 (+ 10506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.770729 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:21.779866 (+  9137us) log.cc:844] Serialized 52 byte log entry
0709 13:01:21.983373 (+203507us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.983427 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.983513 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.032941 (+ 49428us) raft_consensus.cc:1685] finished
0709 13:01:22.032963 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.035158 (+  2195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10096,"raft.queue_time_us":11072,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":18497,"thread_start_us":174322,"threads_started":1,"wal-append.queue_time_us":218119}
I20260709 13:01:22.063412  1634 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:22.067559  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.067740  1634 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:22.068226  1627 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:22.039561  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:22.060395  3668 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.068899  1555 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:22.061311  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:22.065698  3675 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:22.065600  3777 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:22.070073  3675 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:22.071494  3764 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.717s	user 0.003s	sys 0.001s
I20260709 13:01:22.063902  3787 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.060782  3493 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.064287  1635 rpcz_store.cc:276] 0709 13:01:20.876349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.877142 (+   793us) service_pool.cc:224] Handling call
0709 13:01:22.054490 (+1177348us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b540019cb54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:22.054617 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171711744}
I20260709 13:01:22.065260  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.066222  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.071913  3526 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.072350  3675 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.063787  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.051726  3711 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:22.073139  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 260) took 2185 ms. Trace:
I20260709 13:01:22.075009  3782 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.052641  3191 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:22.076447  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.077132  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.077603  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:22.079988  3613 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:22.083212  1552 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:22.068564  1634 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.085726  1557 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:22.081744  3537 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.119696  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.120002  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:22.121825  1552 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:22.120734  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.123229  1557 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:22.088265  1555 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:22.091753  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.101567  1706 rpcz_store.cc:276] 0709 13:01:19.888147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.888288 (+   141us) service_pool.cc:224] Handling call
0709 13:01:19.888554 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.130920 (+242366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.072909 (+1941989us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400042754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:22.073080 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554889088}
I20260709 13:01:22.075642  3712 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:01:22.090699  3484 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:01:22.128680  3484 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:22.129154  3484 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:22.094161  3493 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:22.129614  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.130069  3782 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.130640  3782 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:22.131742  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.094312  3666 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:22.134712  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:22.135433  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:22.137176  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.103704  3787 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.104612  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:22.197000  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 171625us
I20260709 13:01:22.091631  3668 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.060575  3793 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.109115  3553 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:22.093509  3777 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:22.112252  3669 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:22.112870  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2084) took 1232 ms. Trace:
I20260709 13:01:22.111994  3715 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.066033  3550 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.135452  1556 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:22.196169  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60809.4us
I20260709 13:01:22.232090  1626 rpcz_store.cc:276] 0709 13:01:20.880541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.880680 (+   139us) service_pool.cc:224] Handling call
0709 13:01:20.891762 (+ 11082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.904016 (+ 12254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.904046 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.918144 (+ 14098us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.021587 (+103443us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.021640 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.021727 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.044099 (+1022372us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:22.112532 (+ 68433us) raft_consensus.cc:1685] finished
0709 13:01:22.112555 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.112795 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11847,"raft.queue_time_us":12014,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":7854,"spinlock_wait_cycles":18563328,"thread_start_us":90225,"threads_started":1,"wal-append.queue_time_us":339599}
I20260709 13:01:22.116349  3191 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:22.279641  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.195902  3685 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:22.306779  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.460633  1552 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:22.298557  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.211866  3448 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:22.570633  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:22.571684  1556 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
W20260709 13:01:22.577240  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.580068  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.582691  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.196796  3537 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.274258  3777 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.679667  1626 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:22.850673  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.012s	sys 0.004s
I20260709 13:01:22.320465  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:22.302285  3678 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.881186  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.594590  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:22.446163  3668 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:22.864948  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.012s	sys 0.004s
I20260709 13:01:22.308458  3550 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:22.349153  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 470) took 1484 ms. Trace:
I20260709 13:01:22.883498  2293 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:22.847168  3553 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.433084  3787 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.884311  1629 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:22.444890  3715 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:22.441167  3526 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:22.484958  3191 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:22.593760  3697 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.308714  3793 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.465261  3753 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:22.790681  3493 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.000s
I20260709 13:01:22.825977  3290 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.889395  3678 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.564906  3758 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:22.712572  3626 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:22.909060  3663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.007s	sys 0.021s
I20260709 13:01:22.428651  3782 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.898350  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.866308  3778 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.788501  3666 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.506533  3711 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.928064  3678 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:22.929067  3711 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:22.915990  3290 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.931015  3290 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:22.914417  3493 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.904608  3697 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.933652  3493 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:22.934625  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.937203  3678 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.937697  3678 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.940375  3678 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:22.934631  1552 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:22.673408  3685 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:22.920738  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.900259  3553 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.926577  3666 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.926448  3550 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.818427  3725 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:22.925863  2293 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:22.922840  3663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.007s	sys 0.022s
I20260709 13:01:22.917603  3448 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:22.922245  3745 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.958214  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.964561  2293 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:22.914633  3753 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.967610  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.918955  3758 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:22.971722  2293 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:22.922681  3626 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:22.971199  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51401.9us
I20260709 13:01:22.983116  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.984133  2293 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:22.986483  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:22.987756  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1169) took 2761 ms. Trace:
I20260709 13:01:22.988179  1560 rpcz_store.cc:276] 0709 13:01:20.222976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.807044 (+584068us) service_pool.cc:224] Handling call
0709 13:01:20.833523 (+ 26479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.932384 (+ 98861us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:20.990893 (+ 58509us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.588365 (+597472us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.588596 (+   231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.588646 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.588690 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.984013 (+1395323us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54002bb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:22.984084 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.984314 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169550,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":1266,"raft.queue_time_us":3199,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":6596,"spinlock_wait_cycles":226895232,"thread_start_us":580313,"threads_started":2,"wal-append.queue_time_us":567751,"wal-append.run_cpu_time_us":2398,"wal-append.run_wall_time_us":1083469}
I20260709 13:01:22.989672  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1183) took 1275 ms. Trace:
I20260709 13:01:22.990114  1556 rpcz_store.cc:276] 0709 13:01:21.714092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.644526 (+930434us) service_pool.cc:224] Handling call
0709 13:01:22.645013 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.645575 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:01:22.645615 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.873561 (+227946us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.964134 (+ 90573us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:22.965656 (+  1522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.965726 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.989405 (+ 23679us) raft_consensus.cc:1685] finished
0709 13:01:22.989426 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.989596 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":49152,"thread_start_us":90237,"threads_started":1,"wal-append.queue_time_us":114279}
I20260709 13:01:22.992159  1556 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:22.992822  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1897) took 1252 ms. Trace:
I20260709 13:01:22.993144  1556 rpcz_store.cc:276] 0709 13:01:21.740293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.991595 (+1251302us) service_pool.cc:224] Handling call
0709 13:01:22.991933 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.992578 (+   645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.992766 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.994663  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1185) took 1166 ms. Trace:
I20260709 13:01:22.994995  1556 rpcz_store.cc:276] 0709 13:01:21.828620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.993908 (+1165288us) service_pool.cc:224] Handling call
0709 13:01:22.994181 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.994360 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.994388 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.994429 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:22.994463 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.994516 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.994612 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.996563  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1187) took 1130 ms. Trace:
W20260709 13:01:22.997223  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.996897  1556 rpcz_store.cc:276] 0709 13:01:21.865981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.995842 (+1129861us) service_pool.cc:224] Handling call
0709 13:01:22.996092 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.996262 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.996293 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.996332 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:22.996363 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.996416 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.996509 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.998422  1556 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:22.998950  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1191) took 1054 ms. Trace:
I20260709 13:01:22.999197  1556 rpcz_store.cc:276] 0709 13:01:21.944059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.997991 (+1053932us) service_pool.cc:224] Handling call
0709 13:01:22.998245 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.998746 (+   501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.998901 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.000087  1633 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:22.981081  3805 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 13:01:23.009411  2293 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:23.010524  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.011300  2293 raft_consensus.cc:2376] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 3. Re-sending same reply.
I20260709 13:01:23.012905  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.013648  2293 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:23.014127  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.971756  3697 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.984503  3810 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:23.020529  3810 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.021351  3553 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:23.022229  3802 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:01:22.962100  3725 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.022809  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.023413  1629 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:23.025005  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.023343  3666 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.026683  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:23.027303  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 75752.3us
I20260709 13:01:23.028569  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1184) took 1314 ms. Trace:
I20260709 13:01:23.028993  1560 rpcz_store.cc:276] 0709 13:01:21.714428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.990465 (+1276037us) service_pool.cc:224] Handling call
0709 13:01:22.990787 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.991322 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.991349 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.991920 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.017604 (+ 25684us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:23.017665 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.017760 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.028383 (+ 10623us) raft_consensus.cc:1685] finished
0709 13:01:23.028400 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.028513 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":82,"thread_start_us":25348,"threads_started":1,"wal-append.queue_time_us":25733}
I20260709 13:01:23.032841  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1167) took 2811 ms. Trace:
I20260709 13:01:23.033229  1554 rpcz_store.cc:276] 0709 13:01:20.222269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.273105 (+ 50836us) service_pool.cc:224] Handling call
0709 13:01:20.273354 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.638087 (+1364733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.032626 (+1394539us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540036bfd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.032785 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137085696}
I20260709 13:01:23.034438  3536 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:23.034837  3536 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:23.035347  3758 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.000754  3767 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.035717  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.035610  3536 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.036203  3767 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.036801  3753 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.037046  3536 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.036984  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:22.966997  3745 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:22.891706  2288 rpcz_store.cc:276] 0709 13:01:20.836749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.802513 (+965764us) service_pool.cc:224] Handling call
0709 13:01:21.809520 (+  7007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.824093 (+ 14573us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:22.026422 (+202329us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.209610 (+183188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.275175 (+ 65565us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.287526 (+ 12351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.302225 (+ 14699us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.320698 (+ 18473us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":265277,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":13201,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":74788,"thread_start_us":282390,"threads_started":2,"wal-append.queue_time_us":853763}
I20260709 13:01:23.041961  3551 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.043150  3793 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:22.984962  3663 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:01:23.052836  3750 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:23.047199  3807 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:01:23.059579  3725 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.047358  2289 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.060473  3626 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.060248  3448 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.060966  3745 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.064419  2290 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:23.066834  3673 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:23.068591  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.072053  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.078121  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.078936  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.081095  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:23.086496  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.086113  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:23.079816  3793 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.026640  3685 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.061484  3810 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.049001  3730 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:01:23.089862  3782 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.108409  3668 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:23.089274  3678 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:23.102869  3822 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:23.088822  3666 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.090518  3806 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:01:23.097826  3673 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.115116  3668 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.095492  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.113965  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 459) took 3578 ms. Trace:
I20260709 13:01:23.085440  3814 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:01:23.125401  3668 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:23.138509  3763 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.783s	user 0.006s	sys 0.026s
I20260709 13:01:23.131349  2288 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:23.142745  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.143366  2288 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:23.143581  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:23.146512  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 746) took 1054 ms. Trace:
I20260709 13:01:23.146863  2291 rpcz_store.cc:276] 0709 13:01:22.091641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.095968 (+  4327us) service_pool.cc:224] Handling call
0709 13:01:23.145952 (+1049984us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:23.146109 (+   157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.146264 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.146282 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.146315 (+    33us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:23.146345 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.146385 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.146466 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26482432}
I20260709 13:01:23.148438  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1151) took 3313 ms. Trace:
I20260709 13:01:23.150979  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1195) took 1091 ms. Trace:
I20260709 13:01:23.154138  3810 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.148856  1702 rpcz_store.cc:276] 0709 13:01:19.834637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.834797 (+   160us) service_pool.cc:224] Handling call
0709 13:01:19.835235 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.208638 (+373403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.208682 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.208737 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.208781 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.148213 (+2939432us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b540016ba54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.148278 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.148382 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858217216}
I20260709 13:01:23.154183  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:23.154650  3810 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:23.161986  3810 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.164182  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.166587  2291 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:23.162106  3536 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:23.167245  3536 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:23.167614  3536 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:23.171301  3810 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.173625  3810 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.175956  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1934) took 2322 ms. Trace:
I20260709 13:01:23.177000  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 466) took 1069 ms. Trace:
I20260709 13:01:23.177671  2291 rpcz_store.cc:276] 0709 13:01:22.107810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.165938 (+1058128us) service_pool.cc:224] Handling call
0709 13:01:23.166231 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.176613 (+ 10382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.176923 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.175546  3536 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.157209  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.188442  2288 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:23.190874  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:23.193117  3810 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:23.193672  3810 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:23.198558  3810 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.198706  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 468) took 1090 ms. Trace:
I20260709 13:01:23.199182  2291 rpcz_store.cc:276] 0709 13:01:22.108343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.179360 (+1071017us) service_pool.cc:224] Handling call
0709 13:01:23.195024 (+ 15664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.195305 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.196772 (+  1467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.198071 (+  1299us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:23.198101 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.198299 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.198643 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.199326  3484 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.197111  3191 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.200500  3484 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.201094  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:23.202332  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:23.201403  2291 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:23.176606  1701 rpcz_store.cc:276] 0709 13:01:20.853467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.860339 (+  6872us) service_pool.cc:224] Handling call
0709 13:01:20.868979 (+  8640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.869363 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.869397 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.869933 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.104402 (+234469us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.104492 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.107602 (+  3110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.137752 (+2030150us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:23.141243 (+  3491us) raft_consensus.cc:1685] finished
0709 13:01:23.141271 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.175841 (+ 34570us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1937,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":4816,"spinlock_wait_cycles":25089920,"thread_start_us":142938,"threads_started":1,"wal-append.queue_time_us":403647}
I20260709 13:01:23.212980  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.213651  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:23.214748  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.215780  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.216428  1702 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:23.217090  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.218030  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.218536  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:23.220234  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.220873  2294 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:23.221305  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 745) took 1130 ms. Trace:
I20260709 13:01:23.221601  2294 rpcz_store.cc:276] 0709 13:01:22.091251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.091372 (+   121us) service_pool.cc:224] Handling call
0709 13:01:23.220068 (+1128696us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:23.221255 (+  1187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5401472}
I20260709 13:01:23.223054  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.223711  1706 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:23.224470  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:23.224767  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:23.225226  1555 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:23.225404  1708 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.228091  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.228711  1704 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 2.
I20260709 13:01:23.229630  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.230268  3668 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.231101  1707 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:23.202124  3191 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.233117  3729 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:23.233515  3729 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:23.234253  3729 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.235291  3729 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.235895  2294 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:23.237564  2294 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.130895  2290 rpcz_store.cc:276] 0709 13:01:19.532821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.535301 (+  2480us) service_pool.cc:224] Handling call
0709 13:01:23.039111 (+3503810us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:23.048035 (+  8924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.079628 (+ 31593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.111611 (+ 31983us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5750016}
I20260709 13:01:23.240520  3817 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:23.242242  3815 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:23.243121  3758 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.243458  3758 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:23.244258  3758 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.249064  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.255152  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:23.257042  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.257907  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.228576  3782 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:23.256052  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.266719  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.267002  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.267355  1635 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.268162  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.269904  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.271348  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.271606  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.272352  1627 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:23.272986  1555 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:23.273576  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.273072  1708 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:23.274201  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:23.274880  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.275605  1626 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:23.276000  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.277621  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1916) took 3259 ms. Trace:
I20260709 13:01:23.277804  3663 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:23.278018  1700 rpcz_store.cc:276] 0709 13:01:20.017942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.018162 (+   220us) service_pool.cc:224] Handling call
0709 13:01:20.032746 (+ 14584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.244238 (+211492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.277389 (+3033151us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400167954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.277563 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408826752}
I20260709 13:01:23.278365  3663 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.279950  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:23.280834  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.281848  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.283025  1635 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:23.284145  1627 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:23.285410  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.286460  1628 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:23.288697  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.290113  2293 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:23.287882  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.292625  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 751) took 1186 ms. Trace:
I20260709 13:01:23.290544  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:23.258596  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2363658350000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 13:01:23.291693  1626 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.293628  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.296981  1630 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:23.293174  2293 rpcz_store.cc:276] 0709 13:01:22.106177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.012582 (+906405us) service_pool.cc:224] Handling call
0709 13:01:23.292538 (+279956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.298514  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.300210  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:23.313390  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64616.4us
I20260709 13:01:23.293792  1635 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:23.306859  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.297057  1627 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:23.306696  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.346221  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:23.641696  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 293338us
I20260709 13:01:23.299584  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:23.645749  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:23.603996  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2092) took 2688 ms. Trace:
I20260709 13:01:23.647989  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:23.602644  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.648326  1633 rpcz_store.cc:276] 0709 13:01:20.915500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.915652 (+   152us) service_pool.cc:224] Handling call
0709 13:01:20.928941 (+ 13289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.000559 (+2071618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.603317 (+602758us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54001e8054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.603699 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4762418048}
I20260709 13:01:23.650914  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.571424  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1181) took 1904 ms. Trace:
I20260709 13:01:23.608465  1634 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.652494  1552 rpcz_store.cc:276] 0709 13:01:21.666849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.598909 (+932060us) service_pool.cc:224] Handling call
0709 13:01:22.710730 (+111821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.941590 (+230860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.570706 (+629116us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.571187 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514317056}
I20260709 13:01:23.343016  1630 raft_consensus.cc:2376] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 4. Re-sending same reply.
I20260709 13:01:23.655530  1630 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:23.655107  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:23.547731  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.065639   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:23.613132  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.608217  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.609288  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.651602  1633 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:23.590584  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1152) took 3917 ms. Trace:
I20260709 13:01:23.666188  1553 rpcz_store.cc:276] 0709 13:01:19.673170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.588776 (+3915606us) service_pool.cc:224] Handling call
0709 13:01:23.589422 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.589902 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.589942 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.590025 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:23.590089 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.590165 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.590331 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.549852  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1894) took 2671 ms. Trace:
I20260709 13:01:23.553967  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 230) took 4010 ms. Trace:
I20260709 13:01:23.562924  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1170) took 1916 ms. Trace:
I20260709 13:01:23.593787  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.592718  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.594828  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.651453  2263 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:23.598448  1523 connection.cc:256] RPC call timeout handler was delayed by -1.5727924530000337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33130000000000004s for a response
W20260709 13:01:23.678953  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.151513  1560 rpcz_store.cc:276] 0709 13:01:22.059302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.050712 (+991410us) service_pool.cc:224] Handling call
0709 13:01:23.058456 (+  7744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.058595 (+   139us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:23.125710 (+ 67115us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.150559 (+ 24849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.150631 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:23.150689 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.150774 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.150908 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83328,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":1828,"thread_start_us":73104,"threads_started":2}
I20260709 13:01:23.574730  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.426769  3607 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.671672  1558 rpcz_store.cc:276] 0709 13:01:21.645975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.646132 (+   157us) service_pool.cc:224] Handling call
0709 13:01:23.562359 (+1916227us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:23.562695 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55451904}
I20260709 13:01:23.744903  3678 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:23.747572  3583 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:23.748672  3583 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.752272  3697 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.753504  3697 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.441399  2295 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.669516  1559 rpcz_store.cc:276] 0709 13:01:20.878467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.944141 (+ 65674us) service_pool.cc:224] Handling call
0709 13:01:20.944767 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.945108 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:20.982223 (+ 37115us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.280684 (+2298461us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b18003c8680. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:01:23.548570 (+267886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.548740 (+   170us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:23.548902 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.549066 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.549476 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36216,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"spinlock_wait_cycles":6598528,"thread_start_us":267302,"threads_started":1,"wal-append.queue_time_us":2352154}
I20260709 13:01:23.758827  3699 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.336476  3735 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.764232  3729 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:23.764824  3735 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.763424  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.768199  3745 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.769381  1558 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.769148  3745 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.771481  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1935) took 2898 ms. Trace:
I20260709 13:01:23.771978  1703 rpcz_store.cc:276] 0709 13:01:20.872434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.872572 (+   138us) service_pool.cc:224] Handling call
0709 13:01:20.872819 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.485743 (+612924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.770933 (+2285190us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54003dc554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:23.771245 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407912192}
W20260709 13:01:23.437861  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 465) took 3292 ms (client timeout 2173 ms). Trace:
W20260709 13:01:23.774399  2292 rpcz_store.cc:269] 0709 13:01:20.145035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.284880 (+139845us) service_pool.cc:224] Handling call
0709 13:01:23.437566 (+3152686us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400644bd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:23.437748 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3770549504}
I20260709 13:01:23.343830  3550 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.777122  3678 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:23.776821  3550 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.779652  3568 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.780666  3568 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.784041  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:23.784938  1629 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:23.785807  3666 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:23.786409  3666 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.785597  1458 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:23.786859  3666 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:23.393460  3668 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:23.412856  3777 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.789470  3777 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.790284  3777 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.790911  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.432122  3767 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.490566  3758 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:23.793730  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 326) took 11404 ms (client timeout 15000 ms). Trace:
I20260709 13:01:23.288980  3527 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:23.796727  3527 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.797240  3527 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.796531  1458 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:23.797982  1629 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:23.293654  3691 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:23.800173  3607 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:23.776835  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 122210us
I20260709 13:01:23.814256  1559 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:01:23.431299  3769 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 2.071s	user 0.014s	sys 0.026s
I20260709 13:01:23.502381  3818 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:23.422741  3744 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.559s	user 0.013s	sys 0.035s
W20260709 13:01:23.441237  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 736) took 3360 ms (client timeout 2215 ms). Trace:
I20260709 13:01:23.672318  1551 rpcz_store.cc:276] 0709 13:01:19.543356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.544816 (+  1460us) service_pool.cc:224] Handling call
0709 13:01:19.631908 (+ 87092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.708571 (+ 76663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.553332 (+3844761us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b54002f9750. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:23.553685 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20825856}
W20260709 13:01:23.819681  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.343336  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.304950  3191 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:23.799223  1458 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 3, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 3 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:23.805709  2288 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.424609  3311 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:23.827559  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.327694  3812 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.504285  3780 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.359097  3714 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:23.299495  3669 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.836514  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.837630  2293 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:23.838258  3338 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:23.800208  3691 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.836549  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:23.407366  2291 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:23.427487  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:23.429725  3724 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.420863  3663 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.491882  3529 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:23.839785  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.422122  3537 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.359983  3826 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.336640  3800 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.336553  3551 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.418797  3646 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.406457  3536 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:23.890640  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1198) took 1443 ms. Trace:
I20260709 13:01:23.358929  3717 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.528s
I20260709 13:01:23.896173  1552 rpcz_store.cc:276] 0709 13:01:22.447240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.654084 (+1206844us) service_pool.cc:224] Handling call
0709 13:01:23.654611 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.655153 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:23.806139 (+150986us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.890278 (+ 84139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.890335 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:23.890381 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.890454 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.890582 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1768,"thread_start_us":216983,"threads_started":2}
I20260709 13:01:23.298705  3786 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.593s
I20260709 13:01:23.877395  3735 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.823048  3699 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.884516  3550 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:23.874459  3745 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:23.359246  3364 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.542s
I20260709 13:01:23.885874  3691 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:23.359561  3715 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.543s
I20260709 13:01:23.922982  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 471) took 3076 ms. Trace:
I20260709 13:01:23.918390  1632 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:23.427309  3810 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:23.794262  1455 rpcz_store.cc:269] 0709 13:01:12.388864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.389024 (+   160us) service_pool.cc:224] Handling call
0709 13:01:23.793491 (+11404467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:20.281542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:20.282338 (+   796us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:20.282382 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:01:20.282409 (+    27us) tablet.cc:662] Decoding operations
0709 13:01:20.283578 (+  1169us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:20.283661 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:20.283701 (+    40us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:01:20.306538 (+ 22837us) tablet.cc:701] Row locks acquired
0709 13:01:20.306556 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:01:20.306701 (+   145us) write_op.cc:270] Start()
0709 13:01:20.306849 (+   148us) write_op.cc:276] Timestamp: P: 1783602080306655 usec, L: 0
0709 13:01:20.306871 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:01:20.593307 (+286436us) log.cc:844] Serialized 3238 byte log entry
0709 13:01:21.401316 (+808009us) op_driver.cc:464] REPLICATION: finished
0709 13:01:21.765453 (+364137us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9dfe28 00007f70d241a581 00007f70d2419d58 00007f70d241e9f1 00007f70d241e9a9 00007f70d241e939 00007f70d241e901 00007f70d241db2d 000000000038f354 000000000038f199
0709 13:01:21.794019 (+ 28566us) write_op.cc:301] APPLY: starting
0709 13:01:21.794103 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:21.832375 (+ 38272us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:21.833499 (+  1124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:21.872189 (+ 38690us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:21.872438 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:01:21.872480 (+    42us) write_op.cc:312] APPLY: finished
0709 13:01:21.872757 (+   277us) log.cc:844] Serialized 65 byte log entry
0709 13:01:21.872943 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:21.873056 (+   113us) write_op.cc:454] Released schema lock
0709 13:01:21.873101 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:21.873479 (+   378us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:21.873514 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":392419,"apply.run_cpu_time_us":3905,"apply.run_wall_time_us":79439,"num_ops":8,"prepare.queue_time_us":315286,"prepare.run_cpu_time_us":89146,"prepare.run_wall_time_us":1484131,"raft.queue_time_us":408604,"raft.run_cpu_time_us":52709,"raft.run_wall_time_us":262718,"replication_time_us":1089237,"spinlock_wait_cycles":16614144,"thread_start_us":1052434,"threads_started":4,"wal-append.queue_time_us":327578,"wal-append.run_cpu_time_us":12685,"wal-append.run_wall_time_us":1940150}]]}
I20260709 13:01:23.926898  3669 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.927165  3551 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:23.929216  3646 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:23.931910  3720 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.932507  3311 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:23.933283  3724 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:23.933775  2290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.008s	sys 0.012s
W20260709 13:01:24.024441  2290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.008s	sys 0.012s
W20260709 13:01:24.024605  1673 connection.cc:256] RPC call timeout handler was delayed by -1.4601547660000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:01:24.025148  1673 connection.cc:256] RPC call timeout handler was delayed by -1.3750940850000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24660000000000004s for a response
W20260709 13:01:24.025625  1673 connection.cc:256] RPC call timeout handler was delayed by -1.1365522749999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:01:23.934773  3661 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:24.040004  3568 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:23.936167  2296 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:24.041049  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 748) took 1948 ms (client timeout 2060 ms). Trace:
I20260709 13:01:24.041857  3626 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:23.935784  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 99423.9us
W20260709 13:01:23.936818  1524 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.215s (SENT)
I20260709 13:01:23.936510  3826 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.946123  3780 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:23.946565  3529 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.948170  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:23.923238  3818 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:24.071025  3846 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.926892  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1196) took 1761 ms. Trace:
I20260709 13:01:23.952538  3583 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.071914  3529 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:23.954752  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:24.072009  3846 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.072877  2291 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:24.074081  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:24.074608  3846 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.074829  2291 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:24.075929  3657 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:24.076390  3657 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:24.071815  3818 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:23.961728  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.077162  3826 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.077575  3818 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.077510  3657 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.078230  3780 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:24.079416  3657 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:23.925354  2297 rpcz_store.cc:269] 0709 13:01:20.080692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.277781 (+197089us) service_pool.cc:224] Handling call
0709 13:01:23.440976 (+3163195us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54005e1854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:23.441149 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241381120}
I20260709 13:01:23.921528  3765 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.143s
I20260709 13:01:23.927429  3712 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:23.923544  2295 rpcz_store.cc:276] 0709 13:01:20.837049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.817753 (+980704us) service_pool.cc:224] Handling call
0709 13:01:23.441124 (+1623371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.756471 (+315347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.901027 (+144556us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54005e1850. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:23.913589 (+ 12562us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3732390656}
I20260709 13:01:23.997494  1458 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 4, term changed from 2 to 4, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:24.000520  3699 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.001305  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1194) took 1942 ms. Trace:
I20260709 13:01:24.008239  3777 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.010224  1634 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:01:23.924943  3697 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.021165  3646 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.039752  3661 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.113281  1634 pending_rounds.cc:85] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 6
W20260709 13:01:24.041580  2296 rpcz_store.cc:269] 0709 13:01:22.092334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.092444 (+   110us) service_pool.cc:224] Handling call
0709 13:01:23.427294 (+1334850us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:24.040919 (+613625us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5565184}
I20260709 13:01:23.983351  3714 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:23.993057  3669 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.045058  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:23.927254  1627 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:24.079833  1557 rpcz_store.cc:276] 0709 13:01:22.063029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.580828 (+1517799us) service_pool.cc:224] Handling call
0709 13:01:23.581509 (+   681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.582053 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:23.582787 (+   734us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.813694 (+230907us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.813747 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:23.824161 (+ 10414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.824268 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.824430 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114162,"prepare.queue_time_us":87424,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":26140,"raft.queue_time_us":75116,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":203,"spinlock_wait_cycles":9161600,"thread_start_us":101404,"threads_started":1,"wal-append.queue_time_us":238111}
I20260709 13:01:24.111539  1556 rpcz_store.cc:276] 0709 13:01:22.059061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.014629 (+955568us) service_pool.cc:224] Handling call
0709 13:01:23.014947 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.015304 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.015336 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.020229 (+  4893us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.578325 (+558096us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b180002d4c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:23.820184 (+241859us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:23.820242 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.840982 (+ 20740us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.965354 (+124372us) raft_consensus.cc:1685] finished
0709 13:01:23.965377 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.001071 (+ 35694us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10628,"spinlock_wait_cycles":16663296,"thread_start_us":80863,"threads_started":1,"wal-append.queue_time_us":813393}
I20260709 13:01:24.082526  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:24.083791  2289 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.007s
I20260709 13:01:24.084113  1706 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:24.084709  3750 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:24.085577  1705 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:24.086519  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.087023  3720 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:24.087582  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 207738us
W20260709 13:01:24.088706  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49245.1us
I20260709 13:01:24.091435  1702 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:23.952119  3800 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.071329  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.095395  3812 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.156235  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.156630  3666 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.157164  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.158447  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.158962  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.159893  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.160640  1702 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
W20260709 13:01:24.096128  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.162187  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.096860  1708 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:24.164857  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.079536  3780 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:23.918744  3692 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:24.101032  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.178413  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.101547  3191 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:24.191368  3191 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.024219  3724 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:24.189867  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 253281us
I20260709 13:01:24.196174  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:24.197014  3720 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.197227  3777 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:24.197882  3777 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.198607  3812 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.197077  3697 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.103680  3846 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:24.107967  1525 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.547s (SENT)
I20260709 13:01:24.024187  3551 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.120093  2293 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
W20260709 13:01:23.938130  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 287029us
I20260709 13:01:24.081758  3364 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.140620  1709 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:24.140930  2297 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:24.150091  1629 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:24.150432  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.150755  3717 consensus_queue.cc:1243] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:01:24.151896  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 307816us
I20260709 13:01:24.151713  3712 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.121089  3529 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:24.165781  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.129418  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:24.130652  3661 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:24.137650  1634 pending_rounds.cc:107] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:24.235617  1634 raft_consensus.cc:2889] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.160636  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.189867  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:24.138695  3750 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:24.206941  3778 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.207484  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.212234  3810 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.203883  3765 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.232481  3311 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.175236  3666 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.211293  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:24.238543  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.329250  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 286) took 2260 ms. Trace:
W20260709 13:01:24.340632  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 273900us
I20260709 13:01:24.243037  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.242575  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:24.340341  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.281062  1626 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
W20260709 13:01:24.276070  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 86326.6us
I20260709 13:01:24.330742  2296 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:01:24.167029  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
W20260709 13:01:24.396356  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 288402us
I20260709 13:01:24.411685  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.411948  3777 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:24.412982  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 216966us
I20260709 13:01:24.413434  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.414014  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.413619  3750 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.414641  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.415118  3750 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:24.416276  3607 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.419946  3778 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.421587  2292 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.420387  3692 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:24.422712  3778 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:24.424939  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.757s (SENT)
W20260709 13:01:24.428530  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.428606  3364 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:24.429494  3364 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.429963  3714 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.422712  3692 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.433847  3810 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.433919  3550 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.434564  3550 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:24.233917  3691 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.437122  3691 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.220703  3846 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.444455  3846 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.412094  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.446408  2293 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 7. Ops: []
W20260709 13:01:24.447546  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.278743  1458 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:24.331017  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:24.400496  2296 pending_rounds.cc:85] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 5
I20260709 13:01:24.446412  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.454846  2296 pending_rounds.cc:107] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:01:24.455288  2296 raft_consensus.cc:2889] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.278785  3821 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.393303  1634 rpcz_store.cc:276] 0709 13:01:22.062737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.062853 (+   116us) service_pool.cc:224] Handling call
0709 13:01:22.063242 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.586046 (+1522804us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b58001871d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9400aa 00007f70ce9c2447 00007f70ce9dafe7 00007f70ce9dc7de 00007f70ce9e6b49 00007f70ce9e7416 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4
0709 13:01:24.243974 (+657928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.323396 (+ 79422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3342937472}
I20260709 13:01:24.278472  3758 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:24.459414  2297 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:24.381021  3800 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.330499  3699 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:24.339568  3568 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.340616  3851 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:24.477473  1628 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:24.466336  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:24.452395  3527 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.654s	user 0.015s	sys 0.003s
I20260709 13:01:24.469436  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.381712  3529 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:24.456933  2288 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.009s
I20260709 13:01:24.453840  3364 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:24.478091  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 243148us
I20260709 13:01:24.447063  1701 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:24.470805  2289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 480) took 1535 ms (client timeout 1882 ms). Trace:
I20260709 13:01:24.467069  3758 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.461390  3311 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:24.480480  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:24.434522  3810 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.275519  3687 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.230s
I20260709 13:01:24.475085  3699 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:24.498340  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 471) took 1239 ms. Trace:
I20260709 13:01:24.503914  2296 rpcz_store.cc:276] 0709 13:01:23.252635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.200995 (+948360us) service_pool.cc:224] Handling call
0709 13:01:24.330082 (+129087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.487716 (+157634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.491550 (+  3834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.468149  3691 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:24.504879  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122992us
W20260709 13:01:24.496181  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43884.2us
I20260709 13:01:24.508140  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.477015  3851 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:24.515267  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 58579.1us
I20260709 13:01:24.497527  1458 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 4, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.496343  3758 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.513057  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.489827  2291 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:24.490051  2288 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:24.459123  3550 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:24.527935  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.531068  3663 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.532264  3669 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.550266  3745 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.551172  2293 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.490237  3568 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.536854  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 486) took 1492 ms. Trace:
I20260709 13:01:24.483649  3765 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.520540  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:24.506848  3699 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.526952  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.523012  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:24.481104  3777 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:24.546837  3778 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.535597  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.553205  3745 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:24.677022  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 147876us
I20260709 13:01:24.471957  3821 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.556217  2296 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:24.557293  3767 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:24.518252  3851 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:24.493168  2289 rpcz_store.cc:269] 0709 13:01:22.935881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.036750 (+100869us) service_pool.cc:224] Handling call
0709 13:01:24.468326 (+1431576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.611948  3663 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.552513  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 490) took 1365 ms. Trace:
I20260709 13:01:24.613308  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:24.489360  3527 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:24.613649  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54804.5us
I20260709 13:01:24.616557  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:24.624259  1700 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:24.655966  3646 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:24.660178  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 153736us
W20260709 13:01:24.661854  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161644us
I20260709 13:01:24.540159  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.683149  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:24.668040  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.661553  3765 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:24.689039  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.677066  3657 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.001s
I20260709 13:01:24.720930  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:24.722172  1703 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:24.727254  3583 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.730000  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67383.1us
W20260709 13:01:24.734074  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51046.7us
I20260709 13:01:24.734679  3529 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:24.734884  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:24.735308  3529 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:24.736126  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.688583  3661 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.716987  2288 rpcz_store.cc:276] 0709 13:01:23.184235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.189414 (+  5179us) service_pool.cc:224] Handling call
0709 13:01:23.790445 (+601031us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:24.549050 (+758605us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6235520}
I20260709 13:01:24.669055  3818 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:01:24.739255  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1890) took 1442 ms (client timeout 1885 ms). Trace:
I20260709 13:01:24.748896  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 496) took 1101 ms. Trace:
I20260709 13:01:24.739748  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.740048  3666 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:24.755970  3666 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.740682  3290 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:24.756400  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 284) took 2694 ms (client timeout 1568 ms). Trace:
W20260709 13:01:24.757081  1632 rpcz_store.cc:269] 0709 13:01:22.062056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.062183 (+   127us) service_pool.cc:224] Handling call
0709 13:01:24.743173 (+2680990us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54002986d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:24.756276 (+ 13103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3912515456}
I20260709 13:01:24.757529  3529 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.758810  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.758901  3758 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.742937  2296 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:24.744359  1705 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:01:24.749429  2297 rpcz_store.cc:276] 0709 13:01:23.647114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.221424 (+574310us) service_pool.cc:224] Handling call
0709 13:01:24.401817 (+180393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.468688 (+ 66871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.742818 (+274130us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b54002c8dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:24.748832 (+  6014us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111250688}
I20260709 13:01:24.751230  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.752954  3692 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:24.662001  3800 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:01:24.759830  1702 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:24.741444  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 469) took 1489 ms. Trace:
W20260709 13:01:24.760133  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 252854us
I20260709 13:01:24.773433  3715 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:24.748121  1626 rpcz_store.cc:269] 0709 13:01:23.297077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.592016 (+294939us) service_pool.cc:224] Handling call
0709 13:01:24.739019 (+1147003us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54001428d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:24.739145 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229671680}
I20260709 13:01:24.718055  3810 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.734906  2291 raft_consensus.cc:2410] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:24.767297  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.770033  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.731268  3691 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:24.756397  3666 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:24.772365  3493 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:24.773468  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.775264  2292 rpcz_store.cc:276] 0709 13:01:23.252133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.076470 (+824337us) service_pool.cc:224] Handling call
0709 13:01:24.421323 (+344853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.422002 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.741212 (+319210us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400729050. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:24.741378 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762703104}
W20260709 13:01:24.777930  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.772097  3848 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 13:01:24.784071  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 371973us
I20260709 13:01:24.755136  3851 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.785562  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.762519  3568 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.809612  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.751372  3818 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:24.729804  3583 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.818511  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:24.824882  1554 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.041s	sys 0.069s
W20260709 13:01:24.825311  1554 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.042s	sys 0.070s
I20260709 13:01:24.817211  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.827814  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.823390  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.831465  1706 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:24.832516  3717 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.773968  3778 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.790949  3493 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.808827  3777 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:24.845163  3484 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.846002  3484 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:24.817689  3818 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:24.850057  1707 raft_consensus.cc:2410] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:24.852731  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.850412  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1197) took 2756 ms. Trace:
I20260709 13:01:24.824719  3692 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:24.855559  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93842.7us
I20260709 13:01:24.856338  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:24.856144  3692 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.856563  2294 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:24.776350  3715 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:24.857774  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.833030  3657 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:24.861117  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.848517  3493 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:24.866343  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 85386.6us
I20260709 13:01:24.869537  3849 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.870465  3849 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.873539  3778 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.874001  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:24.874624  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.856415  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:24.791342  3290 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.876988  1703 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:24.886155  3717 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.888650  3717 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.895356  1700 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:24.892702  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:24.891327  3493 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.897544  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.899606  1702 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:24.926144  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:24.927495  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.893842  3657 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:24.880055  3568 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:24.857650  1554 rpcz_store.cc:276] 0709 13:01:22.085161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.595823 (+1510662us) service_pool.cc:224] Handling call
0709 13:01:23.596291 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.596808 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:23.681503 (+ 84695us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.831531 (+1150028us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.840464 (+  8933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.840538 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:24.840601 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.840710 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.841217 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388645,"prepare.queue_time_us":149289,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":35455,"raft.queue_time_us":42906,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":8024,"thread_start_us":1066165,"threads_started":3,"wal-append.queue_time_us":218705}
I20260709 13:01:24.863181  3810 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:01:24.952574  1700 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:24.931589  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 315128us
I20260709 13:01:24.848819  3745 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.959363  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:24.919224  3607 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.968961  3702 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.969722  3583 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.971005  3692 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.972543  3493 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:24.972160  3818 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:24.973057  3724 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.973722  3818 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:24.975986  3663 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.976502  3568 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:24.977528  3849 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.978534  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 491) took 1738 ms. Trace:
I20260709 13:01:24.979164  3851 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:24.981297  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 757) took 1213 ms. Trace:
I20260709 13:01:24.979019  1702 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:24.926416  3848 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:24.978668  3568 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:24.981781  2295 rpcz_store.cc:276] 0709 13:01:23.767695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.276536 (+508841us) service_pool.cc:224] Handling call
0709 13:01:24.277051 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.338039 (+ 60988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.338070 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.411412 (+ 73342us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.777445 (+366033us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:24.777498 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.777588 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.981027 (+203439us) raft_consensus.cc:1685] finished
0709 13:01:24.981057 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.981224 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3000,"raft.queue_time_us":60534,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":365613,"threads_started":1,"wal-append.queue_time_us":567637}
I20260709 13:01:24.929212  3717 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:01:25.016620  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 281812us
I20260709 13:01:24.559275  2290 rpcz_store.cc:276] 0709 13:01:23.044698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.239290 (+194592us) service_pool.cc:224] Handling call
0709 13:01:23.240194 (+   904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.241063 (+   869us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:23.300994 (+ 59931us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.470885 (+1169891us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.495935 (+ 25050us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.505308 (+  9373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.522435 (+ 17127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.536543 (+ 14108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.536770 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33704,"prepare.queue_time_us":479343,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":46872,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":7551,"thread_start_us":1126000,"threads_started":3,"wal-append.queue_time_us":662621}
I20260709 13:01:24.978993  2294 rpcz_store.cc:276] 0709 13:01:23.240532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.810727 (+570195us) service_pool.cc:224] Handling call
0709 13:01:23.912441 (+101714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.860277 (+947836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.978287 (+118010us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:24.978474 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988129152}
I20260709 13:01:25.020473  1634 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:24.899471  3811 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:01:25.037333  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.039029  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:24.929816  3290 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.042568  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.045328  3821 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:25.046862  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64220us
I20260709 13:01:24.971006  3484 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:25.035501  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.018350  3607 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.056519  1700 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:25.073999  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:25.022544  3848 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:25.078140  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 342491us
I20260709 13:01:25.060546  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.088385  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:25.087225  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 230856us
I20260709 13:01:25.079254  1632 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:25.093070  1629 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
W20260709 13:01:25.092255  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53336.1us
I20260709 13:01:25.088457  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.084309  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:25.096623  1702 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:25.036823  3851 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:25.097827  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:25.084897  2263 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.022s (SENT) [suppressed 25 similar messages]
I20260709 13:01:25.098045  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.079845  3849 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.102952  2295 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:25.105921  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:25.095283  3668 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.009s	sys 0.031s
W20260709 13:01:25.106588  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 385690us
I20260709 13:01:25.089059  3845 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:25.108407  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.104197  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.095752  3758 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.100703  3810 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.116012  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.036823  3493 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:01:25.116784  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1170) took 1011 ms. Trace:
W20260709 13:01:25.110607  3668 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.010s	sys 0.031s
I20260709 13:01:25.121165  2294 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:25.104960  3780 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:25.118881  1700 rpcz_store.cc:276] 0709 13:01:24.105323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.107252 (+1001929us) service_pool.cc:224] Handling call
0709 13:01:25.116719 (+  9467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.106945  3484 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:25.107335  3692 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.072952  3715 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:25.126403  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77402us
I20260709 13:01:25.048695  3191 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:01:25.128840  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.110776  3845 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:25.131259  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
W20260709 13:01:25.124009  3750 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.708s	user 0.015s	sys 0.000s
I20260709 13:01:25.134028  3777 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.136566  3777 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:25.137037  3777 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.137349  3290 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.137967  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.138228  1627 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:25.139457  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.140352  1633 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:25.115361  3583 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:25.132630  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.143455  3583 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.144093  3553 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:25.144495  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.145135  1634 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:25.145284  3856 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:25.145844  3583 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.147089  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.114198  3849 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.149334  3849 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:25.150131  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:25.150779  2296 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:25.145080  3702 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:25.151932  3692 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.154093  3692 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.154026  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.154809  1708 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:25.144400  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.156526  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.135715  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.157218  3777 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.157745  1700 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.158560  3583 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.158485  3856 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:25.152275  3702 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:25.168399  2295 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:25.162107  1626 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:25.118562  3527 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:25.127511  3311 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.004s
W20260709 13:01:25.162823  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74579.3us
W20260709 13:01:25.165294  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.155717  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:25.167438  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59044.3us
W20260709 13:01:25.171337  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.171783  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.170928  1525 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:25.172408  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.173338  1708 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:25.129756  3780 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:25.118968  3846 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:25.180847  1628 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:25.176097  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.115615  3778 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:25.167876  3849 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:25.186203  3867 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:25.131577  3845 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.186822  3867 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.187554  3845 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.189059  3867 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.190608  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:25.175047  3777 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.132027  3872 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:01:25.191814  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.192314  3872 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:25.192763  3872 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.192926  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.193686  3687 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:25.169750  3715 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.195027  3687 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:25.195432  3687 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.123834  3851 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.197252  3712 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.198151  3851 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.198833  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.198786  3851 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:25.126163  3607 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.180514  3529 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:25.202010  3529 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.125077  3657 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.203156  3364 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:25.194031  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.167564  2289 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:25.194088  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.167829  3800 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:25.199846  1709 raft_consensus.cc:2410] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.204210  3692 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.186223  3849 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.143577  3666 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.207789  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:25.210429  2293 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:25.213389  1703 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.211409  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:25.218885  1700 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:25.218865  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:25.222191  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59675us
W20260709 13:01:25.222330  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46322.8us
I20260709 13:01:25.223798  3529 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:25.224855  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.184088  3290 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:25.226432  3780 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.227242  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.228061  3702 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.229344  3702 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:25.225525  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:25.236589  3484 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.236860  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.237213  3484 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.237716  3484 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.209180  3800 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.218848  3849 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:25.219725  3867 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.252333  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44687.2us
W20260709 13:01:25.256999  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 87898.8us
W20260709 13:01:25.260414  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93661.7us
I20260709 13:01:25.251549  3493 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:25.262574  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.269940  2293 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:25.175197  3848 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.255138  3750 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.252070  3800 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:25.280916  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98226.7us
I20260709 13:01:25.277747  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.290233  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:25.258616  3692 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.226456  3290 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.293535  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.193686  3872 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.266309  3493 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.296761  1708 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:25.297992  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.302186  3868 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.302845  3868 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.305270  3767 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.305446  3657 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.305751  3767 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.306825  3767 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:25.308372  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1891) took 2011 ms (client timeout 2052 ms). Trace:
W20260709 13:01:25.308861  1635 rpcz_store.cc:269] 0709 13:01:23.297329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.594167 (+296838us) service_pool.cc:224] Handling call
0709 13:01:25.308142 (+1713975us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001e9454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:25.308306 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2798283008}
I20260709 13:01:25.311296  1632 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.290903  3750 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.312789  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1168) took 1208 ms. Trace:
I20260709 13:01:25.317548  1708 rpcz_store.cc:276] 0709 13:01:24.104405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.235247 (+1130842us) service_pool.cc:224] Handling call
0709 13:01:25.273911 (+ 38664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.312598 (+ 38687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.312745 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.318187  3750 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.318830  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.319598  1708 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:25.320091  3710 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:25.320969  3583 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:25.290719  3553 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:25.305328  3666 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.321441  3661 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.322834  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.323354  3290 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.324432  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.326020  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:25.293316  3856 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:25.327201  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:25.334477  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 45679.7us
I20260709 13:01:25.336294  3697 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:25.352376  3697 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:25.337013  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:25.353204  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:25.353677  3699 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.001s
I20260709 13:01:25.354125  3699 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:25.354415  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.354516  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.263157  3867 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:25.340790  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83789.6us
I20260709 13:01:25.356043  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:25.357048  3870 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:25.357622  3870 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:25.358228  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:25.362689  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.364135  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:25.368530  3810 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:25.369308  3810 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.311544  3493 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.370666  3810 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.346903  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.374596  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.340502  3872 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:25.380851  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.375300  1706 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:25.347788  3553 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:25.368732  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:25.331014  3767 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:25.342962  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 65595.5us
I20260709 13:01:25.377782  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:25.375275  1557 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:25.383325  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.385869  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:25.358752  3870 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.419456  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.422307  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:25.425895  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.429347  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.430020  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.431479  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.432410  1707 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:25.433501  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57952.9us
I20260709 13:01:25.380842  3856 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:25.386771  3290 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:25.437359  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:25.383720  3867 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:25.419919  3191 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:25.418109  3810 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.383256  2291 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:25.428804  3846 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:25.383930  3868 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.383919  3553 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:25.444463  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 62961.2us
I20260709 13:01:25.444590  1554 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:25.441521  3692 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:25.496174  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61674.7us
I20260709 13:01:25.445438  3484 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.450541  3550 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.451133  1703 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.498534  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1169) took 1393 ms. Trace:
W20260709 13:01:25.452432  3879 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:01:25.525028  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 192584us
I20260709 13:01:25.435060  3715 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:01:25.439003  3290 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.527557  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1176) took 1419 ms. Trace:
W20260709 13:01:25.488658  3874 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:25.488930  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1185) took 1289 ms. Trace:
I20260709 13:01:25.489513  3750 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.494952  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1203) took 2330 ms. Trace:
I20260709 13:01:25.443446  3699 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.498886  1703 rpcz_store.cc:276] 0709 13:01:24.104775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.439085 (+1334310us) service_pool.cc:224] Handling call
0709 13:01:25.450959 (+ 11874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.498255 (+ 47296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.498469 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.450594  3710 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:25.501853  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1173) took 1395 ms (client timeout 1541 ms). Trace:
I20260709 13:01:25.503979  3657 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.508651  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.509187  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.541524  1556 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:25.510332  1552 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:25.510950  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.512122  1559 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:25.489516  3800 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:25.513039  1551 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:25.542824  1553 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:25.518380  3870 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.442744  1706 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:25.526427  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.528573  1557 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.487987  1555 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.530489  1708 rpcz_store.cc:276] 0709 13:01:24.107827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.439150 (+1331323us) service_pool.cc:224] Handling call
0709 13:01:25.439441 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.456873 (+ 17432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.491917 (+ 35044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:25.497719 (+  5802us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:25.527268 (+ 29549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.527344 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.527492 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.532119  1558 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.491892  3846 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.532457  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.435277  3851 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:01:25.536597  1560 rpcz_store.cc:276] 0709 13:01:23.164915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.803557 (+638642us) service_pool.cc:224] Handling call
0709 13:01:23.863213 (+ 59656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.341002 (+477789us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:24.341488 (+   486us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.731033 (+389545us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.731427 (+   394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.731484 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:24.736081 (+  4597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.380471 (+644390us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b18003260e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419
0709 13:01:25.494558 (+114087us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:25.494614 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.494771 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14970,"prepare.queue_time_us":2080,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":95,"raft.queue_time_us":54094,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":21436,"spinlock_wait_cycles":1102902656,"thread_start_us":379583,"threads_started":1,"wal-append.queue_time_us":473654}
W20260709 13:01:25.538326  1705 rpcz_store.cc:269] 0709 13:01:24.106655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.155875 (+ 49220us) service_pool.cc:224] Handling call
0709 13:01:25.501633 (+1345758us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:25.501769 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349814144}
I20260709 13:01:25.533365  3493 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:25.561849  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53291.4us
I20260709 13:01:25.516690  3484 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.531419  3290 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:25.458114  3849 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.605979  3750 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.578012  3846 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:25.656495  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 124099us
W20260709 13:01:25.652298  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.627373  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 103103us
I20260709 13:01:25.580298  1708 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:25.675273  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.676074  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1188) took 1433 ms. Trace:
W20260709 13:01:25.676251  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.615381  3699 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:25.581838  1554 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:25.572636  3724 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.672204  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.574975  1706 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.576871  1703 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:25.580082  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.612118  3856 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.673028  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:25.533661  1701 rpcz_store.cc:276] 0709 13:01:24.199740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.487717 (+1287977us) service_pool.cc:224] Handling call
0709 13:01:25.488279 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.488534 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.488566 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:25.488621 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:25.488659 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.488724 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.488863 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.676910  1708 rpcz_store.cc:276] 0709 13:01:24.242469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.570803 (+1328334us) service_pool.cc:224] Handling call
0709 13:01:25.571184 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.675522 (+104338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.675999 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.678321  1700 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 7. Ops: []
I20260709 13:01:25.678678  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.561019  3553 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.680400  1705 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:01:25.687927  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.690172  1702 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:01:25.692414  3884 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:01:25.696115  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:25.700973  3800 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:25.706461  1704 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:25.708142  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.716265  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1187) took 1510 ms. Trace:
I20260709 13:01:25.710180  1556 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:25.718071  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.737170  1708 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:25.720094  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 140088us
I20260709 13:01:25.730248  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1193) took 1301 ms. Trace:
I20260709 13:01:25.737977  3191 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:25.738394  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.738739  1558 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:25.739485  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.740124  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:25.709635  3750 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.740679  1634 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:25.731439  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58527.9us
I20260709 13:01:25.736223  1706 rpcz_store.cc:276] 0709 13:01:24.200464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.570559 (+1370095us) service_pool.cc:224] Handling call
0709 13:01:25.574780 (+  4221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.710604 (+135824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.710796 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.740304  1701 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:01:25.719106  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1192) took 1302 ms. Trace:
I20260709 13:01:25.740332  1553 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.738029  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.744742  1705 rpcz_store.cc:276] 0709 13:01:24.416975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.660270 (+1243295us) service_pool.cc:224] Handling call
0709 13:01:25.680167 (+ 19897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.718917 (+ 38750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.719065 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.714838  3724 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:25.744161  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.737802  1702 rpcz_store.cc:276] 0709 13:01:24.417270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.689424 (+1272154us) service_pool.cc:224] Handling call
0709 13:01:25.689801 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.718593 (+ 28792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.718776 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.715308  3849 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:25.743788  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47753.3us
I20260709 13:01:25.746981  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:25.748574  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:25.749666  3780 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:25.750334  3780 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.750732  3780 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:25.749598  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:25.741808  3865 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.572s	user 0.002s	sys 0.004s
I20260709 13:01:25.757673  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.744614  1701 pending_rounds.cc:85] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 2
I20260709 13:01:25.758162  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.728250  3699 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:25.759591  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51575us
I20260709 13:01:25.778657  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:25.744669  3767 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:25.779044  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.780032  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.746084  3484 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:25.746991  3553 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.751493  3527 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:25.786863  3710 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:25.754221  3724 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.804301  1554 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:25.784286  1556 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:25.764940  1701 pending_rounds.cc:107] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:01:25.784577  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.809319  1557 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:25.784893  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1163) took 1692 ms. Trace:
I20260709 13:01:25.787870  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.783390  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.790539  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:25.798692  3850 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.798314  3699 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.799456  3821 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:25.784287  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:25.816064  2296 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.740816  3750 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.806866  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1166) took 1711 ms. Trace:
I20260709 13:01:25.820914  1700 rpcz_store.cc:276] 0709 13:01:24.095345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.220800 (+1125455us) service_pool.cc:224] Handling call
0709 13:01:25.221110 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.717014 (+495904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.806543 (+ 89529us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b540010e0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:25.806771 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051024640}
W20260709 13:01:25.821426  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.804514  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1183) took 1648 ms (client timeout 2070 ms). Trace:
I20260709 13:01:25.806151  1633 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:25.818172  1634 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:25.822026  1704 rpcz_store.cc:269] 0709 13:01:24.166739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.446115 (+279376us) service_pool.cc:224] Handling call
0709 13:01:25.804334 (+1358219us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400042c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:25.804449 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581488000}
I20260709 13:01:25.782793  3767 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.823482  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.827911  1704 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:25.810141  1709 rpcz_store.cc:276] 0709 13:01:24.092844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.213498 (+1120654us) service_pool.cc:224] Handling call
0709 13:01:25.218056 (+  4558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.238426 (+ 20370us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:01:25.238462 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.274605 (+ 36143us) log.cc:844] Serialized 26 byte log entry
0709 13:01:25.342797 (+ 68192us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:25.342844 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.342893 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.784637 (+441744us) raft_consensus.cc:1685] finished
0709 13:01:25.784657 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.784805 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17472,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4500,"thread_start_us":34685,"threads_started":1,"wal-append.queue_time_us":200438}
I20260709 13:01:25.828408  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.806593  3712 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:25.835458  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:25.836272  1553 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:25.832715  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:25.815258  3850 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:25.815630  3821 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:25.809101  1701 raft_consensus.cc:2889] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.834147  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1201) took 1362 ms. Trace:
I20260709 13:01:25.818989  3849 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:25.841486  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1200) took 1364 ms. Trace:
W20260709 13:01:25.842439  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.833345  3657 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:25.850299  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:25.852169  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.819187  3527 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:01:25.859879  3890 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:01:25.826072  3810 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.828563  3767 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.861866  1704 rpcz_store.cc:276] 0709 13:01:24.466003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.823907 (+1357904us) service_pool.cc:224] Handling call
0709 13:01:25.824221 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.828472 (+  4251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.828664 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.907246  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.827972  3710 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:25.843824  3851 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.834755  3712 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.908166  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.926136  1634 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.909577  2296 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:25.879338  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.910912  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60972.5us
I20260709 13:01:25.866842  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:25.896478  1700 rpcz_store.cc:276] 0709 13:01:24.465706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.821828 (+1356122us) service_pool.cc:224] Handling call
0709 13:01:25.830088 (+  8260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.830270 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.830298 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:25.830337 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:25.830364 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.830419 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.830486 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.848436  3484 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.860354  3868 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:25.933174  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:25.925909  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.938968  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 289) took 1254 ms. Trace:
I20260709 13:01:25.939599  1704 rpcz_store.cc:276] 0709 13:01:24.684798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.907563 (+1222765us) service_pool.cc:224] Handling call
0709 13:01:25.931151 (+ 23588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.938436 (+  7285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.938486 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:25.938559 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:25.938610 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.938709 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.938888 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.942649  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:25.943827  1556 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:01:25.944157  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.917575  3873 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.914445  3849 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.914820  3895 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.918391  3669 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:25.917815  3493 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:25.930518  3712 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.960055  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:25.957930  3818 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:25.968684  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.947749  3851 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:25.937572  3868 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:25.962247  3849 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.927179  3657 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:25.990136  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:25.917397  3810 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:25.982544  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.013839  1675 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.565s (SENT)
I20260709 13:01:25.946698  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1196) took 1525 ms. Trace:
I20260709 13:01:25.996342  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:25.964215  3892 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:25.859488  3687 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:25.991976  1635 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
W20260709 13:01:26.007702  3872 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260709 13:01:25.984200  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.026377  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.021427  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:25.955624  3873 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:25.962186  3484 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.025713  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44858 (request call id 81) took 1043 ms. Trace:
I20260709 13:01:26.011437  3868 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.024888  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1177) took 1916 ms. Trace:
I20260709 13:01:26.032800  1555 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:26.034438  1627 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:26.027281  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.048334  1552 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.041059  1700 rpcz_store.cc:276] 0709 13:01:24.969176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.937391 (+968215us) service_pool.cc:224] Handling call
0709 13:01:26.012504 (+ 75113us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:26.050625  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.538s (SENT)
I20260709 13:01:26.053354  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1197) took 1634 ms. Trace:
I20260709 13:01:26.054603  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.055222  1635 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:26.053498  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.046074  1707 rpcz_store.cc:276] 0709 13:01:24.108235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.438404 (+1330169us) service_pool.cc:224] Handling call
0709 13:01:25.489960 (+ 51556us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:25.490516 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.490547 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.491231 (+   684us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.616105 (+124874us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:25.616167 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.616273 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.024566 (+408293us) raft_consensus.cc:1685] finished
0709 13:01:26.024600 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.024815 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":849,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":124397,"threads_started":1,"wal-append.queue_time_us":212554}
I20260709 13:01:26.063189  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:25.968489  3895 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.043735  3873 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.069188  1557 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.067461  1704 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:26.058373  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.071880  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:25.969657  3850 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.078733  3848 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:26.079406  3850 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.084153  3657 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.087689  3484 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.094414  3780 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.101112  3697 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.103197  3697 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.105329  3780 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.095796  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:26.109853  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.110502  1627 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:26.111200  3777 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:26.100263  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.111918  3777 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:26.112500  1630 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:26.099149  3724 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.113334  3777 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.095117  1553 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:26.115756  3724 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.119277  1629 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:26.121614  3777 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.122963  2291 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:26.124055  3851 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:26.128118  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.074831  1700 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:26.129289  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.076558  3191 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.058887  3872 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.021008  1701 rpcz_store.cc:276] 0709 13:01:24.418433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.739563 (+1321130us) service_pool.cc:224] Handling call
0709 13:01:25.739926 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.913911 (+173985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.937951 (+ 24040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.136854  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:26.120239  3687 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.024224  3527 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.136270  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:26.144264  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1954) took 1703 ms. Trace:
I20260709 13:01:26.149797  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.129359  3895 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.150763  1556 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.152341  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.153309  1553 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:26.161443  1631 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.142526  3818 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:26.165868  1555 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:26.170436  1708 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:26.172196  1632 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:26.177552  3712 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:26.178529  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.178997  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.180831  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 288) took 1496 ms. Trace:
I20260709 13:01:26.180873  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:26.182243  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.184134  1631 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:26.184808  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.181236  1709 rpcz_store.cc:276] 0709 13:01:24.684468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.828826 (+1144358us) service_pool.cc:224] Handling call
0709 13:01:25.829152 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:25.853410 (+ 24258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.853444 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.909144 (+ 55700us) log.cc:844] Serialized 52 byte log entry
0709 13:01:26.048798 (+139654us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:26.048872 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.048971 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.180614 (+131643us) raft_consensus.cc:1685] finished
0709 13:01:26.180633 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.180777 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14264,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":34605,"thread_start_us":102297,"threads_started":1,"wal-append.queue_time_us":247764}
I20260709 13:01:26.188122  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.188925  1709 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:26.157047  2291 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:26.141669  3191 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.143121  3872 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.190528  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.142851  3290 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.188603  3712 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.196666  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.199584  1704 raft_consensus.cc:2410] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:26.202179  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.205178  3872 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.206965  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.207929  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.208580  1701 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:01:26.209143  1626 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:26.208726  1704 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:26.210249  1701 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:26.212759  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.213529  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:26.213805  3872 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:26.214275  3872 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.214530  2294 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:26.207046  3290 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:26.215027  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.214592  3872 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:26.215975  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:26.216840  2291 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:26.223368  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.225108  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.224750  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.225757  3895 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:26.226197  3895 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.226404  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.224608  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.226574  3895 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.227104  1631 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:26.227519  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.227962  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.228663  1631 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:26.226636  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.229933  1632 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
W20260709 13:01:26.236325  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.237389  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.243499  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.251865  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:26.253348  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1955) took 1812 ms. Trace:
W20260709 13:01:26.254594  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67945.6us
I20260709 13:01:26.253964  1702 rpcz_store.cc:276] 0709 13:01:24.440872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.748265 (+1307393us) service_pool.cc:224] Handling call
0709 13:01:25.748564 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.789033 (+ 40469us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:25.910568 (+121535us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.042220 (+131652us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.050204 (+  7984us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.050437 (+   233us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:26.050672 (+   235us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.063824 (+ 13152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.063944 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.253066 (+189122us) raft_consensus.cc:1685] finished
0709 13:01:26.253095 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.253271 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37635,"prepare.queue_time_us":223781,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":2988,"raft.queue_time_us":22890,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":39227,"spinlock_wait_cycles":83700224,"thread_start_us":182282,"threads_started":2,"wal-append.queue_time_us":235878}
I20260709 13:01:26.260623  1555 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:26.253599  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.740s (SENT)
I20260709 13:01:26.264149  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.268816  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.269016  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.270263  1627 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:26.213569  3845 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:26.276382  1525 connection.cc:256] RPC call timeout handler was delayed by -1.7207774470000459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:01:26.278810  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.274343  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.273288  3810 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.284117  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:26.285436  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:26.295053  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.296545  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:26.297084  1630 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:26.297194  2292 raft_consensus.cc:2376] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3. Re-sending same reply.
W20260709 13:01:26.297705  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 500) took 1504 ms (client timeout 1882 ms). Trace:
W20260709 13:01:26.298025  2292 rpcz_store.cc:269] 0709 13:01:24.792943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.296137 (+1503194us) service_pool.cc:224] Handling call
0709 13:01:26.297634 (+  1497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.298943  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:26.299607  2291 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:26.302228  1632 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:26.305100  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.294642  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.315073  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.316951  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1186) took 2116 ms. Trace:
I20260709 13:01:26.317176  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.273990  3845 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:26.313701  3846 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.000s
I20260709 13:01:26.314175  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.314208  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.324079  1627 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:26.316390  1552 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:26.317328  1558 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:26.318111  1703 rpcz_store.cc:276] 0709 13:01:24.200139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.538339 (+1338200us) service_pool.cc:224] Handling call
0709 13:01:25.570278 (+ 31939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.711127 (+140849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.316690 (+605563us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b5400096d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:26.316886 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14683904}
I20260709 13:01:26.289121  3669 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:26.290314  3800 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:26.314936  1599 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:26.317866  1557 raft_consensus.cc:2376] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7. Re-sending same reply.
I20260709 13:01:26.325734  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.262539  3872 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.343488  1556 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:01:26.333462  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60997.4us
I20260709 13:01:26.352212  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.358999  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.299625  3191 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.320598  3810 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:26.362447  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.365504  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:26.366807  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44353.5us
W20260709 13:01:26.368017  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 73495.2us
I20260709 13:01:26.325158  3867 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:26.339673  3607 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:01:26.331679  3846 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.333679  3697 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:26.343672  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:26.333647  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.366084  3810 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.349810  3895 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.339236  3907 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:01:26.390111  3290 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:26.369535  3905 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:26.375401  1557 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: [4.6-6.7]
W20260709 13:01:26.371354  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 66303.6us
I20260709 13:01:26.375998  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.293601  3818 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.337921  3845 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.381727  3607 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:26.387851  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.387457  3867 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.384543  3810 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:26.389528  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:26.378944  3821 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.411564  1553 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:26.392447  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.392675  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.393172  1552 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:26.412652  1554 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.395242  1558 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.395457  1555 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:26.394675  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.370621  3780 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:26.391067  3290 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:01:26.393249  1551 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.378157  3872 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.415993  1555 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:26.391709  2291 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:26.416153  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.419746  1557 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.391126  3907 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:01:26.413480  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.413985  3780 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:26.426316  3780 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:26.428120  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.428401  3750 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.425068  3821 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.430382  3868 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.433724  3777 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:26.434551  3849 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.435391  3895 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.436755  3849 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:26.438565  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.439361  1552 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.425041  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:26.416685  3905 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:26.447799  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.450207  1707 raft_consensus.cc:2368] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:01:26.450564  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.419292  3697 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:26.421263  3845 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:26.453646  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.452378  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57780.1us
I20260709 13:01:26.455202  1554 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.451691  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:26.456790  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 96116.9us
W20260709 13:01:26.459213  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.461334  2296 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:26.465132  1560 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:26.460093  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:26.466166  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.473012  1553 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:26.473408  1552 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.467724  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.474592  1554 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:26.448551  3607 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.476298  1554 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:26.479665  1554 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:26.467339  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:26.454200  3724 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.460996  1551 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.486116  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:26.458822  3856 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:26.485685  1554 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:26.460878  3907 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:26.460387  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:26.492517  2295 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:26.494405  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:26.462001  2288 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:26.495812  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.463428  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:26.498943  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.465238  3712 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:26.490459  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.522019  1560 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.461846  3817 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:26.474062  3750 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:01:26.506160  3868 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.500402  3907 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:26.483824  3607 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:26.486199  3724 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.507969  2291 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:26.511603  1708 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.473438  3895 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:01:26.512478  3846 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.514669  3191 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:26.495793  3699 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:26.522115  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.530246  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.542052  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:26.054029  1706 rpcz_store.cc:276] 0709 13:01:24.418730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.741650 (+1322920us) service_pool.cc:224] Handling call
0709 13:01:25.753481 (+ 11831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.753765 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:25.902886 (+149121us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.052787 (+149901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.052869 (+    82us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:26.052956 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.053054 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.053247 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":248602,"prepare.run_wall_time_us":6551,"thread_start_us":229337,"threads_started":2}
I20260709 13:01:26.532680  1553 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:26.526909  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.543884  1551 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.537719  1708 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:26.521687  3712 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:26.539012  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47101.5us
I20260709 13:01:26.523957  3817 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:26.532364  3607 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.580979  3918 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:26.581539  3918 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:26.581950  3918 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:26.584753  3780 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.585628  3867 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.586012  2290 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.149017  1705 rpcz_store.cc:276] 0709 13:01:24.440579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.745986 (+1305407us) service_pool.cc:224] Handling call
0709 13:01:25.778392 (+ 32406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.820044 (+ 41652us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:26.008611 (+188567us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.143793 (+135182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.143855 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:26.143915 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.144006 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.144178 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24868,"prepare.queue_time_us":265189,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":104,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4594,"thread_start_us":231937,"threads_started":2}
I20260709 13:01:26.587168  3780 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:26.589555  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.591270  3767 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.005s
I20260709 13:01:26.591928  3850 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.594350  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.597219  3850 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.598340  3846 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:26.600200  2290 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:26.536469  3872 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.574498  3817 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.580708  1634 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:26.585412  3873 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.536897  3724 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:26.573763  3894 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.541s	user 0.003s	sys 0.000s
I20260709 13:01:26.574743  1557 raft_consensus.cc:2410] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.648536  2294 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:26.649395  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:26.596289  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:01:26.596676  3906 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.392s	user 0.003s	sys 0.000s
I20260709 13:01:26.614296  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.621264  1559 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:26.595587  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:26.638011  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.599524  3850 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:26.640161  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.568329  3712 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.644868  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:26.647261  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.540777  3191 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:26.658991  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.657441  1560 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:26.669672  1556 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:26.506494  3905 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.684403  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.684651  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.626597  3872 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.686221  2293 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:26.687312  2295 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:26.674827  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.657722  3821 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:01:26.676560  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.676774  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.692649  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.694144  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.695325  1552 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.696362  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.697688  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.650837  1558 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:26.678961  1555 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.698542  1552 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.680508  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.699321  1559 raft_consensus.cc:2376] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5. Re-sending same reply.
I20260709 13:01:26.681062  1557 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:26.699522  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.681352  2294 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:26.681382  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.700553  1552 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:26.700534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.641718  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.701890  1559 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:26.702121  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60526.9us
I20260709 13:01:26.702699  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.698951  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:26.666822  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 199535us
I20260709 13:01:26.704246  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.698974  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.704829  1558 raft_consensus.cc:2376] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5. Re-sending same reply.
I20260709 13:01:26.704893  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:26.705442  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.706480  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.706545  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.707038  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.707945  1553 raft_consensus.cc:2410] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.707811  1559 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
W20260709 13:01:26.708392  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.708753  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.622326  3767 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:26.702982  1442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 110814us
I20260709 13:01:26.709826  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.708945  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.682706  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:26.683070  1700 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:26.711578  1708 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:26.701948  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.718828  1552 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:01:26.719877  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.719683  1553 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:26.727355  1559 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.681664  3867 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.683471  3777 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:26.682274  3484 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:26.706948  3872 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.730316  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.675783  3687 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:01:26.731909  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.703919  3800 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.733492  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.735849  1558 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5. Re-sending same reply.
I20260709 13:01:26.710053  3697 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:26.671806  3290 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:26.743585  1552 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:26.713279  3529 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 13:01:26.739775  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 245424us
I20260709 13:01:26.766921  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:26.738667  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.703464  3657 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:01:26.763880  1553 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:26.734292  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:26.779244  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 183711us
I20260709 13:01:26.780506  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:26.771086  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.800364  1557 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 2: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:26.801777  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63169.7us
I20260709 13:01:26.802853  1635 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:26.802975  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:26.811612  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:26.812628  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45810us
I20260709 13:01:26.815889  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.744740  3484 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:26.762364  3868 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:26.710906  3191 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:26.847715  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:26.855350  1552 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:26.869431  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.770172  3924 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:26.806500  3750 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.819495  2288 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:26.893990  1552 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:26.832855  3856 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:26.837389  3870 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.761358  3800 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.739519  3867 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.775983  3290 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.900519  1553 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.902897  3923 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:01:26.889022  3714 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.908118  3484 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.837044  3692 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:01:26.897115  3821 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:26.888592  3849 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:26.874508  3550 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.940346  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.928107  1553 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:26.874800  3924 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:26.889420  3657 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:26.912302  3191 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.912621  3767 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:26.917347  3290 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.915417  3800 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:26.987952  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.912858  3846 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.911727  1551 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:26.895744  3750 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.979724  3550 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.989506  3821 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.000057  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.001473  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:27.002620  3926 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:26.975970  3924 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:27.040704  3712 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:26.913571  3856 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:01:26.981706  3873 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.043596  3873 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:27.044103  3873 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:27.044646  1551 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8.
I20260709 13:01:27.045145  3663 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:27.044927  3873 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.045549  3663 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.045675  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:26.995719  3697 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:27.009287  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.010138  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.010910  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.011667  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.049495  1633 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:27.012485  3936 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:27.056505  2291 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.504s	user 0.004s	sys 0.005s
I20260709 13:01:27.056979  2291 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:27.057175  3936 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:27.057670  3936 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:27.058095  1551 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:27.058524  3936 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.012938  3850 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:27.013808  3935 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:27.014901  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:26.919835  3607 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.013851  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.016736  3918 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.019804  3692 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:26.971943  3527 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.022910  3867 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:26.967444  3687 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:26.913791  3870 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:26.975577  3657 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:26.920280  3714 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.039721  3938 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.046446  3663 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.040709  3777 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.116605  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.118680  1700 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:27.053341  3810 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.002s
I20260709 13:01:27.043443  3821 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:27.055352  3845 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.590s	user 0.001s	sys 0.010s
I20260709 13:01:27.035259  1552 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:27.063452  1633 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
W20260709 13:01:27.079372  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.086625  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 239624us
W20260709 13:01:27.084014  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 109271us
I20260709 13:01:27.060562  3936 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:27.102396  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.043668  3750 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:27.131493  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 351059us
I20260709 13:01:27.033612  3846 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.114580  1631 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:27.127590  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:27.122651  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 112416us
I20260709 13:01:27.127939  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.127895  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:27.132452  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.132524  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.135653  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.120128  3777 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.115176  3712 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.135813  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.156742  1700 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:27.138067  1552 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.139444  1559 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:27.128716  3810 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.141013  3714 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.141358  3702 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:27.109141  3527 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.146363  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:27.179896  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47481.8us
I20260709 13:01:27.147753  2290 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:27.160581  3697 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:27.148434  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.144829  3724 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.183125  1557 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.183234  2292 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.159665  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:27.184165  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.184702  3724 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.185349  3724 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:27.185528  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.185763  3724 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:27.175976  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.170123  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.184741  1557 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:27.160918  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:27.176777  3663 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.180748  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:27.113564  3692 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.189240  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.189224  3817 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.161968  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.186997  1632 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:27.187382  1555 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.181195  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:27.155906  3750 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:27.196557  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:27.176488  3935 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.158602  3484 raft_consensus.cc:2764] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:01:27.198719  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.198846  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.205154  3944 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.206794  3917 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:27.206954  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.207643  3846 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:27.209098  3607 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.211822  3919 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:01:27.212677  3918 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.213912  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.198355  3780 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.217886  1552 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:27.200111  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:27.202283  1627 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:27.202872  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.196206  3529 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:27.204563  1629 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.219521  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.219965  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.189347  3724 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:27.222353  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.226548  1634 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:27.220443  3935 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.216089  3940 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:01:27.211871  2288 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:27.226397  2290 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.246896  1555 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:27.223526  3944 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:27.225390  3750 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:27.236508  3937 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:27.236661  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.266793  1700 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:27.237222  3550 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.200677  3702 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:27.220600  3692 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.240373  3846 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.240751  3780 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.224309  3924 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.227489  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.203847  3923 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:27.249339  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58764.1us
I20260709 13:01:27.283349  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.285706  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.160687  3867 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.117087  3938 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.253026  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.176187  3657 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.212688  3870 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.256068  3873 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.262768  2292 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:27.264508  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.231303  3872 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.225167  3529 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:27.279146  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59697.1us
W20260709 13:01:27.358414  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72804.4us
W20260709 13:01:27.358465  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.248965  3917 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.248838  3928 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.281805  1629 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:27.291983  3915 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.295032  3923 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.398468  1635 raft_consensus.cc:2368] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
W20260709 13:01:27.292711  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111593us
I20260709 13:01:27.284889  3697 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:27.278857  3944 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:27.279049  3935 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:27.282114  3527 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.362223  3529 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.363173  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:27.288620  3550 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:27.266258  3750 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.300328  1551 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:27.411149  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 191702us
I20260709 13:01:27.402987  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:27.359359  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.406210  3810 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:27.409308  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47154.2us
I20260709 13:01:27.366080  1457 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 4, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:27.305590  3846 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.422979  3935 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.398056  3657 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.423938  3948 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.424491  3948 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.425108  3872 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.366375  3919 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:01:27.425935  3919 consensus_queue.cc:1243] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:27.423573  3935 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.414862  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:27.410869  3750 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.400885  3697 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:27.424942  3944 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.421242  1457 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 3 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:27.398842  3870 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.437171  1457 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:27.428364  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65653.2us
I20260709 13:01:27.449949  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.437985  1553 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:27.449641  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.452499  1560 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:27.452992  1558 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.427732  3948 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.430945  3937 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:27.455451  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 3 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 5 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:27.455649  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 96373.8us
I20260709 13:01:27.457886  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.459748  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 207) took 4503 ms. Trace:
I20260709 13:01:27.469908  3937 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:27.460101  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57225.2us
I20260709 13:01:27.460691  3919 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:27.455538  3948 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:27.430650  3930 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:27.437132  3870 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.470129  1457 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 4, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.445238  3915 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:01:27.411325  3928 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
W20260709 13:01:27.467698  3953 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:27.477486  3657 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:27.435535  3945 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:27.430302  3935 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.476874  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.469099  3750 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.437785  3924 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.435573  3939 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.432404  3697 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.495980  1558 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:27.461797  3527 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.434927  3724 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.533270  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.533896  1557 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.423985  3702 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.534914  3724 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.486371  1458 rpcz_store.cc:276] 0709 13:01:22.936902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.937025 (+   123us) service_pool.cc:224] Handling call
0709 13:01:27.439717 (+4502692us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:25.157321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:25.158054 (+   733us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:25.158106 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:01:25.158130 (+    24us) tablet.cc:662] Decoding operations
0709 13:01:25.159034 (+   904us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:25.159126 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:25.159158 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:01:25.179637 (+ 20479us) tablet.cc:701] Row locks acquired
0709 13:01:25.208537 (+ 28900us) write_op.cc:260] PREPARE: finished
0709 13:01:25.232118 (+ 23581us) write_op.cc:270] Start()
0709 13:01:25.261044 (+ 28926us) write_op.cc:276] Timestamp: P: 1783602085221693 usec, L: 0
0709 13:01:25.263290 (+  2246us) op_driver.cc:348] REPLICATION: starting
0709 13:01:25.346741 (+ 83451us) log.cc:844] Serialized 2515 byte log entry
0709 13:01:25.763563 (+416822us) op_driver.cc:464] REPLICATION: finished
0709 13:01:25.839631 (+ 76068us) write_op.cc:301] APPLY: starting
0709 13:01:25.860668 (+ 21037us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:25.899968 (+ 39300us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:25.942322 (+ 42354us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:26.083841 (+141519us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:26.084152 (+   311us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:01:26.084199 (+    47us) write_op.cc:312] APPLY: finished
0709 13:01:26.084566 (+   367us) log.cc:844] Serialized 53 byte log entry
0709 13:01:26.084756 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.084869 (+   113us) write_op.cc:454] Released schema lock
0709 13:01:26.084930 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:26.085353 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.085378 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":668400,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":75722,"apply.run_cpu_time_us":5745,"apply.run_wall_time_us":245718,"num_ops":6,"prepare.queue_time_us":395983,"prepare.run_cpu_time_us":33745,"prepare.run_wall_time_us":334742,"raft.queue_time_us":150959,"raft.run_cpu_time_us":15615,"raft.run_wall_time_us":66001,"replication_time_us":500074,"spinlock_wait_cycles":1234304,"thread_start_us":538777,"threads_started":4,"wal-append.queue_time_us":190044,"wal-append.run_cpu_time_us":16273,"wal-append.run_wall_time_us":1535737}]]}
I20260709 13:01:27.522018  3867 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:01:27.414278  3917 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.539433  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.540040  1553 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:27.527962  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:27.529009  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52271.2us
I20260709 13:01:27.542577  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.531927  1457 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 5 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.545502  1554 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.548702  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.548017  1457 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 3, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:27.552860  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.525918  2290 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:27.524518  3937 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.526695  3938 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.502532  3290 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:01:27.557871  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 257473us
I20260709 13:01:27.525254  3750 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.562704  1557 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:27.505136  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:27.504586  3657 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:27.477945  3923 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.565447  1457 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.568588  1598 leader_election.cc:400] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:01:27.568905  1598 leader_election.cc:403] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:27.570219  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.571172  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.572494  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.529203  3935 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:27.577203  1706 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:27.578961  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.581593  1629 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:27.523754  3948 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:27.564200  3849 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.586164  1629 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:27.564574  3936 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.586418  1627 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:27.587744  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.505231  3930 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.597329  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.598045  1634 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
W20260709 13:01:27.598556  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49970.2us
I20260709 13:01:27.599304  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.599447  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.599941  1553 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:27.601471  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.567956  3657 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:27.567559  3938 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:27.604677  3946 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.267s	user 0.003s	sys 0.000s
I20260709 13:01:27.605582  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.605788  3845 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:27.606208  2296 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.607923  3850 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.608352  3948 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.608793  3873 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:27.610704  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1234) took 1116 ms. Trace:
W20260709 13:01:27.611621  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46420.8us
I20260709 13:01:27.579857  3867 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:27.573158  3856 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.612624  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:27.566339  3777 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.612637  3867 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:27.613698  3867 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:27.614295  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.614758  3777 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.615103  2290 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:27.616333  3697 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.617624  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.617858  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.623668  3850 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.524869  3939 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.624202  3850 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.625761  3939 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.626178  3850 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.623471  3657 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.628487  3907 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.611081  1556 rpcz_store.cc:276] 0709 13:01:26.494531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.721117 (+226586us) service_pool.cc:224] Handling call
0709 13:01:26.727951 (+  6834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.728119 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:26.889046 (+160927us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.114315 (+225269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.114390 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:27.114455 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.114552 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.477921 (+363369us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:27.610533 (+132612us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b54001ee1d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:27.610644 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5175,"prepare.queue_time_us":270948,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":130,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5054,"spinlock_wait_cycles":22814848,"thread_start_us":375838,"threads_started":2,"wal-append.queue_time_us":335055}
I20260709 13:01:27.631636  3850 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:27.632822  3851 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:27.634618  3851 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.597556  1629 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:27.636745  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.577193  3868 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:27.638337  3868 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:27.638412  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.638650  3918 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:27.639127  3918 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.639463  3868 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.533143  3527 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.642402  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.643049  2291 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:27.643345  3527 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.638185  2296 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:27.580190  3935 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:27.644176  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.644728  3935 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.645002  2288 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:27.644129  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:27.648981  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.628957  3657 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:27.650254  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48873.9us
I20260709 13:01:27.651304  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.579247  3712 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.655687  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:27.656064  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158039us
I20260709 13:01:27.657330  2263 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:27.663609  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51075us
I20260709 13:01:27.663996  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.664912  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.666944  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:27.677290  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.677392  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.576366  3692 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
W20260709 13:01:27.685353  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77533.9us
I20260709 13:01:27.653633  3952 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.688238  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.690968  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.691596  1629 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:27.622886  3529 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.694087  3687 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:01:27.700074  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.700752  1705 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:27.701586  3895 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:27.701846  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.702324  3895 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.702832  1707 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:27.704424  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.674970  3845 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:27.694233  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.708707  1706 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.705154  1704 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:27.675982  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.710680  1633 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:27.665067  3939 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:27.712080  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:27.706296  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 153327us
I20260709 13:01:27.679013  3924 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:27.716065  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.688015  3527 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:27.713477  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.714870  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:27.694428  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.623239  3867 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.756371  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.757113  1703 raft_consensus.cc:2376] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3. Re-sending same reply.
I20260709 13:01:27.716892  1629 raft_consensus.cc:2410] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:27.700673  3529 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.692376  3922 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:27.715029  1708 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:01:27.747459  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:27.760190  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82895.3us
I20260709 13:01:27.710934  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.751507  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.717829  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.761194  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:27.686473  3936 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:01:27.703452  3895 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.827821  2263 leader_election.cc:400] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8.
W20260709 13:01:27.768325  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56306.1us
W20260709 13:01:27.772857  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49880.1us
I20260709 13:01:27.753196  3850 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.774930  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.828387  2263 leader_election.cc:403] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:27.753610  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:27.831838  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49062.8us
W20260709 13:01:27.832042  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 78484.4us
I20260709 13:01:27.832526  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.685578  3918 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.831182  1709 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:27.773480  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 91505.5us
I20260709 13:01:27.747633  2288 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:27.715147  3912 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.760179  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.780826  3905 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:27.752640  3527 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:27.829152  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:27.842968  2292 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:27.843572  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.846611  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:27.873006  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43954.6us
I20260709 13:01:27.873256  1556 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:27.874101  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.846858  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.849260  1558 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:27.849838  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.851846  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.753456  3939 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:27.718502  3915 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:01:27.860467  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:27.879730  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.880517  1560 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:27.880669  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48230.1us
I20260709 13:01:27.880843  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.881417  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.863875  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:27.709563  3952 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.868886  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.839810  3956 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:27.881479  1633 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:27.784538  2290 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:27.845147  3780 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8.
I20260709 13:01:27.877887  1705 raft_consensus.cc:2410] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:27.860471  3692 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.863127  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.858465  3529 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:27.892095  2296 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:27.858289  3669 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:27.906467  3964 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:01:27.747495  3930 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.908413  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.828235  3957 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:27.919965  3957 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.920326  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:27.858952  3687 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:01:27.850605  3924 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.896804  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.899302  2293 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:27.902510  2294 raft_consensus.cc:2368] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
W20260709 13:01:27.903852  3965 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:01:27.905340  3934 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:27.905828  1701 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.884850  3956 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:27.937225  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.918037  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:27.918737  3958 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:01:27.920463  3957 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.845525  3702 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:27.885665  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:27.921373  3944 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:27.924474  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.892542  3870 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.846683  3850 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:27.909144  3967 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:27.928041  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:27.953648  1702 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 13:01:27.954452  3850 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:27.929317  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.955524  1633 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:27.956278  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.956920  1633 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:01:27.957854  3550 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.958647  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:27.959403  1553 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:27.939556  3714 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.916893  3867 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:27.931420  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.922360  3692 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:27.965273  3657 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.934311  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.959276  3550 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:01:27.966522  1631 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:27.935271  1627 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.937819  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:27.910527  3529 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.968017  3529 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:27.908470  3963 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:01:27.944502  1704 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:27.922896  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:27.950765  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:27.950927  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.959523   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:27.970068  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:27.977254  1708 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:27.979343  1628 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:27.941869  1635 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:27.922350  3905 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:27.980628  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:27.983018  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:27.934509  3939 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:27.922434  3937 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:27.941545  3956 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.988458  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:27.947150  3944 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:27.999639  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:28.006842  3907 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.004107  3924 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.013880  1708 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:28.018285  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.018424  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:27.971448  3702 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.022004  1560 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:27.987596  3980 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:27.997575  3669 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:28.000438  3944 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.004489  3937 consensus_queue.cc:1243] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:27.996773  3934 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:28.036701  2293 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:28.040218  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158866us
I20260709 13:01:28.020617  3692 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.046995  1706 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:27.998852  3527 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.024250  3724 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:28.033934  3699 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:28.033504  3980 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:28.043694  3972 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:01:28.285857  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.158962  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:28.469595  3937 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.793089  3980 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.796432  3692 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.796588  1557 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:28.796947  3692 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:01:28.775609  3712 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.113s	user 0.008s	sys 0.000s
I20260709 13:01:28.797868  3712 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:28.798013  1557 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:28.798483  1556 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:28.799010  3692 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:28.799202  1557 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:28.783449  3935 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.138s	user 0.012s	sys 0.000s
I20260709 13:01:28.364362  3777 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.850153  3777 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.850976  1557 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:28.852912  3777 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.151793  3919 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:28.166452  3979 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:01:28.065948  3849 raft_consensus.cc:2764] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:01:28.055805  3905 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.854507  3935 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.856649  3935 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:28.208319  3934 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.857978  3777 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.858645  3777 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.211035  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:28.795842  3937 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.155692  3981 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.714s
I20260709 13:01:28.860677  3981 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:01:28.085072  3969 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.186s	user 0.003s	sys 0.000s
I20260709 13:01:28.860996  1551 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:28.861016  3981 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:28.789389  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 770989us
I20260709 13:01:28.862061  3937 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:28.862178  3981 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.863070  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:28.863305  3981 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.864197  3777 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:28.077699  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:28.168265  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:28.604998  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.002s	sys 0.009s
W20260709 13:01:28.867961  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.005s	sys 0.010s
W20260709 13:01:28.867842  3846 consensus_queue.cc:1175] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
W20260709 13:01:28.221174  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.159533  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:28.869158  1627 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:28.537266  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:28.073658  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:28.870651  2293 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:28.692379  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 258719us
I20260709 13:01:28.871718  3692 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:28.872306  3692 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.874442  3692 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.874717  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:28.874951  3692 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:28.489405  1552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.010s	sys 0.010s
W20260709 13:01:28.876017  1552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.011s	sys 0.015s
I20260709 13:01:28.789691  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:28.805557  3980 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.877388  1632 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:28.877699  3937 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.878218  3937 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:28.790941  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 764) took 1105 ms. Trace:
I20260709 13:01:28.792754  3699 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.789619  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:28.880503  1560 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:28.879976  2296 rpcz_store.cc:276] 0709 13:01:27.685233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.693774 (+  8541us) service_pool.cc:224] Handling call
0709 13:01:28.790676 (+1096902us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:28.790856 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323045120}
I20260709 13:01:28.881639  3980 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:28.875442  3692 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.883545  3699 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.884270  3699 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:28.417900  3952 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.158970  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:28.800487  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.889195  3692 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.885011  3607 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:28.884874  3699 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.898159  3777 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.898984  3777 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:28.903239  3777 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:28.885465  3919 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.880740  3937 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:28.900375  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:28.974148  3935 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.898300  3699 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:28.946660  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:01:29.043377  1442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 96762us
W20260709 13:01:28.946911  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.981665  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:28.947154  3714 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.987s	user 0.007s	sys 0.004s
I20260709 13:01:28.987481  1635 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:29.044713  1627 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:29.046677  3714 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:29.050652  3657 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.085s	user 0.008s	sys 0.004s
I20260709 13:01:29.050868  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.051335  1700 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:28.707280  2290 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:29.026563  1626 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:01:28.972666  3702 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:29.029707  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 129185us
I20260709 13:01:29.066221  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:29.041600  3527 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.051187  3657 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.894398  3692 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:29.071080  1632 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:29.077045  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1965) took 1148 ms. Trace:
I20260709 13:01:29.080096  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1967) took 1110 ms. Trace:
I20260709 13:01:29.120304  1558 rpcz_store.cc:276] 0709 13:01:27.929089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.929251 (+   162us) service_pool.cc:224] Handling call
0709 13:01:27.929557 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.939828 (+ 10271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.939854 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.975513 (+ 35659us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.131331 (+155818us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:28.131392 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.131489 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.076740 (+945251us) raft_consensus.cc:1685] finished
0709 13:01:29.076767 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.076964 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6392,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5842,"thread_start_us":125741,"threads_started":1,"wal-append.queue_time_us":1068934}
I20260709 13:01:29.120396  1553 rpcz_store.cc:276] 0709 13:01:27.970087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.970241 (+   154us) service_pool.cc:224] Handling call
0709 13:01:27.970545 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.970884 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.970911 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.978067 (+  7156us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.789897 (+811830us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:28.789944 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.790016 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.079783 (+289767us) raft_consensus.cc:1685] finished
0709 13:01:29.079814 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.080011 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":9642,"thread_start_us":242037,"threads_started":1,"wal-append.queue_time_us":1087173}
I20260709 13:01:29.043030  3937 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:29.059811  2290 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:29.046056  3777 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.029908  3919 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:29.085147  3957 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.113s	user 0.008s	sys 0.003s
I20260709 13:01:29.046000  3935 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:29.094358  3944 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.030s	user 0.007s	sys 0.002s
I20260709 13:01:28.509126  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:29.100021  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:29.100740  1702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.000s	sys 0.005s
W20260709 13:01:29.133890  1702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.000s	sys 0.006s
I20260709 13:01:29.123627  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1931) took 1047 ms. Trace:
I20260709 13:01:29.134800  1632 rpcz_store.cc:276] 0709 13:01:28.076010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.076518 (+   508us) service_pool.cc:224] Handling call
0709 13:01:29.123543 (+1047025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.145895  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:29.118299  3907 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.111s	user 0.006s	sys 0.004s
I20260709 13:01:29.117372  3699 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:29.150949  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.168953  1631 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:01:29.171402  1631 pending_rounds.cc:85] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 3
I20260709 13:01:29.171844  1631 pending_rounds.cc:107] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:01:29.172247  1631 raft_consensus.cc:2889] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:29.146032  3712 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:29.174122  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.176065  1632 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:29.177340  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.119323  3527 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.148640  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:29.171252  3767 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.181044  3767 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.181591  3919 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.183301  3767 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:29.183420  3956 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.173s	user 0.010s	sys 0.001s
I20260709 13:01:29.144338  3777 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.145328  3957 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.179399  3750 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:29.187788  3956 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.190503  3750 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:29.191650  3957 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.191669  3750 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.192902  3957 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.187906  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.193182  3777 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.193522  3957 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.194046  3956 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.194638  3956 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.195063  3956 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.195487  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:29.178838  3191 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.196413  1632 raft_consensus.cc:2376] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3. Re-sending same reply.
I20260709 13:01:29.145202  3944 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.200984  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:29.186777  3939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.013s	sys 0.005s
I20260709 13:01:29.198338  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1225) took 1347 ms. Trace:
I20260709 13:01:29.184600  2290 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:29.236219  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 90498us
W20260709 13:01:29.237006  1559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.000s	sys 0.011s
I20260709 13:01:29.208863  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 500) took 1573 ms. Trace:
I20260709 13:01:29.251014  2292 rpcz_store.cc:276] 0709 13:01:27.635327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.635534 (+   207us) service_pool.cc:224] Handling call
0709 13:01:27.635879 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.872671 (+236792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.208560 (+1335889us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400329e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:29.208779 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345261312}
W20260709 13:01:29.252607  1559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.000s	sys 0.012s
I20260709 13:01:29.253517  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1964) took 1325 ms. Trace:
I20260709 13:01:29.240746  1704 rpcz_store.cc:276] 0709 13:01:27.842295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.989750 (+147455us) service_pool.cc:224] Handling call
0709 13:01:27.999373 (+  9623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.999583 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:28.868460 (+868877us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.189155 (+320695us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.189269 (+   114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.189297 (+    28us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:29.189330 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.189373 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.189468 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1042362,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":713,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":185,"thread_start_us":1168101,"threads_started":2}
I20260709 13:01:29.253942  1559 rpcz_store.cc:276] 0709 13:01:27.927826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.927986 (+   160us) service_pool.cc:224] Handling call
0709 13:01:27.928298 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.928500 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.928526 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.940160 (+ 11634us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.003419 (+ 63259us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:28.526677 (+523258us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.253081 (+726404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.253186 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.253258 (+    72us) raft_consensus.cc:1685] finished
0709 13:01:29.253276 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.253443 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4001,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":33,"raft.queue_time_us":72397,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":15523,"thread_start_us":1225137,"threads_started":3,"wal-append.queue_time_us":26622}
I20260709 13:01:29.205080  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:29.202134  1632 raft_consensus.cc:2376] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5. Re-sending same reply.
I20260709 13:01:29.195637  3957 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.270284  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.271174  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1959) took 1419 ms. Trace:
I20260709 13:01:29.272508  3944 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.273644  1559 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:01:29.202834  3939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.013s	sys 0.005s
I20260709 13:01:29.282495  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:29.284332  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:29.237795  3919 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.236722  3845 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.285226  1552 rpcz_store.cc:276] 0709 13:01:27.851683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.851814 (+   131us) service_pool.cc:224] Handling call
0709 13:01:27.874246 (+ 22432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.874443 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:27.895669 (+ 21226us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.876654 (+980985us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.270832 (+394178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.270884 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:29.270930 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.271003 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.271116 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27216,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":42629,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"spinlock_wait_cycles":25344,"thread_start_us":1374327,"threads_started":2,"wal-append.queue_time_us":1137424}
I20260709 13:01:29.212371  3767 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.289733  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.280679  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:29.209133  3714 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.292177  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 239805us
I20260709 13:01:29.204897  3937 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:29.292946  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:29.296304  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.276345  3712 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.281433  3692 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.285261  3699 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.281786  3907 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:29.212337  3191 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.288741  3957 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.320112  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:29.311947  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.321630  1635 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:29.316897  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:29.276659  3777 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:29.209777  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:29.326228  1705 raft_consensus.cc:606] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:29.326723  1705 raft_consensus.cc:614] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:29.296144  4006 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.328122  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 123147us
I20260709 13:01:29.338227  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:29.300050  3939 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:29.302902  3956 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.303634  4008 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:29.311899  3527 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.296578  3845 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.299178  3767 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.302426  3714 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:29.355741  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 70314.8us
I20260709 13:01:29.341753  3939 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.340401  3957 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.318602  3692 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.359720  3957 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.360424  3957 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:29.326619  3944 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:29.343291  4003 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:29.361922  3944 consensus_queue.cc:1243] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:29.296276  3919 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.362625  3944 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:29.363023  3944 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:29.364017  3944 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.364625  3919 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.354920  3527 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:29.361931  4003 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.344911  4006 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.365693  4003 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:29.350252  4008 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:29.368221  4008 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.369549  3845 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.336673  3907 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.371697  3907 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.361290  3692 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.373237  3714 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.324318  3699 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.356827  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:29.381682  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.382459  1635 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:29.313658  3712 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.312245  3998 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:29.349364  4009 raft_consensus.cc:2764] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:01:29.400089  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52916.8us
I20260709 13:01:29.336337  4010 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:01:29.349562  3777 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:01:29.376564  3939 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:29.514497  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 112914us
I20260709 13:01:29.514833  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:29.356223  4014 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:29.520190  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 173214us
I20260709 13:01:29.386127  3935 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.392810  4016 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.393316  3692 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:29.537295  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 180512us
W20260709 13:01:29.521476  3529 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.549s	user 0.007s	sys 0.006s
I20260709 13:01:29.543588  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:29.518075  4012 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:29.518354  3907 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.548040  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 209397us
I20260709 13:01:29.515829  4006 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.394166  4004 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.160s
I20260709 13:01:29.533707  4016 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.551110  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:29.555070  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:29.555822  1707 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.558630  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:29.545547  4013 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.547829  4012 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:29.570701  2292 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:29.577068  2296 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:29.548234  4015 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.550316  3699 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.581923  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:29.561633  3935 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:29.557646  4016 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.547200  3529 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.539608  4014 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.573995  3919 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.572705  3777 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.578764  4013 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.569967  4024 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:29.578492  4004 consensus_queue.cc:1243] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:29.550617  3692 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.589257  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:29.590824  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:29.553602  3907 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.592401  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:29.593262  3919 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.548436  4017 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:01:29.598352  3919 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.598343  4004 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.598615  4027 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.598632  4024 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:29.599392  4004 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.586714  4022 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:01:29.593632  4013 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.601287  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:29.596525  2293 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:29.588160  3935 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.587453  3699 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.608201  2294 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:29.600489  4024 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.599654  4027 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.609484  3935 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:29.614014  4027 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.591820  3845 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:29.591933  3767 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:29.620625  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 300593us
I20260709 13:01:29.625579  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:29.618136  1707 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:29.626965  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1213) took 1872 ms (client timeout 1715 ms). Trace:
W20260709 13:01:29.627499  1707 rpcz_store.cc:269] 0709 13:01:27.754808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.755149 (+   341us) service_pool.cc:224] Handling call
0709 13:01:29.554913 (+1799764us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:29.626871 (+ 71958us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142646656}
I20260709 13:01:29.632527  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:29.633984  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:29.635161  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:29.610320  2289 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:29.636686  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 298787us
I20260709 13:01:29.642326  3845 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:29.642756  3845 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.585783  4012 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.643092  3845 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:29.643268  4023 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:29.643759  4023 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.643849  4012 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.644155  4023 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:29.645041  4003 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:29.586228  4015 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:29.647218  4003 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:29.638290  4010 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:29.652355  3767 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.652889  3767 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:29.655576  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 112056us
I20260709 13:01:29.594238  3777 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.622285  4017 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:29.647468  4021 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:29.637312  3712 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.643014  3692 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.615646  3699 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.643014  3907 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.657539  3714 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:29.659278  4017 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.663349  4015 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:29.661499  3919 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:30.122527  3944 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.026245  4010 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:29.876792  3873 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:30.077512  3999 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.048s	user 0.001s	sys 0.016s
I20260709 13:01:29.988512  4029 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.148097  3919 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:30.138307  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.008s	sys 0.012s
I20260709 13:01:29.997098  3935 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.116725  3938 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:29.873257  4012 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.279s
I20260709 13:01:30.152412  3938 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:30.153185  3919 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:30.041421  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.669159  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.148108  4017 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.032673  4016 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.671324  3527 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.482s
I20260709 13:01:29.834635  3907 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.155352  3907 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.156212  4016 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.156879  4016 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:29.843179  4004 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.157727  3907 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.158116  3937 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:29.668996  3529 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.158730  3937 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:29.693040  4031 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.526s
I20260709 13:01:30.160918  3937 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.862751  3767 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.161640  3937 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:29.844946  4014 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:29.667191  4026 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.162298  3937 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.162945  4026 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.025718  3939 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.164625  3937 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.165148  3937 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.664182  3998 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.488s
I20260709 13:01:30.165560  3937 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:29.723426  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.166262  3998 consensus_queue.cc:1243] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:30.167016  3998 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:30.167461  3998 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:01:30.168738  1673 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.693s (SENT) [suppressed 38 similar messages]
I20260709 13:01:29.995880  3699 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:30.139122  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 478708us
I20260709 13:01:30.169360  4017 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.019181  4013 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.170706  2262 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.172556  4013 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.172931  3956 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:30.172753  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:29.670943  4018 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.537s
I20260709 13:01:29.693493  4033 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.492s
I20260709 13:01:30.174661  3846 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:30.023960  4023 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.175118  3846 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:29.667650  4021 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.176117  3846 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.176749  4023 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.174074  3956 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.849198  4008 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.172711  4017 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.149566  3873 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.179268  4008 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.135373  4041 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.187371  4041 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.177661  3956 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:30.189225  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.189996  2297 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.190325  3956 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.191458  4041 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:30.153355  3935 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.165912  3939 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:30.151643  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.012s
I20260709 13:01:30.188441  2295 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:30.150146  4029 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.194420  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:29.997726  3845 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.188580  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.196180  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.197275  2288 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:30.196156  4029 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.198060  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.198185  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.198958  2288 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:30.199596  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.200121  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.200791  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.200933  2288 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:30.202024  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.202922  2288 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:30.203953  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.204679  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.205129  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.205927  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.209343  4031 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:30.209836  1635 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:30.210333  2297 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:30.210006  4031 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.129128  4009 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:30.210819  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.212531  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.213732  3529 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:30.215173  4040 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.212603  4031 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.211186  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.214447  3529 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.216660  4040 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.218003  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1930) took 2142 ms. Trace:
I20260709 13:01:30.213735  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.177631  4021 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:30.218852  1635 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5. Re-sending same reply.
I20260709 13:01:30.120450  3918 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.218619  1631 rpcz_store.cc:276] 0709 13:01:28.075651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.076370 (+   719us) service_pool.cc:224] Handling call
0709 13:01:28.161092 (+ 84722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.177539 (+1016447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.217774 (+1040235us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54002977d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:30.217944 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279033728}
I20260709 13:01:30.220218  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.222899  4031 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.223572  4031 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.214104  2288 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:30.226145  4031 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.218613  3529 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.228942  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 774) took 2244 ms. Trace:
I20260709 13:01:30.229303  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.230568  2291 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:01:30.229416  2294 rpcz_store.cc:276] 0709 13:01:27.984035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.984181 (+   146us) service_pool.cc:224] Handling call
0709 13:01:27.984591 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.609796 (+1625205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.228592 (+618796us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54005186d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:30.228832 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718243712}
I20260709 13:01:30.233603  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:30.234264  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.234450  2289 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:30.234956  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:30.235598  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1962) took 2353 ms. Trace:
I20260709 13:01:30.235839  1552 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:30.236570  4026 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.168460  3998 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.236055  1555 rpcz_store.cc:276] 0709 13:01:27.882471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.882716 (+   245us) service_pool.cc:224] Handling call
0709 13:01:27.882997 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.967619 (+ 84622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.967648 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.968147 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.233189 (+265042us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:28.419007 (+185818us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.430478 (+ 11471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.124845 (+1694367us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b18000ca600. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:30.235258 (+110413us) spinlock_profiling.cc:243] Waited 84.2 ms on lock 0x7b54003fb454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:30.235311 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.235368 (+    57us) raft_consensus.cc:1685] finished
0709 13:01:30.235382 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.235532 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":189544,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":11889,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":215,"spinlock_wait_cycles":282953344,"thread_start_us":264648,"threads_started":1,"wal-append.queue_time_us":1074492,"wal-append.run_cpu_time_us":2138,"wal-append.run_wall_time_us":1038704}
I20260709 13:01:30.238260  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.239568  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.240388  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:30.240967  1555 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.240880  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.241289  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.241930  2291 raft_consensus.cc:2368] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:30.242928  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.243755  3750 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.244114  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.246479  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.244992  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.247709  2291 raft_consensus.cc:2410] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:30.248414  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.248785  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.249667  3939 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.249858  4021 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.250873  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.251322  1704 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:30.250234  3939 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.256708  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.251777  4021 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.265306  3529 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:30.253716  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.266412  4021 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:30.266546  1706 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:30.266952  4021 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.267364  3939 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.267488  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:30.276319  3529 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:30.276445  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:30.276911  3529 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:30.252612  3850 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:30.277873  3529 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.278846  3850 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.281208  3850 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:30.282719  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1221) took 2502 ms. Trace:
I20260709 13:01:30.267616  2289 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:30.284670  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.251844  1705 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:30.286212  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:30.286957  1705 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:30.292158  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.257479  3873 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.211313  4009 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:30.259298  3845 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.270506  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.301183  1559 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:30.252306  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.302486  1707 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:30.271190  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.303625  1558 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:30.263247  1701 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:30.265429  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.308866  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.309491  1559 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:30.310576  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.310789  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.311211  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:30.283205  1702 rpcz_store.cc:276] 0709 13:01:27.780293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.955054 (+174761us) service_pool.cc:224] Handling call
0709 13:01:27.955511 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.055882 (+100371us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:28.418948 (+363066us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.134859 (+715911us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.148575 (+ 13716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.148625 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:29.148672 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.204228 (+1055556us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18004363c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:30.282036 (+ 77808us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b18004363c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419
0709 13:01:30.282479 (+   443us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:30.282523 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.282647 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13381,"prepare.queue_time_us":995222,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":80,"raft.queue_time_us":2515,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8460,"spinlock_wait_cycles":263269760,"thread_start_us":1047044,"threads_started":2,"wal-append.queue_time_us":707102,"wal-append.run_cpu_time_us":2145,"wal-append.run_wall_time_us":1074554}
I20260709 13:01:30.267864  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.312168  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.292303  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:30.292809  3970 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.292773  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.186570  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1218) took 2429 ms. Trace:
W20260709 13:01:30.154743  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 485622us
I20260709 13:01:30.181825  3191 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.268935  4040 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.322857  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:30.269546  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:30.313053  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.314821  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.325162  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.326030  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.316885  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.312816  1554 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:30.321136  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:30.327656  1707 raft_consensus.cc:2376] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6. Re-sending same reply.
I20260709 13:01:30.299319  1705 raft_consensus.cc:2410] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:30.314097  1703 rpcz_store.cc:276] 0709 13:01:27.756945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.773398 (+ 16453us) service_pool.cc:224] Handling call
0709 13:01:27.773749 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.242801 (+1469052us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:29.560806 (+318005us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b18004365a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf
0709 13:01:29.560964 (+   158us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.615151 (+ 54187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:29.615593 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.615689 (+    96us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:29.615708 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.186167 (+570459us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:30.186239 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.186301 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:30.186318 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.186494 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55621,"prepare.queue_time_us":60328,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":23258,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":28249,"spinlock_wait_cycles":3300246144,"thread_start_us":71821,"threads_started":2,"wal-append.queue_time_us":92632}
I20260709 13:01:30.276842  3998 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:30.153368  3938 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.338646  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.341447  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:30.342916  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:30.167675  3937 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:30.348366  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:30.348639  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 101586us
I20260709 13:01:30.350117  1524 leader_election.cc:400] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:30.366286  1524 leader_election.cc:403] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:30.350839  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.367513  1706 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:30.267397  4021 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:30.350425  4031 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:30.380780  1554 raft_consensus.cc:1217] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-4.4]   Dedup: 3.3->[4.4-4.4]
I20260709 13:01:30.381330  1554 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 13:01:30.381947  1557 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:30.383359  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.384363  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:30.384599  1557 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:30.345676  4008 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:30.346312  3870 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:30.384506  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.391970  1555 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:30.392874  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.393463  1555 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:30.394224  1555 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:30.346053  4040 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:30.377844  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 83722.9us
I20260709 13:01:30.386862  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.387585  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:30.386090  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 217362us
I20260709 13:01:30.387943  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:30.378863  3924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.007s	sys 0.011s
W20260709 13:01:30.396147  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46164.1us
I20260709 13:01:30.399215  2294 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:30.377297  3956 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:30.379163  3937 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:30.402096  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.418040  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.379062  4033 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:30.403016  3702 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.407196  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.400108  1560 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:30.405149  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:30.409776  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:30.391788  3873 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4.
I20260709 13:01:30.391209  3529 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:30.386713  4008 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:30.421437  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:30.398370  4040 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.421083  3724 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:30.427168  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.428987  2290 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.498646  4008 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:30.436153  2260 connection.cc:256] RPC call timeout handler was delayed by -1.1522609429999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:01:30.436690  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8582890809999526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
W20260709 13:01:30.536705  1601 connection.cc:256] RPC call timeout handler was delayed by -1.1198065069999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 13:01:30.537352  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:30.483057  3702 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:30.540494  3702 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:30.541366  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.542003  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:30.406409  3924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.370s	user 0.008s	sys 0.011s
I20260709 13:01:30.377328  4023 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:30.547216  4023 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:30.547618  4023 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:30.548532  4023 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.548961  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.549769  4023 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:30.326669  3970 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.550299  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:30.550961  1627 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:30.518654  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.552258  2291 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.552628  3970 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:30.401124  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.400974  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.445523  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:30.438853  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:30.554883  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 116099us
I20260709 13:01:30.555552  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:30.437176  1555 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:30.391652  4024 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:01:30.416852  3956 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.528102  4040 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:30.445114  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.391914  3687 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:01:30.568933  4050 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:01:30.569415  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:30.570098  1706 raft_consensus.cc:2376] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5. Re-sending same reply.
I20260709 13:01:30.571120  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:30.575649  4005 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:30.578012  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:30.403805  3870 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:30.404409  3191 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.485718  3529 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.498317  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:30.422715  4033 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:30.482051  3845 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:30.417752  3937 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.486508  3767 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:30.433338  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:30.544963  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 123633us
I20260709 13:01:30.557466  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.486347  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.437129  4034 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
W20260709 13:01:30.562645  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 117212us
I20260709 13:01:30.401510  4009 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:30.417014  3850 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:01:30.579702  2289 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:30.982064  3868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.799s	user 0.017s	sys 0.016s
W20260709 13:01:31.026085  3868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.844s	user 0.017s	sys 0.017s
I20260709 13:01:31.020840  4024 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:30.597700  1706 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:31.027022  4024 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:30.605965  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 45089us
I20260709 13:01:30.982512  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.686409  3657 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.028611  1558 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:31.029299  3529 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.030081  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:31.030232  4024 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:31.031108  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:30.562021  3956 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.032543  1558 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:31.034020  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.034742  1626 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:30.589265  1627 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:30.697322  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:31.037369  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.037951  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.627445  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.019985  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 423716us
I20260709 13:01:30.702744  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.047109  1631 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 2: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:01:31.066488  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.082062  1555 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:31.083724  1555 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.093353  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.094082  1627 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:31.094416  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:31.095052  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.095660  1626 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:31.095053  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:31.141455  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47097.8us
I20260709 13:01:31.164403  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:30.694217  3895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.775s	user 0.008s	sys 0.044s
I20260709 13:01:30.811000  1701 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:31.042744  3988 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 2.901s	user 0.012s	sys 0.039s
I20260709 13:01:30.650801  3724 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:30.579365  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:31.019593  3777 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:31.411149  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 1960) took 3559 ms (client timeout 2860 ms). Trace:
I20260709 13:01:31.093331  3191 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:31.287456  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.011s	sys 0.033s
W20260709 13:01:31.342533  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.097141  3767 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.321203  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1253) took 1038 ms. Trace:
I20260709 13:01:31.108101  3870 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.283617  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.438318  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1990) took 1052 ms. Trace:
I20260709 13:01:31.438730  1705 rpcz_store.cc:276] 0709 13:01:30.385830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.283197 (+897367us) service_pool.cc:224] Handling call
0709 13:01:31.438257 (+155060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.241304  4024 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:31.097343  3845 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.097481  3937 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.165902  4021 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:31.323184  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:30.559286  3818 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.457700  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:31.443166  2288 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.218s	user 0.010s	sys 0.006s
W20260709 13:01:31.416024  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.009s	sys 0.006s
W20260709 13:01:31.464414  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.010s	sys 0.007s
I20260709 13:01:31.417490  1560 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:31.425453  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.017s	sys 0.048s
W20260709 13:01:31.424763  1556 rpcz_store.cc:269] 0709 13:01:27.851995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.873843 (+ 21848us) service_pool.cc:224] Handling call
0709 13:01:31.226323 (+3352480us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b54000767d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:31.411057 (+184734us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124508416}
I20260709 13:01:31.416862  3724 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:31.428748  4042 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.772s	user 0.004s	sys 0.006s
I20260709 13:01:31.098361  4033 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:30.559643  4029 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:30.561777  3948 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:31.447947  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 244979us
I20260709 13:01:31.488739  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:31.454983  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 425863us
I20260709 13:01:31.454999  1627 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:31.415222  3936 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:31.416487  3970 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:31.172616  3895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.254s	user 0.016s	sys 0.058s
W20260709 13:01:31.415612  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 395226us
I20260709 13:01:31.438510  3767 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.480108  3850 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:31.468096  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1228) took 3554 ms. Trace:
I20260709 13:01:31.440284  4024 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.450032  3845 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:31.440608  3657 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:31.481120  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.449771  4006 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:31.489969  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1956) took 1005 ms. Trace:
I20260709 13:01:31.522676  1627 rpcz_store.cc:276] 0709 13:01:30.484692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.191971 (+707279us) service_pool.cc:224] Handling call
0709 13:01:31.489906 (+297935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.478752  3724 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:31.438027  1701 rpcz_store.cc:276] 0709 13:01:30.273071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.379732 (+106661us) service_pool.cc:224] Handling call
0709 13:01:31.311625 (+931893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.491539  3750 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.505781  3924 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.462082  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 380939us
I20260709 13:01:31.465736  2288 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:31.445746  3870 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.512979  2294 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:31.513136  1700 rpcz_store.cc:276] 0709 13:01:27.913577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.000595 (+ 87018us) service_pool.cc:224] Handling call
0709 13:01:28.021709 (+ 21114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.051683 (+1029974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.467837 (+2416154us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540016ba54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:31.468028 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367415040}
I20260709 13:01:31.487119  3948 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.523856  3191 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:31.470561  3928 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:31.530987  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1983) took 1203 ms. Trace:
I20260709 13:01:31.563244  1705 rpcz_store.cc:276] 0709 13:01:30.327518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.458707 (+1131189us) service_pool.cc:224] Handling call
0709 13:01:31.530917 (+ 72210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.518075  4024 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.511447  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:31.571784  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65011.9us
I20260709 13:01:31.554239  3845 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.512987  3850 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:31.540398  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.574788  1701 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:31.575224  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1995) took 1030 ms. Trace:
I20260709 13:01:31.575523  1701 rpcz_store.cc:276] 0709 13:01:30.544517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.540047 (+995530us) service_pool.cc:224] Handling call
0709 13:01:31.575147 (+ 35100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.512549  3970 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:31.533344  4006 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:31.518983  4008 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.969s	user 0.011s	sys 0.000s
I20260709 13:01:31.576639  3928 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:31.577083  3928 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:31.576956  3970 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:31.578038  3928 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.582165  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.583001  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1250) took 1310 ms. Trace:
I20260709 13:01:31.583309  1703 rpcz_store.cc:276] 0709 13:01:30.272391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.581896 (+1309505us) service_pool.cc:224] Handling call
0709 13:01:31.582940 (+  1044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.563872  3990 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 2.522s	user 0.020s	sys 0.031s
I20260709 13:01:31.584568  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.585325  1707 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:31.585525  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 323) took 3656 ms. Trace:
I20260709 13:01:31.545660  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:31.586277  1627 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:31.586764  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 510) took 1406 ms. Trace:
W20260709 13:01:31.545401  3529 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
I20260709 13:01:31.588217  3529 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.510062  3868 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:31.590267  3529 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:31.590507  3868 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:31.590935  3868 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.591418  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:31.592219  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1240) took 1323 ms (client timeout 1745 ms). Trace:
W20260709 13:01:31.592600  1706 rpcz_store.cc:269] 0709 13:01:30.268692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.591143 (+1322451us) service_pool.cc:224] Handling call
0709 13:01:31.592155 (+  1012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.576994  4008 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:31.595345  4008 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:31.553942  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.595654  4015 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:31.545611  2292 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:31.560523  3750 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:31.555786  3948 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.566264  3657 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.552145  3870 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:31.566802  3777 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:31.567337  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.612018  1705 raft_consensus.cc:2376] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6. Re-sending same reply.
I20260709 13:01:31.612624  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1248) took 1341 ms. Trace:
I20260709 13:01:31.613032  1705 rpcz_store.cc:276] 0709 13:01:30.270881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.564122 (+1293241us) service_pool.cc:224] Handling call
0709 13:01:31.612556 (+ 48434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.546082  3981 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.569278  3724 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:31.571019  3998 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.225s	user 0.006s	sys 0.002s
I20260709 13:01:31.585675  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1247) took 1315 ms. Trace:
I20260709 13:01:31.614650  1707 rpcz_store.cc:276] 0709 13:01:30.270646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.584298 (+1313652us) service_pool.cc:224] Handling call
0709 13:01:31.585625 (+  1327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.587122  2296 rpcz_store.cc:276] 0709 13:01:30.180381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.180503 (+   122us) service_pool.cc:224] Handling call
0709 13:01:30.180811 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:30.181245 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.181274 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:30.181970 (+   696us) log.cc:844] Serialized 52 byte log entry
0709 13:01:30.261691 (+ 79721us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:30.320626 (+ 58935us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.586339 (+1265713us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400188ba0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95f733 00007f70ce9e4dbf 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c
0709 13:01:31.586391 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.586471 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.586508 (+    37us) raft_consensus.cc:1685] finished
0709 13:01:31.586588 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.586709 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7827,"prepare.queue_time_us":44454,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":2121,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":2814,"spinlock_wait_cycles":43663744,"thread_start_us":137775,"threads_started":2,"wal-append.queue_time_us":79644,"wal-append.run_cpu_time_us":3643,"wal-append.run_wall_time_us":1183950}
W20260709 13:01:31.543744  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.001s	sys 0.016s
W20260709 13:01:31.616144  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.001s	sys 0.016s
W20260709 13:01:31.616888  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.22891e+06us
I20260709 13:01:31.617733  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:31.561256  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:31.619109  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57973.8us
I20260709 13:01:31.620538  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:31.621411  3191 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:31.621793  3191 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.622166  3191 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.623852  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:31.597445  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.586097  1634 rpcz_store.cc:276] 0709 13:01:27.929234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.930675 (+  1441us) service_pool.cc:224] Handling call
0709 13:01:27.930957 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.986040 (+ 55083us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:27.986073 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.986492 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.150992 (+164500us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:28.151050 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.151142 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.286510 (+3135368us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:31.585173 (+298663us) raft_consensus.cc:1685] finished
0709 13:01:31.585201 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.585446 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67878,"raft.queue_time_us":57185,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10156,"spinlock_wait_cycles":22719872,"thread_start_us":133882,"threads_started":1,"wal-append.queue_time_us":1043354}
I20260709 13:01:31.601872  2289 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:31.602061  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:31.597574  4023 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.038s	user 0.007s	sys 0.000s
W20260709 13:01:31.551110  2290 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.007s	user 0.009s	sys 0.000s
I20260709 13:01:31.605917  1627 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:31.661906  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1251) took 1389 ms. Trace:
I20260709 13:01:31.550758  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 521) took 1360 ms. Trace:
I20260709 13:01:31.635061  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:31.662484  1700 rpcz_store.cc:276] 0709 13:01:30.272614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.601766 (+1329152us) service_pool.cc:224] Handling call
0709 13:01:31.661816 (+ 60050us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.662395  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44764.5us
I20260709 13:01:31.675153  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:31.652837  2295 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:31.652849  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.600787  3928 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:31.600593  4015 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.665717  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1962) took 1105 ms. Trace:
I20260709 13:01:31.687949  4015 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.688015  1627 rpcz_store.cc:276] 0709 13:01:30.560607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.533615 (+973008us) service_pool.cc:224] Handling call
0709 13:01:31.665612 (+131997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.689138  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:31.689641  1627 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:31.689904  3868 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:31.664686  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.683894  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1952) took 1263 ms. Trace:
I20260709 13:01:31.684245  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:31.627067  2291 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.074s	user 0.010s	sys 0.002s
I20260709 13:01:31.673091  2288 rpcz_store.cc:276] 0709 13:01:30.190790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.204888 (+ 14098us) service_pool.cc:224] Handling call
0709 13:01:31.550692 (+1345804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.687204  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:31.562920  3924 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.810418  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.911859  1626 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:31.912611  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1960) took 1380 ms. Trace:
I20260709 13:01:31.913067  1626 rpcz_store.cc:276] 0709 13:01:30.532278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.810165 (+1277887us) service_pool.cc:224] Handling call
0709 13:01:31.912535 (+102370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.627570  3907 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:31.691748  3529 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:31.917527  4024 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:31.918246  3529 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.839421  3191 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:31.932632  1634 rpcz_store.cc:276] 0709 13:01:30.420231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.652490 (+1232259us) service_pool.cc:224] Handling call
0709 13:01:31.683790 (+ 31300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.936883  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 261795us
I20260709 13:01:31.937987  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:31.938880  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.940014  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.940644  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
W20260709 13:01:31.940940  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 305950us
I20260709 13:01:31.661265  3895 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:31.941691  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.943821  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:31.920650  3529 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:31.946697  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:31.953244  3191 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:31.827105  4040 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.139s	user 0.001s	sys 0.009s
W20260709 13:01:31.943331  3934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.067s	user 0.015s	sys 0.059s
I20260709 13:01:31.957289  4040 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:31.957808  3934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.081s	user 0.015s	sys 0.059s
I20260709 13:01:31.809016  3724 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:31.960240  1551 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:31.815256  2290 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:31.960598  3868 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:31.664342  3777 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.814952  3937 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.672632  3870 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.963078  3870 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:31.963440  3870 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.963593  3937 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:31.963857  3870 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:31.964094  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:31.964669  1554 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:31.961247  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 779) took 3536 ms. Trace:
I20260709 13:01:31.825529  3818 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:31.964257  3937 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:31.920634  4024 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:31.664736  3981 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:31.956125  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:31.761329  3956 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.002s
I20260709 13:01:31.655264  3767 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.184924  1629 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:32.179338  3191 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.834033  4015 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.189169  3937 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.800544  4005 consensus_queue.cc:1243] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:31.685880  3998 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.965194  2290 rpcz_store.cc:276] 0709 13:01:28.425183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.504044 (+ 78861us) service_pool.cc:224] Handling call
0709 13:01:28.615675 (+111631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.428431 (+1812756us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b58007e05d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9400aa 00007f70ce9c2447 00007f70ce9dafe7 00007f70ce9dc7de 00007f70ce9e6b49 00007f70ce9e7416 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4
0709 13:01:31.960893 (+1532462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.961166 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84440960}
I20260709 13:01:31.800582  4072 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.185526  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:32.190238  3956 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.135854  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1991) took 1740 ms (client timeout 1991 ms). Trace:
I20260709 13:01:31.944756  2291 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
W20260709 13:01:31.944466  3930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.549s	user 0.013s	sys 0.025s
I20260709 13:01:31.760180  3850 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.686034  3944 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.192267  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:32.177452  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 230813us
W20260709 13:01:32.048341  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.216284  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.683751  3687 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:32.180620  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 425538us
W20260709 13:01:32.157609  1523 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.653s (SENT)
I20260709 13:01:31.829561  3948 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.194659  3191 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.155012  1627 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:32.218161  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60600.6us
I20260709 13:01:32.218351  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1961) took 1672 ms. Trace:
I20260709 13:01:32.218770  1627 rpcz_store.cc:276] 0709 13:01:30.546405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.688871 (+1142466us) service_pool.cc:224] Handling call
0709 13:01:32.218295 (+529424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.900413  4023 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:32.198050  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 241970us
I20260709 13:01:31.965524  4086 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:31.975124  3750 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:32.202140  3998 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.205348  1700 rpcz_store.cc:269] 0709 13:01:30.395664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.674102 (+1278438us) service_pool.cc:224] Handling call
0709 13:01:32.135744 (+461642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.104708  3895 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:32.202610  1598 leader_election.cc:336] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.974s (SENT)
I20260709 13:01:31.951004  4029 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:31.961254  3868 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.965785  4031 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.971921  4040 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:32.224906  3944 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.230614  4029 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:31.692750  4008 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:32.192381  4034 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:32.217500  2293 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:32.228955  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.235455  1700 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:32.224671  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.236207  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1252) took 1963 ms (client timeout 2135 ms). Trace:
I20260709 13:01:32.216150  3870 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.209610  3930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.822s	user 0.014s	sys 0.026s
I20260709 13:01:31.978327  4033 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.232522  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:32.224169  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.201943  4084 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:32.236562  1700 rpcz_store.cc:269] 0709 13:01:30.272852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.228705 (+1955853us) service_pool.cc:224] Handling call
0709 13:01:32.236152 (+  7447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.205104  3777 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.199450  3937 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:32.231980  4008 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:32.239336  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.260277  3191 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.205936  4072 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.261467  1627 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:32.262197  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.226756  3956 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.230458  4031 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.229705  3895 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:32.263087  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1985) took 1927 ms (client timeout 2073 ms). Trace:
I20260709 13:01:32.262239  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:32.286005  2296 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:32.281793  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:32.293880  1700 rpcz_store.cc:269] 0709 13:01:30.335807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.261942 (+1926135us) service_pool.cc:224] Handling call
0709 13:01:32.263029 (+  1087us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.267227  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:32.209406  2291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 523) took 1804 ms (client timeout 1618 ms). Trace:
I20260709 13:01:32.296869  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.297004  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:32.298524  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.306291  1704 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.306946  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1989) took 1928 ms. Trace:
I20260709 13:01:32.307333  1704 rpcz_store.cc:276] 0709 13:01:30.378332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.298223 (+1919891us) service_pool.cc:224] Handling call
0709 13:01:32.306846 (+  8623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.309126  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.309846  1704 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.310411  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1988) took 1943 ms. Trace:
I20260709 13:01:32.310813  1704 rpcz_store.cc:276] 0709 13:01:30.367011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.308782 (+1941771us) service_pool.cc:224] Handling call
0709 13:01:32.310350 (+  1568us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.241662  2288 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:32.304838  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.315917  2294 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:32.306351  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.224632  4024 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.261274  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:32.314339  1704 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:32.316125  1700 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
W20260709 13:01:32.316627  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1992) took 1882 ms (client timeout 2202 ms). Trace:
I20260709 13:01:32.261847  4008 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:32.288345  4072 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:32.320676  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1994) took 1802 ms (client timeout 2000 ms). Trace:
W20260709 13:01:32.320647  1702 rpcz_store.cc:269] 0709 13:01:30.435087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.296558 (+1861471us) service_pool.cc:224] Handling call
0709 13:01:32.316577 (+ 20019us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.321259  1700 rpcz_store.cc:269] 0709 13:01:30.518681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.300220 (+1781539us) service_pool.cc:224] Handling call
0709 13:01:32.320617 (+ 20397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.270886  3918 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:32.228368  3998 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.233218  3850 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:32.285634  4033 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.230420  3868 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.272225  3948 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.324630  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1249) took 2052 ms. Trace:
I20260709 13:01:32.330255  1704 rpcz_store.cc:276] 0709 13:01:30.272082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.311562 (+2039480us) service_pool.cc:224] Handling call
0709 13:01:32.314082 (+  2520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.324380 (+ 10298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.324579 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.235354  4023 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.318439  3777 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:32.319024  3981 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.317955  4089 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:32.338786  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 297) took 2038 ms. Trace:
I20260709 13:01:32.339179  1553 rpcz_store.cc:276] 0709 13:01:30.300798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.300910 (+   112us) service_pool.cc:224] Handling call
0709 13:01:30.301255 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.553973 (+252718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.554000 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.019910 (+465910us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.897848 (+877938us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:31.897904 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.210038 (+312134us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54000771d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:32.210082 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.333794 (+123712us) raft_consensus.cc:1685] finished
0709 13:01:32.333814 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.338709 (+  4895us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":615,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":49671,"spinlock_wait_cycles":580364544,"thread_start_us":484355,"threads_started":1,"wal-append.queue_time_us":1118954}
I20260709 13:01:32.226154  4086 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.238842  4034 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.344102  3290 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:32.344573  3290 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:32.344756  4086 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:32.345499  3290 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.346683  3290 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:32.352327  1627 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:32.353272  2296 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:32.358016  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1934) took 4191 ms. Trace:
W20260709 13:01:32.302441  2291 rpcz_store.cc:269] 0709 13:01:30.404914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.417560 (+ 12646us) service_pool.cc:224] Handling call
0709 13:01:32.209299 (+1791739us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.364758  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.360512  2292 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:32.359891  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 792) took 2044 ms (client timeout 2052 ms). Trace:
I20260709 13:01:32.324899  3918 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:32.360479  1627 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:32.361042  1633 rpcz_store.cc:276] 0709 13:01:28.166542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.866934 (+700392us) service_pool.cc:224] Handling call
0709 13:01:28.867217 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.449292 (+2582075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.449329 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.449390 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:31.449449 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.357788 (+908339us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54001c34d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:32.357831 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.357955 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":919681,"raft.queue_time_us":888221,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7326,"spinlock_wait_cycles":3906983680}
I20260709 13:01:32.320158  4024 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.365938  2293 rpcz_store.cc:269] 0709 13:01:30.315163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.387349 (+ 72186us) service_pool.cc:224] Handling call
0709 13:01:32.359630 (+1972281us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:32.359821 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4165697792}
I20260709 13:01:32.354367  3937 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.370180  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:32.360360  3993 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 3.176s	user 0.025s	sys 0.031s
I20260709 13:01:32.367080  4089 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.367588  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:32.388105  1553 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:32.387079  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.396502  1560 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:32.398854  1633 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:32.400059  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:32.400964  1633 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:32.404223  1560 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:32.410807  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:32.409112  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:32.416543  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.417315  1553 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:32.418362  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.419335  1553 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:32.384038  3868 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.421314  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.424883  1633 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:32.422657  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:32.424986  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.425678  1553 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.425864  1560 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:32.427238  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.428557  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.429790  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.410036  3767 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:32.450294  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 787250us
I20260709 13:01:32.455291  1553 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:32.458549  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:32.459079  1553 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:32.423512  3970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.028s	sys 0.023s
I20260709 13:01:32.420979  4015 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:32.413830  2288 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:32.465526  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:32.475324  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:32.473798  1553 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:32.476971  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.482635  1633 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.482506  3191 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:32.444757  3777 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:32.397593  4089 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.470642  3970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.028s	sys 0.023s
I20260709 13:01:32.492488  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1231) took 4512 ms. Trace:
I20260709 13:01:32.492909  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.493738  1633 raft_consensus.cc:2376] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 3. Re-sending same reply.
I20260709 13:01:32.493085  1708 rpcz_store.cc:276] 0709 13:01:27.968010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.031842 (+ 63832us) service_pool.cc:224] Handling call
0709 13:01:28.032199 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.032594 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.032622 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.042604 (+  9982us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.372496 (+329892us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:28.372556 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.372654 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.328572 (+3955918us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:32.473242 (+144670us) raft_consensus.cc:1685] finished
0709 13:01:32.473263 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.480359 (+  7096us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1159,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10260,"spinlock_wait_cycles":142883584,"thread_start_us":249465,"threads_started":1,"wal-append.queue_time_us":1119590}
I20260709 13:01:32.466199  3937 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:32.505625  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.465452  3981 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:01:32.471973  4092 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:01:32.509758  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.523890  1553 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:32.432137  3918 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.502059  4034 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.519155  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1987) took 2169 ms. Trace:
I20260709 13:01:32.535207  1708 rpcz_store.cc:276] 0709 13:01:30.349391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.501761 (+2152370us) service_pool.cc:224] Handling call
0709 13:01:32.518423 (+ 16662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.518687 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.518724 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.518782 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.518846 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.518920 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.519072 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.537914  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62668.6us
I20260709 13:01:32.492671  3952 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:01:32.539866  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:32.528144  3918 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:32.552381  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:32.553117  1633 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.558597  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:32.559293  1553 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.563997  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44858 (request call id 86) took 2160 ms. Trace:
I20260709 13:01:32.564489  1708 rpcz_store.cc:276] 0709 13:01:30.403452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.536589 (+2133137us) service_pool.cc:224] Handling call
0709 13:01:32.563924 (+ 27335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.565523  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.566357  1633 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:32.528414  4077 consensus_queue.cc:1175] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "444b494bb53842b59408a7230884398a" responder_term: 3 status { last_received { term: 2 index: 4 } last_committed_idx: 4 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:32.562641  3777 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.578048  4009 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:32.528576  4095 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:01:32.558640  4034 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:32.586910  1553 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:32.588135  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.588644  1553 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:32.557225  4096 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:32.595993  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 313) took 2273 ms. Trace:
I20260709 13:01:32.596487  1702 rpcz_store.cc:276] 0709 13:01:30.322311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.321595 (+1999284us) service_pool.cc:224] Handling call
0709 13:01:32.321953 (+   358us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:32.331789 (+  9836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.331821 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:32.338070 (+  6249us) log.cc:844] Serialized 78 byte log entry
0709 13:01:32.467500 (+129430us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.560313 (+ 92813us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.595643 (+ 35330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.595708 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.595739 (+    31us) raft_consensus.cc:1685] finished
0709 13:01:32.595748 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.595930 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28973,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5693,"thread_start_us":185533,"threads_started":3,"wal-append.queue_time_us":176846}
I20260709 13:01:32.602628  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1986) took 2258 ms. Trace:
I20260709 13:01:32.604075  4024 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.604254  4092 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:32.604741  4092 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:32.605633  4092 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:32.607084  4060 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.073s	user 0.011s	sys 0.008s
I20260709 13:01:32.603071  1704 rpcz_store.cc:276] 0709 13:01:30.344491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.331306 (+1986815us) service_pool.cc:224] Handling call
0709 13:01:32.331820 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:32.421922 (+ 90102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.421965 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:32.422624 (+   659us) log.cc:844] Serialized 52 byte log entry
0709 13:01:32.483171 (+ 60547us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:32.483222 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.483290 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.602394 (+119104us) raft_consensus.cc:1685] finished
0709 13:01:32.602418 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.602566 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33001,"spinlock_wait_cycles":10952960,"thread_start_us":97802,"threads_started":2,"wal-append.queue_time_us":97247}
I20260709 13:01:32.610883  1633 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:32.611624  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:32.612103  1633 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:32.612344  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.613719  1553 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:32.628918  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 319) took 4700 ms. Trace:
I20260709 13:01:32.586330  4034 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:32.665954  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 344) took 2459 ms. Trace:
I20260709 13:01:32.584378  3981 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:32.673897  1633 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.672536  1635 rpcz_store.cc:276] 0709 13:01:30.206365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.221114 (+ 14749us) service_pool.cc:224] Handling call
0709 13:01:30.221416 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.335591 (+114175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.335624 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.339056 (+  3432us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.464780 (+1125724us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:31.464816 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.464879 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.606495 (+1141616us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:32.656740 (+ 50245us) raft_consensus.cc:1685] finished
0709 13:01:32.656771 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.665853 (+  9082us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180783,"raft.queue_time_us":116728,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":4706048,"thread_start_us":1125341,"threads_started":1,"wal-append.queue_time_us":1142755}
I20260709 13:01:32.675278  1635 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:32.673295  3981 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:32.676455  3981 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.677196  1635 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:32.679025  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 1984) took 2351 ms. Trace:
I20260709 13:01:32.679467  1700 rpcz_store.cc:276] 0709 13:01:30.327836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.321905 (+1994069us) service_pool.cc:224] Handling call
0709 13:01:32.322164 (+   259us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:32.420152 (+ 97988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.420190 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:32.423061 (+  2871us) log.cc:844] Serialized 52 byte log entry
0709 13:01:32.549870 (+126809us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.549919 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.550184 (+   265us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.678789 (+128605us) raft_consensus.cc:1685] finished
0709 13:01:32.678811 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.678962 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3178,"raft.queue_time_us":38313,"spinlock_wait_cycles":10691968,"thread_start_us":173546,"threads_started":2,"wal-append.queue_time_us":124483}
I20260709 13:01:32.667723  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.683238  1553 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6. Re-sending same reply.
I20260709 13:01:32.684782  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.687242  1553 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:32.689018  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:32.668512  1629 rpcz_store.cc:276] 0709 13:01:27.928150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.928506 (+   356us) service_pool.cc:224] Handling call
0709 13:01:27.928768 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.194938 (+4266170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.628564 (+433626us) spinlock_profiling.cc:243] Waited 4.26 s on lock 0x7b54001432d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:32.628805 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9787609344}
I20260709 13:01:32.695279  1553 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:32.667102  4105 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.673943  4034 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.695099  4008 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.702678  4089 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.708258  3845 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.742903  4006 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:32.745473  4101 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:01:32.748435  3938 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.757817  4023 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.759739  3924 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.761392  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.762234  1700 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.764176  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:32.771919  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:32.774999  1704 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.763684  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.771976  4068 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:32.788303  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:32.788903  4068 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.789476  2292 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:32.791283  4068 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.703846  4105 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:32.768283  4031 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:32.793195  4105 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:32.769191  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.794725  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.795675  4008 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.795810  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.774462  3687 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:32.796984  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.797086  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.797560  1702 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:32.798126  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.798360  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.798656  1708 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:32.798964  1702 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:32.776772  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.800061  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:32.800594  1551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1265) took 2419 ms (client timeout 2738 ms). Trace:
I20260709 13:01:32.777585  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:32.801028  1551 rpcz_store.cc:269] 0709 13:01:30.381476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.386398 (+  4922us) service_pool.cc:224] Handling call
0709 13:01:32.800406 (+2414008us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b540030a2d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:32.800531 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122910464}
I20260709 13:01:32.801465  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:32.802496  1551 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:32.803850  3527 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.804620  1551 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:32.804352  3527 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.809042  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:32.810868  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:32.812522  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.796124  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.784461  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 549) took 2500 ms. Trace:
I20260709 13:01:32.814296  4086 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.814541  2289 rpcz_store.cc:276] 0709 13:01:30.284131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.025119 (+740988us) service_pool.cc:224] Handling call
0709 13:01:31.025466 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.665171 (+639705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.784179 (+1119008us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54006c02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:32.784378 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312076672}
W20260709 13:01:32.778496  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.816174  2296 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:32.816669  2296 pending_rounds.cc:85] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 7
I20260709 13:01:32.816958  2296 pending_rounds.cc:107] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:32.817234  2296 raft_consensus.cc:2889] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.817911  1704 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:32.784662  3851 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:32.832784  3851 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:32.833101  4063 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.284s	user 0.011s	sys 0.010s
I20260709 13:01:32.833629  3851 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.834438  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.834673  3851 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.835346  4029 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:32.837138  4029 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:32.840144  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.840860  1704 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:01:32.841894  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.842424  1704 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.844065  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.844671  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:32.842451  1706 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:32.845767  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.846772  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.848021  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1219) took 5090 ms. Trace:
I20260709 13:01:32.848532  1709 rpcz_store.cc:276] 0709 13:01:27.757232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.923499 (+166267us) service_pool.cc:224] Handling call
0709 13:01:27.923869 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.926993 (+  3124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.927019 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.940049 (+ 13030us) log.cc:844] Serialized 26 byte log entry
0709 13:01:30.568245 (+2628196us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b18004366c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:31.412620 (+844375us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:31.412660 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.412742 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.832357 (+1419615us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:32.847705 (+ 15348us) raft_consensus.cc:1685] finished
0709 13:01:32.847735 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.847938 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2746,"raft.queue_time_us":3222,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8299,"spinlock_wait_cycles":41915904,"thread_start_us":377797,"threads_started":1,"wal-append.queue_time_us":3585194}
I20260709 13:01:32.850965  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.856345  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:32.858247  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:32.861785  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.862594  1700 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.864904  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.866461  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.877331  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.878304  1709 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:32.880192  1552 raft_consensus.cc:2410] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:32.886158  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.886305  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:32.898041  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.885622  4086 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:32.874524  4006 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:32.902951  1703 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:32.846685  4029 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:01:32.892325  3924 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:01:32.892325  4068 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:32.914016  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.914330  1703 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:32.915670  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.916451  1703 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:32.874753  3952 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:01:32.902978  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:32.891386  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:32.925787  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.903065  4091 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:32.941316  1673 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.693s (SENT)
I20260709 13:01:32.962042  3870 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:32.909343  4009 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:32.907486  4086 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.963980  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1967) took 1023 ms. Trace:
I20260709 13:01:32.927052  1709 raft_consensus.cc:2376] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:01:32.898758  3928 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:01:32.920799  3290 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:01:32.986675  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.987600  1705 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:32.972977  1633 rpcz_store.cc:276] 0709 13:01:31.940165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.674667 (+734502us) service_pool.cc:224] Handling call
0709 13:01:32.674962 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.675158 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:32.709096 (+ 33938us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.963616 (+254520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.963666 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:32.963717 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.963805 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.963923 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35,"prepare.queue_time_us":41130,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":51,"raft.queue_time_us":6668,"thread_start_us":287637,"threads_started":2}
I20260709 13:01:32.989351  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.990262  1709 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:32.972636  3687 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:32.982882  4086 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:32.973531  4009 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.985800  3290 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:32.997973  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.999985  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.002351  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 100319us
I20260709 13:01:32.926949  4068 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:32.985628  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.012866  1708 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:33.013334  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:32.970788  4091 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.014171  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:33.020109  4084 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.000s
I20260709 13:01:33.020565  4084 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:33.023978  3290 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.025033  3290 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:33.026486  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.028877  1555 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:33.029709  3944 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.031314  4006 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:33.034678  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:33.035356  2290 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.015496  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.036051  1709 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:33.036944  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.037540  2292 raft_consensus.cc:2376] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4. Re-sending same reply.
I20260709 13:01:33.038525  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.040397  2293 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:32.999317  4009 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:33.019130  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.017139  1703 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:33.041347  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:33.043241  2293 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.039532  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:33.044097  2292 raft_consensus.cc:2376] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5. Re-sending same reply.
I20260709 13:01:33.046185  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:33.046845  2292 raft_consensus.cc:2376] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7. Re-sending same reply.
I20260709 13:01:33.047837  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.048604  2292 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
W20260709 13:01:33.050822  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.070892  2293 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:33.058596  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.089726  1705 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.060277  3767 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.058920  4006 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.092394  3290 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:33.092795  3290 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:33.092682  4006 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:33.093701  3290 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:33.094159  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.094880  4006 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.064878  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.065352  1631 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:33.095979  3981 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:33.096483  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.096583  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.097149  1633 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:33.097848  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.098483  1633 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:33.065821  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.065236  2291 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:33.071564  2292 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:33.055748  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.075114  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.094905  3290 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.100315  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
W20260709 13:01:33.101264  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 292) took 2864 ms (client timeout 1575 ms). Trace:
I20260709 13:01:33.101334  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:33.101773  1552 rpcz_store.cc:269] 0709 13:01:30.236938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.237957 (+  1019us) service_pool.cc:224] Handling call
0709 13:01:33.101052 (+2863095us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:33.101193 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6073367936}
I20260709 13:01:33.102560  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.075385  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:33.076669  2262 connection.cc:256] RPC call timeout handler was delayed by -1.350280865000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:01:33.104029  2262 connection.cc:256] RPC call timeout handler was delayed by -1.1180808650000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:01:33.104476  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.106820  1629 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:33.107664  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.107829  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.108146  1558 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:33.080436  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.108870  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:33.085537  4068 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.109483  1709 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:33.043063  4117 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:01:33.110088  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1277) took 2648 ms. Trace:
I20260709 13:01:33.110510  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.110442  1558 rpcz_store.cc:276] 0709 13:01:30.462025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.034030 (+572005us) service_pool.cc:224] Handling call
0709 13:01:33.107845 (+2073815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.108501 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.109920 (+  1419us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54000e8b50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:33.110032 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4768726784}
I20260709 13:01:33.111083  4117 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:33.111533  2297 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:33.111832  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.111610  4117 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:33.112576  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 312) took 2797 ms (client timeout 2053 ms). Trace:
W20260709 13:01:33.113075  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 518) took 2923 ms (client timeout 1577 ms). Trace:
W20260709 13:01:33.112987  1706 rpcz_store.cc:269] 0709 13:01:30.314695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.660258 (+1345563us) service_pool.cc:224] Handling call
0709 13:01:33.112415 (+1452157us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54001342d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:33.112507 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900766464}
I20260709 13:01:33.112618  2291 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:33.113632  2295 rpcz_store.cc:269] 0709 13:01:30.189913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.194180 (+  4267us) service_pool.cc:224] Handling call
0709 13:01:33.112879 (+2918699us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400350054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:33.112979 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3352620800}
I20260709 13:01:33.114271  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.112416  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.115022  1706 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.088582  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.110505  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.116084  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.116235  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.116789  1708 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.117244  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.117589  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.117869  1709 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:33.118629  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.099238  1634 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:33.120112  1634 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:33.068073  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.122325  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 1971) took 1176 ms. Trace:
I20260709 13:01:33.122689  1627 rpcz_store.cc:276] 0709 13:01:31.945594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.281501 (+335907us) service_pool.cc:224] Handling call
0709 13:01:33.122167 (+840666us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b54003435d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:33.122271 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132404608}
I20260709 13:01:33.124054  1705 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
W20260709 13:01:33.124625  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.124959  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.125541  1705 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:33.126291  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.126945  1705 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.127192  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.087819  1704 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:33.094072  1707 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:33.128026  1551 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.128111  3937 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.128352  1634 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:33.109879  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:33.118700  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.129571  2292 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:33.131026  1630 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:33.108011  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.103533  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.131790  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.132573  1556 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:33.133260  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.133666  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:33.107699  1632 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:33.079365  2290 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:33.141559  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:33.138149  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.141701  3290 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:33.141842  1556 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.137641  1558 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:33.142278  3290 consensus_queue.cc:1243] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:33.143257  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.147097  1700 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:33.148092  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.148286  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.148793  1556 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 1.
I20260709 13:01:33.149129  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.149555  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:33.143900  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173320us
I20260709 13:01:33.150157  1556 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:33.150923  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.128868  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.157964  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.154104  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.155639  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1262) took 2775 ms. Trace:
I20260709 13:01:33.159246  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.160871  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.159536  1559 rpcz_store.cc:276] 0709 13:01:30.380603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.384879 (+  4276us) service_pool.cc:224] Handling call
0709 13:01:30.385119 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.385315 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:30.560920 (+175605us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.628130 (+2067210us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b1800403e40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a
0709 13:01:33.155332 (+527202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.155381 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:33.155436 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.155493 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.155585 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":487818,"prepare.queue_time_us":1075916,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":4183,"raft.queue_time_us":489887,"spinlock_wait_cycles":2896768,"thread_start_us":1065157,"threads_started":2,"wal-append.queue_time_us":910090,"wal-append.run_cpu_time_us":1176,"wal-append.run_wall_time_us":1077440}
I20260709 13:01:33.162407  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.163326  1559 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.155275  3290 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:33.153986  1558 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.156701  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:33.170138  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.157613  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.140830  3924 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:33.147601  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:33.174154  1558 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:01:33.174930  1675 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:33.167074  1631 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:33.167183  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.177438  1633 raft_consensus.cc:2368] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6.
I20260709 13:01:33.168017  1560 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.168341  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.170241  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.180374  1559 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.180603  1634 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:33.128682  3937 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.172889  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.156967  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.185670  1555 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:33.113909  4117 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.190357  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.191440  1552 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.155931  4006 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:33.206470  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.207365  3290 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:33.207885  3290 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.208253  3290 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.187399  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.189182  1555 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:33.183030  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.191728  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.171186  4008 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:33.158645  4004 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.195335  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43619.7us
I20260709 13:01:33.220018  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.198324  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.198802  3850 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:33.199395  4021 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:33.158306  4034 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.201643  3937 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.173849  3750 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.191002  3702 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:33.189927  3657 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:33.215584  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:33.247818  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.248656  1628 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:33.249995  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.224256  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.173823  1551 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:33.225117  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.227344  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.227949  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.232061  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.224447  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.257560  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.258469  1559 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:33.258664  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.259505  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.254774  1634 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:33.262526  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.232824  4059 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.671s	user 0.005s	sys 0.039s
I20260709 13:01:33.262928  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.266976  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.263621  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.267997  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:33.264045  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 258061us
I20260709 13:01:33.264250  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.265455  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.267645  1631 raft_consensus.cc:2368] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:33.269639  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.271653  4119 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.272028  3527 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:33.225919  4008 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.277659  4008 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.281167  4034 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:33.285344  4023 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:33.285660  3980 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:33.285989  4023 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:33.286212  3980 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:33.286928  4023 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.266440  4024 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:33.291236  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.289454  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.292863  1703 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:33.294252  1559 raft_consensus.cc:1217] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-6.7]   Dedup: 3.5->[4.6-6.7]
I20260709 13:01:33.297801  1559 pending_rounds.cc:85] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:01:33.299306  1559 pending_rounds.cc:107] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:01:33.293004  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.295656  3290 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:33.304781  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.305594  1703 raft_consensus.cc:2376] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7. Re-sending same reply.
I20260709 13:01:33.306931  3290 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.307305  2260 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:33.247318  4084 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:33.309042  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.192440  3924 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.268088  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:33.253172  3657 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:33.272202  4119 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.287237  3980 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.232568  4040 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:01:33.232704  4004 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.521095  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.412360  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.523524  2260 leader_election.cc:403] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:33.304234  1559 raft_consensus.cc:2889] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:33.524194  1709 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:33.406286  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1266) took 3024 ms. Trace:
I20260709 13:01:33.465343  2294 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:33.525071  1554 rpcz_store.cc:276] 0709 13:01:30.381736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.382098 (+   362us) service_pool.cc:224] Handling call
0709 13:01:30.382351 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.965006 (+1582655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.405895 (+1440889us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000e9554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:33.406189 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3638580480}
I20260709 13:01:33.526620  1555 raft_consensus.cc:2376] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5. Re-sending same reply.
I20260709 13:01:33.299664  3937 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:33.517222  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.481521  3687 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:33.528156  3687 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:33.432269  2292 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:33.321478  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 214366us
I20260709 13:01:33.517599  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1268) took 3135 ms. Trace:
I20260709 13:01:33.478168  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.530551  1628 raft_consensus.cc:2368] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:33.311141  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.362917  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 192825us
I20260709 13:01:33.531404  1708 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:33.532320  4122 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
I20260709 13:01:33.532433  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.533041  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 225865us
I20260709 13:01:33.533643  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:33.527076  3657 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.534405  4122 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:33.534746  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.534917  4122 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:33.536221  1634 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:01:33.267195  4059 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.749s	user 0.005s	sys 0.039s
I20260709 13:01:33.326236  1702 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
W20260709 13:01:33.537535  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.338241  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.290740  3800 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:33.519840  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.540318  1700 raft_consensus.cc:2376] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:01:33.524012  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1980) took 1103 ms. Trace:
I20260709 13:01:33.541190  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.542194  1700 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:33.462750  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.544109  1704 raft_consensus.cc:2376] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:01:33.545297  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.545468  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.546180  1704 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
W20260709 13:01:33.545697  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 281490us
I20260709 13:01:33.567732  3290 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.568605  3290 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.569484  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.569744  1631 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:33.521837  1633 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:33.531425  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.547292  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:33.554478  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:33.571992  1628 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.352082  4118 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:33.573596  4077 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.574213  4077 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.574709  3290 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.522987  3846 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:33.581063  4068 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:33.581497  4068 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.581831  4068 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.529978  1557 rpcz_store.cc:276] 0709 13:01:30.382192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.387017 (+  4825us) service_pool.cc:224] Handling call
0709 13:01:30.387286 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.166435 (+2779149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.166483 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.166542 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:33.166606 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.352878 (+186272us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b540036f6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:33.352940 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.517537 (+164597us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":397953,"raft.queue_time_us":405468,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":11880,"spinlock_wait_cycles":5475286912}
I20260709 13:01:33.308293  4008 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:33.547474  2262 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5.
W20260709 13:01:33.553400  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 244511us
I20260709 13:01:33.541693  1553 rpcz_store.cc:276] 0709 13:01:32.420635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.698237 (+277602us) service_pool.cc:224] Handling call
0709 13:01:32.698592 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.698883 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:33.045446 (+346563us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.523632 (+478186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.523684 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:33.523740 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.523824 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.523954 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":319095,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":35,"raft.queue_time_us":13988,"thread_start_us":809524,"threads_started":2}
W20260709 13:01:33.556892  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 260650us
I20260709 13:01:33.560166  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.597673  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.561405  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:33.528618  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.310873  4024 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:33.574512  4119 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.641520  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.384169  4121 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:33.643245  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.538980  4084 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:33.645476  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76069.4us
I20260709 13:01:33.646962  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.537626  3970 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:33.533697  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.651137  1708 raft_consensus.cc:2376] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
W20260709 13:01:33.631963  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45242.5us
I20260709 13:01:33.652565  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:33.633764  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:33.654615  1700 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:33.656296  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59823.4us
I20260709 13:01:33.539984  3800 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:33.664258  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.664785  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.664899  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.666107  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:33.653889  1628 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:33.668038  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:33.669726  3657 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.579618  4006 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:33.672834  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.673761  1557 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:33.675029  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.673032  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:33.640215  4009 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.002s
I20260709 13:01:33.678779  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.676890  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:33.680325  1553 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.589715  3937 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:33.634742  4024 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.662987  4098 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:33.682643  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 560) took 1506 ms. Trace:
I20260709 13:01:33.658041  4084 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.655947  3873 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.698252  2296 rpcz_store.cc:276] 0709 13:01:32.176027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.354087 (+178060us) service_pool.cc:224] Handling call
0709 13:01:32.354345 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.817780 (+463435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.682385 (+864605us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:33.682576 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061174144}
I20260709 13:01:33.683224  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.642735  4118 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:33.684001  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.642822  3777 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:33.709309  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.688726  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.712857  1709 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:33.699148  4024 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:33.706066  1702 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:33.681560  4086 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.004s
I20260709 13:01:33.729744  4086 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.734862  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.736249  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.737118  1702 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.737977  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.681463  4040 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.682102  1554 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:33.673799  4119 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:33.724062  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:33.696416  4012 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:33.742893  1629 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:33.722805  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.744388  1627 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:33.703361  4098 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:33.673584  4008 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:01:33.749378  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.756063  1674 leader_election.cc:400] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6.
I20260709 13:01:33.674536  3970 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.710651  1551 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:33.686573  4009 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.764302  1674 leader_election.cc:403] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:33.733510  4118 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:33.732693  4077 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.740401  3777 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:33.761639  3750 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 13:01:33.717469  3873 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.740694  3870 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:33.754594  4012 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.535841  4122 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.763592  4098 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.774806  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:33.779423  3981 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:33.745868  3845 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:01:33.790798  2297 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:33.776957  4004 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.788661  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:47210
W20260709 13:01:33.812848  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.785888  4127 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.818892  2289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.006s	sys 0.010s
I20260709 13:01:33.787881  3750 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:33.831090  1709 raft_consensus.cc:606] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:33.866780  1709 raft_consensus.cc:614] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:33.798703  4012 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.742308  3800 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.827642  4086 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:33.879012  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.861763  4128 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.866179  3981 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6.
I20260709 13:01:33.860366  3657 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:33.785349  3937 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.885402  1709 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:33.864928  3924 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.896533  4031 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:33.895426  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.897317  1709 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:33.899210  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:33.901736  1709 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:33.916818  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:33.917724  1708 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:33.869275  4127 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.922710  3845 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:33.923166  4127 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.923270  3845 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.913705  3657 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:33.875177  4118 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.943125  4084 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:33.943805  4091 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.944468  4009 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:33.911545  3777 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:33.946185  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:34.023962  1703 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:33.947122  1704 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:33.948393  4029 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:33.948629  3970 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:33.870555  4079 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:01:33.977056  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:33.876472  4021 consensus_queue.cc:1175] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b2c3934407d422495728f84374c6b53" responder_term: 7 status { last_received { term: 4 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:33.980019  1708 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
W20260709 13:01:34.040637  4139 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:01:34.041282  1708 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:34.041837  1705 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:34.042203  1705 pending_rounds.cc:85] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:01:34.042435  1705 pending_rounds.cc:107] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:34.042673  1705 raft_consensus.cc:2889] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.042698  1708 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:34.044302  1708 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 13:01:34.045243  4134 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:01:34.046108  4131 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
W20260709 13:01:34.047138  4132 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:01:34.023247  4135 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:01:34.049963  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 568) took 1156 ms. Trace:
I20260709 13:01:34.050372  2289 rpcz_store.cc:276] 0709 13:01:32.893229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.900891 (+  7662us) service_pool.cc:224] Handling call
0709 13:01:32.963117 (+ 62226us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:33.242628 (+279511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.242684 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:33.245751 (+  3067us) log.cc:844] Serialized 52 byte log entry
0709 13:01:33.896202 (+650451us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:33.896285 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.897588 (+  1303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.049733 (+152145us) raft_consensus.cc:1685] finished
0709 13:01:34.049751 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.049904 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120075,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":775854,"threads_started":2,"wal-append.queue_time_us":801805}
I20260709 13:01:34.031644  3918 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:34.052640  3918 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:34.053490  3918 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.054509  3918 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:34.055236  3980 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:34.056327  1705 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:34.034101  4040 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:34.034529  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 504943us
W20260709 13:01:34.034912  4130 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:01:34.036407  3845 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:34.039884  4125 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:01:33.936060  3687 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.061103  4029 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:34.069196  4009 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:33.906632  4128 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.077833  3873 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.091336  4009 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:34.181080  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.010s
W20260709 13:01:34.182078  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.011s	sys 0.010s
W20260709 13:01:34.184507  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 537501us
W20260709 13:01:34.187208  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.008s	sys 0.024s
W20260709 13:01:34.188141  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.009s	sys 0.024s
I20260709 13:01:34.188997  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1305) took 1009 ms. Trace:
W20260709 13:01:34.188894  1598 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.745s (SENT)
I20260709 13:01:34.190156  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1304) took 1020 ms. Trace:
W20260709 13:01:34.187331  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.064s	sys 0.069s
W20260709 13:01:34.192101  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.065s	sys 0.069s
W20260709 13:01:33.971067  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 103391us
W20260709 13:01:34.194382  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.197140  1601 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.552s (SENT)
I20260709 13:01:34.189739  1560 rpcz_store.cc:276] 0709 13:01:33.178920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.179154 (+   234us) service_pool.cc:224] Handling call
0709 13:01:33.204375 (+ 25221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.204695 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.204725 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.205166 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.697479 (+492313us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:33.739476 (+ 41997us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.739736 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.739833 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.188076 (+448243us) raft_consensus.cc:1685] finished
0709 13:01:34.188159 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.188603 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16700,"prepare.queue_time_us":35552,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":148358,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":31630,"thread_start_us":491821,"threads_started":1,"wal-append.queue_time_us":656164}
W20260709 13:01:34.195523  1585 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37587 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37587 timed out after 15.000s (SENT)
I20260709 13:01:34.201354  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 1966) took 2261 ms. Trace:
I20260709 13:01:33.899578  1632 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:34.202143  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 389831us
I20260709 13:01:34.203974  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.201967  1626 rpcz_store.cc:276] 0709 13:01:31.939880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.940028 (+   148us) service_pool.cc:224] Handling call
0709 13:01:31.940433 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.940980 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:31.941021 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.945459 (+  4438us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.477186 (+1531727us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b18000ca3c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:33.964820 (+487634us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:33.964878 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.964977 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.965044 (+    67us) raft_consensus.cc:1685] finished
0709 13:01:33.965062 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.201056 (+235994us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2584,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":3405,"spinlock_wait_cycles":14478720,"thread_start_us":487353,"threads_started":1,"wal-append.queue_time_us":1949874}
I20260709 13:01:34.207160  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:34.190795  1558 rpcz_store.cc:276] 0709 13:01:33.169768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.175570 (+  5802us) service_pool.cc:224] Handling call
0709 13:01:33.175964 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.176234 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.176270 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.283898 (+107628us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b2c00000f48. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:01:33.284357 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.189113 (+904756us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:34.189296 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.189513 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.189638 (+   125us) raft_consensus.cc:1685] finished
0709 13:01:34.189659 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.190075 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20014976,"thread_start_us":903818,"threads_started":1,"wal-append.queue_time_us":811283}
I20260709 13:01:34.210880  1674 leader_election.cc:400] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:34.211254  1674 leader_election.cc:403] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:34.213752  1674 leader_election.cc:400] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
W20260709 13:01:34.019891  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 381201us
I20260709 13:01:34.021535  1706 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:34.216441  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 492252us
I20260709 13:01:34.256608  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1262) took 1201 ms. Trace:
I20260709 13:01:34.256719  4140 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:34.257486  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.218185  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 143577us
W20260709 13:01:34.263195  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77063us
I20260709 13:01:34.264081  4006 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.264554  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.265170  4006 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.266549  4128 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:34.218987  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.028s	sys 0.077s
W20260709 13:01:34.219906  3527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.000s	sys 0.028s
W20260709 13:01:34.271220  3527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.000s	sys 0.028s
I20260709 13:01:34.272568  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:34.273409  4084 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.276413  4059 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:34.276957  4059 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.277459  4059 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:34.281657  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:34.283028  2296 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:34.257093  1706 rpcz_store.cc:276] 0709 13:01:33.055171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.951810 (+896639us) service_pool.cc:224] Handling call
0709 13:01:34.021327 (+ 69517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.255886 (+234559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.256366 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.284209  3870 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:34.221251  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:34.225265  3687 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.285869  3870 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.286748  2297 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:34.288144  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:34.288925  2297 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:34.221799  2289 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:01:34.294598  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:34.075757  2288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.012s	sys 0.016s
W20260709 13:01:34.268896  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.029s	sys 0.077s
I20260709 13:01:34.232120  1674 leader_election.cc:403] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:34.298202  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 84696.9us
I20260709 13:01:34.298375  4029 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.299430  4006 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.300284  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1310) took 1068 ms. Trace:
I20260709 13:01:34.300745  1559 rpcz_store.cc:276] 0709 13:01:33.231808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.293578 (+ 61770us) service_pool.cc:224] Handling call
0709 13:01:33.294018 (+   440us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:33.553134 (+259116us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:33.573381 (+ 20247us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.269388 (+696007us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:34.270470 (+  1082us) log.cc:844] Serialized 52 byte log entry
0709 13:01:34.270690 (+   220us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:34.270748 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.270910 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.299925 (+ 29015us) raft_consensus.cc:1685] finished
0709 13:01:34.299955 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.300202 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91648,"prepare.queue_time_us":19801,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":40,"raft.queue_time_us":194126,"raft.run_cpu_time_us":576,"raft.run_wall_time_us":21357,"thread_start_us":618899,"threads_started":1,"wal-append.queue_time_us":550007}
I20260709 13:01:34.302832  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:34.295636  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:34.303988  4009 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.301082  4098 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.304337  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:34.305164  4006 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:34.282928  3846 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.004s
I20260709 13:01:34.305505  2289 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
W20260709 13:01:34.308089  4121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.005s	sys 0.003s
I20260709 13:01:34.310408  3845 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:34.310513  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.269285  4154 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:34.270398  4144 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:34.314203  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51501.7us
W20260709 13:01:34.325904  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 135755us
I20260709 13:01:34.315066  4006 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:34.361915  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44499.3us
I20260709 13:01:34.373150  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:34.365070  4006 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.300302  4029 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:34.382663  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.317619  4084 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:01:34.383982  4128 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:34.363229  4140 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:01:34.400354  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 90058.1us
I20260709 13:01:34.401098  4153 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:01:34.402791  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.380820  4144 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:34.401386  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:34.402912  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.015s	sys 0.006s
W20260709 13:01:34.408859  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.410568  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.016s	sys 0.006s
I20260709 13:01:34.392071  4154 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.398990  4128 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.424629  2297 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:34.424860  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 157370us
I20260709 13:01:34.426488  3687 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.427009  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.427178  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:34.427687  3687 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.426533  1706 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:34.435712  3687 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:34.436496  4144 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:34.436537  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.006s	sys 0.007s
W20260709 13:01:34.422533  1601 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.719s (SENT)
W20260709 13:01:34.416947  4068 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.014s
I20260709 13:01:34.420224  4128 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:34.437511  3777 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
W20260709 13:01:34.505165  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.505594  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.011s	sys 0.011s
I20260709 13:01:34.475464  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:34.467064  1556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.020s	sys 0.035s
I20260709 13:01:34.417271  4096 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.442718  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 763965us
I20260709 13:01:34.437901  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 527) took 1404 ms. Trace:
W20260709 13:01:34.513842  1556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.020s	sys 0.040s
I20260709 13:01:34.510788  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:34.516773  4012 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.004s
I20260709 13:01:34.517338  4012 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.430482  4140 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:34.524974  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 123767us
W20260709 13:01:34.522675  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 95045.2us
I20260709 13:01:34.512068  4068 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.527614  3928 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.528671  3850 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:34.529628  3924 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.530385  3924 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:34.531041  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 88637.9us
I20260709 13:01:34.533236  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.536761  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1995) took 1315 ms. Trace:
I20260709 13:01:34.537228  1556 rpcz_store.cc:276] 0709 13:01:33.221290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.269373 (+ 48083us) service_pool.cc:224] Handling call
0709 13:01:33.269716 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:33.270082 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.270110 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:33.302956 (+ 32846us) log.cc:844] Serialized 52 byte log entry
0709 13:01:34.536314 (+1233358us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:34.536390 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.536476 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.536518 (+    42us) raft_consensus.cc:1685] finished
0709 13:01:34.536533 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.536677 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":457,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":16256,"thread_start_us":1132477,"threads_started":1,"wal-append.queue_time_us":1122847}
I20260709 13:01:34.548163  2295 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:01:34.553791  3850 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:34.555452  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 128619us
I20260709 13:01:34.557512  4068 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:34.558022  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:34.558495  4137 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:34.559317  4086 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:34.560047  4086 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:34.560467  4086 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:34.569303  4140 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.555663  3850 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:34.569875  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.515380  2288 rpcz_store.cc:276] 0709 13:01:33.032820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.039854 (+  7034us) service_pool.cc:224] Handling call
0709 13:01:33.040177 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.040409 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:33.539363 (+498954us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.436828 (+897465us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:34.436975 (+   147us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:34.437171 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.437352 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.437678 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125820,"prepare.queue_time_us":247683,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":181274,"spinlock_wait_cycles":16357760,"thread_start_us":1201203,"threads_started":2,"wal-append.queue_time_us":570117}
I20260709 13:01:34.523640  4096 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.579998  1701 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 2.
I20260709 13:01:34.532956  3924 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:34.486271  4150 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.214s
I20260709 13:01:34.642119  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:34.597952  4012 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:34.588085  4153 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:34.658998  2292 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:34.647993  3924 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.650592  3928 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.673225  4140 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:34.673768  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:34.675757  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 142748us
I20260709 13:01:34.682126  3846 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:34.678674  3870 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.700407  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1273) took 1592 ms. Trace:
W20260709 13:01:34.705837  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.724769  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.713369  4137 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:34.735597  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 328050us
W20260709 13:01:34.737658  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.739832  4119 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:01:34.742743  4140 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:34.743038  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69460.6us
I20260709 13:01:34.744103  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:34.741130  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 123148us
I20260709 13:01:34.745790  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:34.748117  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:34.750626  4154 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.731703  4144 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:01:34.754199  3924 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.755158  4144 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:34.755623  4144 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:34.756644  4144 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.757447  3924 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.757943  4144 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.701048  1709 rpcz_store.cc:276] 0709 13:01:33.073838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.039637 (+965799us) service_pool.cc:224] Handling call
0709 13:01:34.046591 (+  6954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.046863 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:34.397169 (+350306us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.639835 (+242666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:34.644677 (+  4842us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:34.655512 (+ 10835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.659794 (+  4282us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.666616 (+  6822us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13338,"prepare.queue_time_us":306524,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":66,"raft.queue_time_us":32170,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":10535,"thread_start_us":483465,"threads_started":2}
I20260709 13:01:34.748246  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:34.761261  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.745s (SENT)
I20260709 13:01:34.761808  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:34.771430  4024 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.022s	user 0.006s	sys 0.008s
I20260709 13:01:34.701241  4006 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:34.775547  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 129800us
I20260709 13:01:34.788527  4137 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:34.789175  4137 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:34.790237  4137 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:34.791874  4140 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:34.792347  4140 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:34.792815  3918 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.006s
I20260709 13:01:34.793535  3918 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:34.792682  4031 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:34.767545  4012 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:34.805190  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69846.3us
W20260709 13:01:34.805492  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44286.1us
W20260709 13:01:34.809583  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 227822us
I20260709 13:01:34.799507  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2162) took 1627 ms. Trace:
W20260709 13:01:34.811364  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63403.3us
W20260709 13:01:34.811995  1601 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.787s (SENT)
W20260709 13:01:34.795702  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2155) took 1688 ms (client timeout 1713 ms). Trace:
W20260709 13:01:34.813057  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68743.7us
I20260709 13:01:34.810586  1632 rpcz_store.cc:276] 0709 13:01:33.172382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.172589 (+   207us) service_pool.cc:224] Handling call
0709 13:01:34.799109 (+1626520us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:34.799321 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577897728}
W20260709 13:01:34.813720  1629 rpcz_store.cc:269] 0709 13:01:33.107192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.107591 (+   399us) service_pool.cc:224] Handling call
0709 13:01:34.795314 (+1687723us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54000ccbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:34.795524 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437169536}
W20260709 13:01:34.805985  4059 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.528s	user 0.016s	sys 0.000s
I20260709 13:01:34.816591  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.814790  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1272) took 1590 ms. Trace:
I20260709 13:01:34.795804  4006 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.804006  4148 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:34.804797  3980 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.006s
I20260709 13:01:34.792721  4140 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:34.807488  4031 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:34.839866  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:34.840096  1707 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:34.841552  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:34.824083  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 99461.6us
W20260709 13:01:34.829720  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 84121.9us
I20260709 13:01:34.804299  3800 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:34.847317  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:34.848289  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.786494  4024 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.860184  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 337) took 1484 ms. Trace:
I20260709 13:01:34.860723  1708 rpcz_store.cc:276] 0709 13:01:33.303081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.045136 (+742055us) service_pool.cc:224] Handling call
0709 13:01:34.045642 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.045903 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:34.388623 (+342720us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.708662 (+320039us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.729557 (+ 20895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:34.735184 (+  5627us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:34.757568 (+ 22384us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:01:34.761842 (+  4274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.779604 (+ 17762us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.787001 (+  7397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8766,"prepare.queue_time_us":349887,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":144,"raft.queue_time_us":15714,"raft.run_cpu_time_us":3448,"raft.run_wall_time_us":124661,"spinlock_wait_cycles":3265280,"thread_start_us":510652,"threads_started":2,"wal-append.queue_time_us":356164}
W20260709 13:01:34.864831  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52870.5us
I20260709 13:01:34.864024  3918 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.862802  4149 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:34.868948  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44199.8us
I20260709 13:01:34.808148  4009 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.819486  1704 rpcz_store.cc:276] 0709 13:01:33.073487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.025087 (+951600us) service_pool.cc:224] Handling call
0709 13:01:34.215973 (+190886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.216647 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:34.366399 (+149752us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.657660 (+291261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:34.657887 (+   227us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:34.658102 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.658373 (+   271us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.664159 (+  5786us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136867,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":22987,"spinlock_wait_cycles":10097920,"thread_start_us":395714,"threads_started":2,"wal-append.queue_time_us":386497}
I20260709 13:01:34.849357  4119 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:34.873101  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 125385us
I20260709 13:01:34.873610  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.878247  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:34.881373  1552 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:01:34.884084  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:34.884251  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.885656  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.886986  4012 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.887740  4012 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:34.827468  4059 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.889961  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:34.871251  4149 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:34.896592  1601 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.826s (SENT)
I20260709 13:01:34.875013  4006 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:34.897406  4149 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:34.840023  4031 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.900308  3918 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:34.896443  4086 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:34.878924  4128 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:34.928436  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 85507.9us
I20260709 13:01:34.907882  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:34.918727  1601 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.974s (SENT)
W20260709 13:01:34.945684  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.947526  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.928328  4191 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.931305  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2166) took 1704 ms. Trace:
I20260709 13:01:34.901722  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:34.953128  1627 rpcz_store.cc:276] 0709 13:01:33.226873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.227119 (+   246us) service_pool.cc:224] Handling call
0709 13:01:34.930924 (+1703805us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:34.931099 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011880192}
W20260709 13:01:34.914983  4184 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
I20260709 13:01:34.909246  4009 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:34.930588  4150 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:34.959096  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.965632  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.936790  4153 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:34.928711  4119 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:34.930464  4086 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:34.910894  4059 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:34.952857  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 86093.6us
I20260709 13:01:34.958006  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.916186  3924 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:01:34.969273  1673 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.577s (SENT)
I20260709 13:01:34.972064  4004 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.977515  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74112.9us
I20260709 13:01:34.963949  2289 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.922780  3767 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:01:34.965386  4150 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:34.913136  4140 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:34.987388  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:34.990834  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:34.993391  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:34.924723  4005 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:01:34.996295  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:34.976171  3846 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:35.009153  4150 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.009635  3846 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.010623  4150 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.011889  3850 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.012085  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 305) took 1792 ms (client timeout 1982 ms). Trace:
I20260709 13:01:35.012514  3850 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:35.012738  1555 rpcz_store.cc:269] 0709 13:01:33.219439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.219642 (+   203us) service_pool.cc:224] Handling call
0709 13:01:34.979590 (+1759948us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002efed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:35.011867 (+ 32277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699846400}
W20260709 13:01:35.014850  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 1999) took 2259 ms (client timeout 1926 ms). Trace:
I20260709 13:01:35.015141  4004 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:35.016302  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 97620.9us
W20260709 13:01:35.017045  1601 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.061s (SENT)
I20260709 13:01:35.018147  4004 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.019205  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:34.992872  3918 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.020447  4153 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:35.020685  3918 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.021883  4153 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:35.022174  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:35.023010  1705 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.023550  3850 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:35.023727  4178 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.345s	user 0.001s	sys 0.001s
I20260709 13:01:35.026418  2289 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:35.032864  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.034178  4098 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.034754  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.037040  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:35.037977  1558 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:35.039646  1702 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:34.986294  4119 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.002426  1556 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:35.002641  4146 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:35.003057  4192 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.003500  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.003963  4185 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:01:35.005218  4148 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:34.979552  3924 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.008128  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:35.008877  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.022248  4153 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.039234  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.040922  4031 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.042550  4096 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.043159  1560 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:35.043548  4009 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.042261  4024 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.020468  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:35.001682  4161 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:01:34.999966  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 104624us
I20260709 13:01:34.952241  4191 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.047783  3846 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.129038  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:35.129304  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110245us
I20260709 13:01:35.130123  4191 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.130542  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.130841  4191 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:35.100179  1700 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:35.043944  4012 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:35.001093  4128 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:35.135919  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.052765  4146 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.135288  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:35.015389  1701 rpcz_store.cc:269] 0709 13:01:32.755132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.763305 (+  8173us) service_pool.cc:224] Handling call
0709 13:01:35.014615 (+2251310us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540016c454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:35.014759 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4132396288}
W20260709 13:01:35.136611  2263 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.982s (SENT)
I20260709 13:01:35.145437  1706 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:35.147948  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 159517us
I20260709 13:01:35.151288  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.126322  3800 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:35.154783  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 179496us
W20260709 13:01:35.160349  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 197000us
I20260709 13:01:35.160961  1703 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.144711  4146 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.163214  1560 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:35.172776  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.173053  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:35.157925  4189 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.175415  1673 leader_election.cc:336] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.618s (SENT)
I20260709 13:01:35.172364  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.176780  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.178906  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:35.171049  4148 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:35.178830  1554 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
I20260709 13:01:35.169960  4175 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:35.190936  4105 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.191363  1629 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:35.200974  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:35.191428  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:35.189086  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58679us
I20260709 13:01:35.196009  1560 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:35.172663  4098 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:35.189536  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2158) took 1899 ms. Trace:
I20260709 13:01:35.197549  1700 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.195479  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 316) took 1527 ms. Trace:
I20260709 13:01:35.165428  4128 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:35.202913  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75681.4us
I20260709 13:01:35.196731  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.168045  3918 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:01:35.146134  4192 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:35.208321  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.211206  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.178737  4118 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.221273  1632 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:35.222218  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:35.204176  4098 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.223033  4098 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:35.223903  1673 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.009s (SENT) [suppressed 68 similar messages]
I20260709 13:01:35.200042  4189 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:35.224764  1673 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.227s (SENT)
I20260709 13:01:35.226127  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:35.226840  4189 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.202386  1635 rpcz_store.cc:276] 0709 13:01:33.145847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.146095 (+   248us) service_pool.cc:224] Handling call
0709 13:01:33.172324 (+ 26229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.533502 (+1361178us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b180005f740. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9dcbfb 00007f70ce9e764b
0709 13:01:34.533841 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.533883 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.534682 (+   799us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.650338 (+115656us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:34.659521 (+  9183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.671726 (+ 12205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.914121 (+242395us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:35.045477 (+131356us) raft_consensus.cc:1685] finished
0709 13:01:35.045491 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.045606 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":313276,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":1560,"spinlock_wait_cycles":109819520,"thread_start_us":421621,"threads_started":2,"wal-append.queue_time_us":138906}
I20260709 13:01:35.231937  4191 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:35.232479  4191 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.233171  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.233464  4191 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.171339  4153 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.234488  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.235064  4153 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.235215  1554 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:35.236380  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.236902  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:35.237377  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.203456  1557 rpcz_store.cc:276] 0709 13:01:33.666904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.677830 (+ 10926us) service_pool.cc:224] Handling call
0709 13:01:33.678180 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.678433 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:33.693132 (+ 14699us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.842171 (+1149039us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b1800127000. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:01:35.151154 (+308983us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.155922 (+  4768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.161041 (+  5119us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:35.176372 (+ 15331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.176481 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.194122 (+ 17641us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12250,"prepare.queue_time_us":14476,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":6956,"raft.queue_time_us":15070,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":10197,"spinlock_wait_cycles":7514880,"thread_start_us":204112,"threads_started":2,"wal-append.queue_time_us":1296358}
I20260709 13:01:35.218780  4023 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:35.211387  4148 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:35.250350  1555 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:35.254033  1703 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:35.260349  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.267448  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.269284  1635 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:01:35.262919  4029 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.870s	user 0.008s	sys 0.008s
I20260709 13:01:35.236680  4105 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.274156  4105 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.274046  4146 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.274791  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.277160  2292 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:35.277724  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.278990  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.280260  4201 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:35.280679  2289 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:35.280336  4148 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.281033  4201 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:35.281792  4201 consensus_queue.cc:1243] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:35.282286  4201 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:35.282649  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.282931  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.283304  2294 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.283620  2289 raft_consensus.cc:2376] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5. Re-sending same reply.
I20260709 13:01:35.284351  4024 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:35.284758  4024 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.200719  4119 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.285269  4009 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:35.287088  4152 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.290619  4152 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.292973  4152 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.285147  4024 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.293926  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.294773  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:35.295375  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.300551  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.302604  2292 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:35.305357  4189 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:35.307086  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:35.307135  3870 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:35.308037  4189 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:35.308473  4189 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.301463  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.308982  4189 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:35.309546  2291 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:35.310633  3937 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.311426  3937 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.312500  4192 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.314025  3937 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.314798  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.308352  3870 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:35.260792  4140 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.316660  3870 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.317533  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.318480  1628 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:35.223945  3924 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.252389  3800 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:35.320183  3800 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:35.320701  3800 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.322180  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.323617  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.324030  4152 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:35.324430  1627 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:35.325201  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 99134.2us
I20260709 13:01:35.326179  1558 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:35.329536  1554 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:01:35.329694  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 297) took 17548 ms (client timeout 15000 ms). Trace:
I20260709 13:01:35.330973  1630 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:35.331326  1627 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:35.332079  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.332326  4059 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:35.332728  1635 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.332872  4059 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:35.333841  4059 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.336231  1629 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:35.283265  1551 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:35.337378  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:35.239138  4118 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.341809  4118 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.346570  4006 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:35.347075  1706 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:35.347931  4029 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:01:35.347981  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46611.1us
I20260709 13:01:35.348496  4029 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.349395  4029 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.351006  4029 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.351564  1557 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:35.339501  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:35.349916  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 1994) took 2128 ms. Trace:
W20260709 13:01:35.352603  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45596.5us
I20260709 13:01:35.300968  1700 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:35.361907  1705 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:35.362946  1706 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:35.367851  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.355727  1552 rpcz_store.cc:276] 0709 13:01:33.221027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.268960 (+ 47933us) service_pool.cc:224] Handling call
0709 13:01:33.313398 (+ 44438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.881911 (+1568513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.349608 (+467697us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540036c4d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:35.349847 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3602404736}
I20260709 13:01:35.369768  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:35.375500  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67974.4us
I20260709 13:01:35.378445  1632 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:01:35.389037  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.347092  4006 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.394436  4199 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
W20260709 13:01:35.395455  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58174.1us
W20260709 13:01:35.396051  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:35.395502  3850 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:35.397130  3850 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:35.398212  3850 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.399930  3845 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.400511  3845 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.401718  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.403110  3845 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.392385  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.393371  1626 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:35.394064  4181 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.671s	user 0.000s	sys 0.003s
I20260709 13:01:35.394178  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:35.394737  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2047) took 2093 ms (client timeout 2239 ms). Trace:
W20260709 13:01:35.415607  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53058.1us
I20260709 13:01:35.282883  4201 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:01:35.416236  1635 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:35.390758  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:35.415756  1707 rpcz_store.cc:269] 0709 13:01:33.301123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.679708 (+378585us) service_pool.cc:224] Handling call
0709 13:01:35.394502 (+1714794us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:35.394652 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603801344}
W20260709 13:01:35.414386  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63446.6us
I20260709 13:01:35.407063  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:35.398238  4096 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.420024  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 334) took 2222 ms (client timeout 2769 ms). Trace:
I20260709 13:01:35.430078  1630 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
W20260709 13:01:35.424814  1702 rpcz_store.cc:269] 0709 13:01:33.197400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.737693 (+540293us) service_pool.cc:224] Handling call
0709 13:01:35.403914 (+1666221us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400134cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:35.419952 (+ 16038us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2988895488}
I20260709 13:01:35.430524  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.430639  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.432195  1601 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:35.434073  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.430624  1630 pending_rounds.cc:85] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 7
I20260709 13:01:35.419608  1701 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.439720  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.435961  1630 pending_rounds.cc:107] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:35.459373  1630 raft_consensus.cc:2889] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.460139  1703 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.463347  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:35.469908  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.475104  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:35.475965  1551 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:35.476240  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.477022  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:35.477687  1554 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.477744  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.478724  3924 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.480347  1558 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:35.480669  2288 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:35.481786  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.482611  2294 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:35.482785  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.483528  2296 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:35.484813  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.485335  1628 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:35.486162  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.486547  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.486816  2290 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:35.487146  1628 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.488483  1552 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:01:35.489079  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.489697  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.490298  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.491367  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:35.492137  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61965.8us
I20260709 13:01:35.493129  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.494518  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.494987  1628 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:35.495354  3937 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.496199  1628 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:35.440589  4096 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.521339  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:35.522912  4096 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.440526  2291 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:35.387537  4098 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.499285  1634 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:35.503368  1628 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:35.504699  4023 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:35.505988  1559 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:35.511845  4118 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:35.513643  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.417032  4149 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.514564  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:35.520068  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.523299  3767 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:35.479364  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.545012  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:35.545068  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81809.3us
W20260709 13:01:35.545490  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 113487us
I20260709 13:01:35.546181  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.546427  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:35.532725  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 125708us
I20260709 13:01:35.549803  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:35.552299  1703 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:35.556882  1557 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:35.558573  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.528002  4148 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.566130  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:35.567364  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47436.7us
I20260709 13:01:35.542476  4205 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:35.591799  4205 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.592329  4205 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.548040  4023 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:35.330076  1457 rpcz_store.cc:269] 0709 13:01:17.781050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.781215 (+   165us) service_pool.cc:224] Handling call
0709 13:01:35.329452 (+17548237us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:27.910503 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:28.006103 (+ 95600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:28.006141 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:01:28.006162 (+    21us) tablet.cc:662] Decoding operations
0709 13:01:28.007110 (+   948us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:28.007168 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:28.007205 (+    37us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:01:28.044439 (+ 37234us) tablet.cc:701] Row locks acquired
0709 13:01:28.044457 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:01:28.044625 (+   168us) write_op.cc:270] Start()
0709 13:01:28.044769 (+   144us) write_op.cc:276] Timestamp: P: 1783602088044570 usec, L: 0
0709 13:01:28.044793 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:01:28.045622 (+   829us) log.cc:844] Serialized 3244 byte log entry
0709 13:01:30.576187 (+2530565us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:32.271865 (+1695678us) op_driver.cc:464] REPLICATION: finished
0709 13:01:32.352906 (+ 81041us) write_op.cc:301] APPLY: starting
0709 13:01:32.352989 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:32.353200 (+   211us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:32.353228 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:32.354192 (+   964us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:32.354451 (+   259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:01:32.354499 (+    48us) write_op.cc:312] APPLY: finished
0709 13:01:32.354761 (+   262us) log.cc:844] Serialized 65 byte log entry
0709 13:01:32.354928 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:32.355049 (+   121us) write_op.cc:454] Released schema lock
0709 13:01:32.355098 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:32.355558 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:32.355586 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2364208,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":80805,"apply.run_cpu_time_us":2609,"apply.run_wall_time_us":2610,"num_ops":8,"prepare.queue_time_us":171145,"prepare.run_cpu_time_us":7797,"prepare.run_wall_time_us":170339,"raft.queue_time_us":789651,"raft.run_cpu_time_us":21857,"raft.run_wall_time_us":46487,"replication_time_us":4226865,"spinlock_wait_cycles":189594880,"thread_start_us":506445,"threads_started":4,"wal-append.queue_time_us":95087,"wal-append.run_cpu_time_us":102152,"wal-append.run_wall_time_us":5222249}]]}
I20260709 13:01:35.546483  3767 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.548849  4150 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.588456  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:35.606105  1558 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:35.561514  4086 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:35.609951  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:35.618739  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.570206  4153 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:35.619256  4196 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:35.620011  4140 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:35.609946  4213 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:01:35.621272  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75163.9us
I20260709 13:01:35.623905  4031 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:35.628144  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 80708.5us
I20260709 13:01:35.628284  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:35.628917  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:35.629225  4213 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:35.629652  4213 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.630282  3750 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:35.630734  3750 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.631085  4119 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:35.630620  4213 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.631803  4119 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.631980  4153 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.613027  4086 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.632915  4086 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.633176  4119 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:35.634133  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.624167  3687 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:35.620576  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:35.635982  4196 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:35.633227  4153 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.620045  3767 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.637634  3767 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.631315  3750 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.637223  4196 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.640291  4144 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.640543  4196 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.640748  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.640880  4144 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.619944  3527 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.643332  4144 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.643445  3527 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.628166  3928 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:35.644811  4117 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.590335  4192 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.652019  3928 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.646343  3527 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.646282  2262 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:35.652534  3928 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.623467  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.652501  4192 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.653072  4024 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.654222  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:35.648185  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.644037  4118 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:35.589098  3845 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.630661  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:35.656338  1552 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 4: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:35.660018  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127974us
I20260709 13:01:35.635916  1702 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:35.593580  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.615816  4175 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:35.682924  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.635917  4031 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:35.692463  2294 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:01:35.597160  3800 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.694697  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.695587  1634 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:35.695758  3800 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.699263  1701 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:35.678830  2290 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.678421  3845 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.703948  3687 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:35.708602  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.712146  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.713459  4098 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.620152  4150 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.717193  3800 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.685801  4140 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:01:35.720072  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:35.721138  4189 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.673681  4024 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.722541  4059 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.758985  4059 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.760943  3845 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.761724  4059 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.675895  4185 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:01:35.693269  4031 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.762283  4149 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.741835  3937 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:35.763101  4031 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.764535  4031 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.765125  2288 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:35.766161  4098 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:35.766562  4098 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:35.766924  4098 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.725445  3750 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.769327  3750 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:35.781011  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90369.8us
I20260709 13:01:35.726174  4023 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:35.732241  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60345.7us
I20260709 13:01:35.732903  4006 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.661882  4200 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:35.692885  2294 pending_rounds.cc:85] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 5
I20260709 13:01:35.804963  2294 pending_rounds.cc:107] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:01:35.805294  2294 raft_consensus.cc:2889] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.752597  1554 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:35.679486  4117 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:35.756636  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:35.758010  3800 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.802562  4189 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:35.808810  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:35.809813  1634 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:01:35.802736  1701 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:01:35.821321  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64757.3us
I20260709 13:01:35.825116  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:35.837136  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.839424  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:35.861050  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:35.867983  4127 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:35.868600  4127 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.869540  4127 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.888916  3527 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:35.889437  3527 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:35.890276  3527 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.897404  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.900614  2295 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:35.906946  4146 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:35.911608  4023 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:35.912820  4221 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:01:35.811828  2291 raft_consensus.cc:2376] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7. Re-sending same reply.
I20260709 13:01:35.916337  4006 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.919361  4192 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:35.844094  4096 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:35.925032  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:35.833175  4086 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.846617  4149 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.943274  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.931794  4117 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:35.849146  4189 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:01:35.928355  2294 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.956943  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:35.957018  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:35.959101  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 294790us
W20260709 13:01:35.959664  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 329046us
I20260709 13:01:35.965674  1633 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 2.
W20260709 13:01:35.967163  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 232807us
I20260709 13:01:35.968788  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:35.951578  4096 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.957710  4218 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:35.962257  4199 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:35.987298  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:35.968359  3527 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.977957  4096 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:35.997352  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.000375  2292 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.998440  4200 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:35.999289  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.003755  1553 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.959471  4149 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:35.970036  3845 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:01:36.003885  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.007947  1554 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 2.
I20260709 13:01:35.985356  4218 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:35.965433  4205 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:35.969534  4196 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:36.005105  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.045197  2295 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.045075  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:36.009716  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.009851  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:36.008782  3928 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.015203  3918 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:36.017117  4199 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.018577  1701 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:36.009858  3845 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:36.068440  4086 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.068569  3845 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:36.020117  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.069939  1629 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:36.069626  4199 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.020617  4200 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:36.070729  4200 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.021325  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:36.071602  1630 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.072036  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.072718  1557 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:36.019443  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.076032  1703 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.023157  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.032269  1633 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:36.047374  2292 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:36.049644  2297 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:35.991135  4150 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.066138  2294 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:36.046324  3918 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.067035  3944 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:01:36.103302  2294 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:36.088466  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43481.5us
W20260709 13:01:36.090319  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.973559  4117 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.092217  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.099172  1703 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:36.100394  1627 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:36.041729  4218 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.101450  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.103168  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.108385  1558 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:36.087821  4197 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.130646  2292 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:36.100967  3918 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:35.994345  4119 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:01:36.084937  1630 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:36.105450  2290 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:36.141764  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.102968  4031 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:36.149246  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 177780us
I20260709 13:01:36.157991  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.162803  1630 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:36.177163  1631 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:36.103533  4086 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:01:36.069594  3845 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.179004  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.184966  1557 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:36.143728  3800 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:36.181527  1524 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.052s (SENT)
I20260709 13:01:36.145437  3687 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.145838  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.176484  3928 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.189901  1552 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:36.190878  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.191275  1552 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:36.195336  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.195977  1560 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:36.178229  4031 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:36.199630  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.198212  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:36.160567  4197 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.204101  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:36.204589  3870 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.195603  3687 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.216431  4149 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.217770  4205 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.193648  4098 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.222855  4098 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.223449  3928 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:36.223927  3928 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.224318  3928 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.226871  3981 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.227319  4197 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.227676  3981 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:36.229702  4096 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.230867  3981 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:36.231937  3845 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.232777  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.233547  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:36.235277  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:36.240221  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60625.6us
I20260709 13:01:36.241604  4153 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:36.242023  4153 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:36.242080  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.243453  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.244246  1556 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:36.245247  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.245848  1557 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.250886  1635 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:36.254752  1553 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:36.259450  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55442.6us
I20260709 13:01:36.260313  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:36.260612  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.261243  1553 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:36.263644  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:36.265690  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.264277  1631 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6. Re-sending same reply.
I20260709 13:01:36.267316  1552 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.268354  1552 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.272074  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.272656  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.272795  1555 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:36.272986  4006 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.272827  4029 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.273754  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.274211  3767 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.274601  3767 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.274989  3767 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.277832  3750 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.278338  3750 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.280637  4117 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.280745  3750 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.244305  1551 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:36.282423  1551 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:36.282852  1631 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:01:36.283236  1631 pending_rounds.cc:85] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 3
I20260709 13:01:36.283527  1631 pending_rounds.cc:107] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:36.283831  1631 raft_consensus.cc:2889] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.283282  1551 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:36.284511  1551 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.286015  1630 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:36.287626  1630 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.289812  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.290469  1631 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:36.291376  4098 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.292783  3870 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:36.295295  4096 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.295460  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.296000  4096 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.296288  3870 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:36.296909  3870 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:36.297770  3870 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.301745  4213 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:36.302816  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:36.304015  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.282015  4031 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.309568  1560 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:36.307185  2293 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:36.242939  4153 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.340992  2295 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:36.344601  1552 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:36.351267  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.345546  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.354133  1555 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:01:36.354650  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.356986  1553 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.360373  1560 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:36.306043  4197 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:36.296514  4096 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:36.364230  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61531.6us
I20260709 13:01:36.369733  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.370438  4146 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.370905  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.371435  1552 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.368765  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.340382  4098 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:36.380384  1551 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:01:36.380775  1551 pending_rounds.cc:85] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 3
I20260709 13:01:36.380998  1551 pending_rounds.cc:107] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:01:36.381217  1551 raft_consensus.cc:2889] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.382329  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.366902  4197 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.271682  3928 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.400230  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.391386  1557 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:36.424269  1551 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:36.405732  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.406876  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.405658  3918 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.410383  4218 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.413874  1553 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 3.
I20260709 13:01:36.430043  1552 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:36.359663  4024 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:01:36.362731  4023 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:36.413039  4153 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:36.443648  1629 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 3: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:36.413794  4098 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:36.418991  4197 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:36.455117  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.488164  4059 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.491732  4059 raft_consensus.cc:2764] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:36.492285  4218 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.492715  3928 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:36.496542  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89777.4us
I20260709 13:01:36.497416  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.501137  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.501771  1703 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:36.502638  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.503876  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.504300  1705 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:01:36.505779  1703 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:01:36.505818  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.507364  1706 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:36.507570  1700 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.503923  1707 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:36.506619  1708 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:36.513227  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.514339  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.514683  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.514961  2295 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:36.515726  1700 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:36.516780  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.517762  1701 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:36.519351  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.520121  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:36.521718  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.522082  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.522871  1704 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:36.523368  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47210 (request call id 23) took 1080 ms. Trace:
I20260709 13:01:36.523705  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.523728  1702 rpcz_store.cc:276] 0709 13:01:35.442696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.521121 (+ 78425us) service_pool.cc:224] Handling call
0709 13:01:36.523188 (+1002067us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400165b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9de853 00007f70d39ac60a 00007f70ccf327d4 00007f70ccf32762 00007f70ccf32691 00007f70ccf3260c 00007f70ccf318c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:36.523275 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261648128}
I20260709 13:01:36.524909  4200 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.525688  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.526001  4200 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.526278  1706 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.528378  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.529446  4137 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:36.530018  4137 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.530997  4137 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.532640  4137 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.533386  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.534106  1705 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:36.534050  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.535732  1703 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:36.536787  4205 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:36.537087  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.537933  1598 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:36.538210  4205 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:36.538698  4205 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.538998  4192 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:36.539942  4005 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:36.540017  1708 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:36.539527  4192 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.541353  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.541513  1704 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:36.541930  1707 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.541651  4192 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.542734  3845 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.543231  1702 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:36.558638  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.495195  4175 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.560221  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:36.564131  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:36.567461  1702 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:36.563477  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.572806  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.573467  1700 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.573947  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.572961  1629 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
W20260709 13:01:36.561858  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64523.5us
W20260709 13:01:36.574209  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 207611us
I20260709 13:01:36.574191  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.576608  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.578837  3981 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:36.581382  4006 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:36.588217  4006 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.588635  4006 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.587837  1635 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:36.539523  4205 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:36.592346  4242 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:01:36.588219  3981 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:36.561372  4086 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.594904  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:36.543282  3845 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.607558  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.577908  3928 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:36.617486  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.632357  1700 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:36.633466  1702 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.634083  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.635743  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:36.637007  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.637759  2289 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.634186  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:36.602185  1707 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:36.640097  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.643709  2293 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:36.645280  1706 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:36.576857  4218 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:01:36.603070  4150 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.608680  4119 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.578037  3850 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:01:36.616129  4140 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:36.632675  3918 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.640775  3924 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:36.633158  3928 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.577410  4175 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.641072  4144 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.652166  3750 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:36.671942  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.685827  2291 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:36.665944  4250 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 13:01:36.688452  3981 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.689396  3845 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:36.690021  3845 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:36.690387  3845 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:36.666527  4119 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:36.692230  4244 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:01:36.693428  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 117343us
I20260709 13:01:36.693526  4023 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.694192  4119 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:36.694691  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.666193  4150 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.694954  4023 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:36.614490  4086 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:36.696525  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106039us
I20260709 13:01:36.696684  4255 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:36.697227  4150 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:36.697379  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 121822us
I20260709 13:01:36.697641  4086 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.697912  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.698350  4059 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:36.698833  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:36.699453  4175 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:36.700055  4247 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:36.699980  4144 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:36.667042  4243 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:01:36.700964  4247 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.702086  4248 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:36.702423  4144 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:36.703434  1458 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 4, now has a pending config: VOTER 444b494bb53842b59408a7230884398a (127.0.107.193), VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195), VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:36.672791  3928 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.705906  1458 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.671718  4149 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:36.707847  3687 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.707950  4140 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:36.697227  4255 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:36.709987  3767 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.710310  4255 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.712488  3767 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:36.651691  4096 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:36.660815  4029 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.699904  4059 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:36.712910  1706 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:36.698870  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.709373  4140 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:36.714646  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106901us
I20260709 13:01:36.716701  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.716912  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:36.718189  1559 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:36.670696  3924 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.729115  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.723351  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.713362  4059 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.730278  1702 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:36.741506  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.726810  2289 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:36.731935  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.742295  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.736209  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.743149  1558 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.721841  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.718017  1458 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.744076  1557 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 3: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:36.616516  4098 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.747262  1553 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:36.729225  3750 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:36.748735  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:36.749075  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:36.749118  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:36.746243  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:36.764221  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.758572  4144 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.744904  1458 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 6, now has a pending config: VOTER 444b494bb53842b59408a7230884398a (127.0.107.193), VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194), VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195), VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:36.713323  4029 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.737360  4153 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:36.843379  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 73635.2us
W20260709 13:01:36.847445  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98400.5us
I20260709 13:01:36.848419  3527 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.849128  4253 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:36.767225  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.850569  4023 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:36.851444  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84300.9us
I20260709 13:01:36.852348  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.852895  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:36.843858  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 94877.8us
I20260709 13:01:36.853633  4252 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:01:36.854470  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:36.855805  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.857306  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.858496  1633 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:36.858688  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.859624  2289 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:36.859958  4256 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:01:36.691279  3845 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:36.742334  3870 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.860631  4117 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:36.848344  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.868194  1706 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:36.844075  1559 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:36.872592  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.873327  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.745975  4059 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:36.878109  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.889505  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:36.891722  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:36.893728  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.894587  1700 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
W20260709 13:01:36.895503  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 264736us
I20260709 13:01:36.774240  3767 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:36.875844  1458 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:36.861756  4119 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:36.862318  4153 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.749181  4137 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.136s
I20260709 13:01:36.765775  4254 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.911609  2289 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:01:36.913426  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 65180.5us
I20260709 13:01:36.989460  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:36.870571  4117 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.869937  4150 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:36.892838  4213 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
W20260709 13:01:36.917924  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 204536us
W20260709 13:01:37.006145  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.918591  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:36.919824  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.772270  4208 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 13:01:36.867007  3937 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:01:36.888124  4146 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.870755  3687 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:01:37.038079  1458 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:37.039669  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161619us
I20260709 13:01:36.867208  4248 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:01:36.902947  4252 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:36.909785  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:36.871038  4144 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:37.064894  4257 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:01:37.072178  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:36.923249  4153 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.917889  3767 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:37.005044  4247 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:37.005386  4117 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:36.897516  4096 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.160s
I20260709 13:01:37.079885  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:37.128607  2292 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:37.041880  4023 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:37.066604  3845 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:37.056079  4265 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:37.084024  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:37.056728  4024 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:37.133651  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:37.130092  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 266620us
I20260709 13:01:37.055814  4150 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:37.061173  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:37.062391  4149 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.068346  3918 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.092260  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:37.150470  1702 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:37.101533  2294 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:37.054147  4189 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:37.076889  3850 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:37.041644  4119 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.110736  1559 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:37.110064  4230 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:01:37.109642  1553 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:37.110448  4267 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:37.220006  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:37.224699  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:37.226328  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.227068  1553 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:37.222112  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:37.230139  1626 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:37.233414  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:37.113094  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 223675us
I20260709 13:01:37.116937  1458 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:37.056550  3924 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:36.862349  4218 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:37.089811  3928 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.243038  1458 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 5, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.257455  1458 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:37.112740  4252 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:37.127808  4031 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.183599  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:37.187371  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.262236  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.212733  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.078807  3750 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.263448  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.263857  1555 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:37.235782  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:37.263435  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 155581us
I20260709 13:01:37.136854  4248 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
W20260709 13:01:37.212951  4257 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:01:37.254729  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:37.213405  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.158439  4005 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.265471  1634 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:37.216877  4189 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:37.239689  4260 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:01:37.270143  1557 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:37.237254  4029 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:37.298919  1634 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:37.159787  4149 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:37.272389  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 88855.8us
I20260709 13:01:37.281985  1458 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 6, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:37.290007  1626 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:37.290689  1555 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:37.218878  3850 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:37.290262  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 413643us
I20260709 13:01:37.179601  3918 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:37.282917  3981 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.008s
I20260709 13:01:37.270007  4199 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:37.273602  4254 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:37.306777  1551 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:37.307367  1524 leader_election.cc:400] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:37.348789  1524 leader_election.cc:403] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:01:37.349264  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43667.3us
I20260709 13:01:37.290198  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:37.311094  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 463319us
W20260709 13:01:37.314426  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.303524  1458 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:37.354766  4140 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:37.355803  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65723.1us
I20260709 13:01:37.354924  4277 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.268934  4031 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.298915  4253 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.360289  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:37.360590  1634 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:01:37.358145  1558 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.001s
I20260709 13:01:37.360941  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:37.304379  4213 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.347702  3981 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.008s	sys 0.009s
I20260709 13:01:37.366962  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:37.294855  4196 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:01:37.268620  3767 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.347847  4029 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:37.384230  4260 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:01:37.368500  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:37.412284  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43879.5us
I20260709 13:01:37.365751  1558 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:37.376371  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:37.137233  4265 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.394115  1458 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:37.298707  4005 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:37.365407  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 103120us
I20260709 13:01:37.383401  3944 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:01:37.357725  4247 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:37.415136  1552 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [6.7-9.8]
I20260709 13:01:37.386098  4254 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:37.367937  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:37.421918  1458 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 4, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 4 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.284032  4150 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:37.432389  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75701.7us
I20260709 13:01:37.400682  4279 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.442786  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:37.363446  4140 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:37.443943  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83022.1us
I20260709 13:01:37.445544  4140 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:37.445480  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:37.388348  4127 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:01:37.383915  4256 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:01:37.432698  4175 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:37.408669  4029 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.388454  4213 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.363088  4199 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.439213  4275 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:37.370193  4277 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.481426  1458 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.590632  1630 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:37.436686  4144 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:37.565215  3527 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:37.432883  4267 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:37.446834  4140 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
W20260709 13:01:37.466907  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99011.7us
I20260709 13:01:37.650972  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:37.442163  4279 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:37.371944  4252 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.451013  4137 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:37.633517  4208 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:37.593456  4029 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:37.639132  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45546.2us
I20260709 13:01:37.640333  1634 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:37.565533  4175 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.599040  4262 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.394s	user 0.001s	sys 0.000s
I20260709 13:01:37.384591  4253 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.583998  4254 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:37.647526  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 223000us
I20260709 13:01:37.667867  1458 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:01:37.646762  4259 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
W20260709 13:01:37.711835  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 270375us
W20260709 13:01:37.713735  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62834.4us
I20260709 13:01:37.641467  3527 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:37.442932  4189 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.728142  1458 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.735504  1458 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.670661  4175 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:37.664623  4279 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.755122  1635 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:37.711822  3928 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:37.713725  4267 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:37.651676  4248 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:01:37.716768  4096 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:37.709718  4254 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.727674  4196 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:37.712752  4119 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:37.769562  1458 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 5 to 8, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 8 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.775357  1458 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 6, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 6 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.751349  4276 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.389s
I20260709 13:01:37.724334  4278 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.368s
I20260709 13:01:37.778702  4023 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:37.758734  4286 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.366s
I20260709 13:01:37.780469  1554 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:37.781981  4274 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
I20260709 13:01:37.782409  4299 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:37.782936  4280 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:37.785533  4273 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.547s	user 0.003s	sys 0.000s
W20260709 13:01:37.787349  4269 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.557s	user 0.001s	sys 0.001s
W20260709 13:01:37.788527  4291 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:01:37.788383  4267 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.773178  4254 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:37.790014  4282 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:37.696657  4137 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:37.757089  4127 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:37.772426  4006 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.183s	user 0.011s	sys 0.002s
I20260709 13:01:37.787540  4005 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:37.791831  4283 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:01:37.671743  4208 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:37.816948  4117 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.000s
I20260709 13:01:37.817546  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.818882  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.768760  4149 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:37.832038  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 196103us
I20260709 13:01:37.809475  4119 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.928789  4119 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.930331  4293 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:01:37.844815  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:37.812568  3944 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:37.932576  3944 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.934454  3944 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:37.845435  4265 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:37.845711  2294 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:37.848567  4197 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.398s	user 0.010s	sys 0.002s
I20260709 13:01:37.849013  4300 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:01:37.851137  4059 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.947s	user 0.009s	sys 0.000s
I20260709 13:01:37.772670  4196 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.861255  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 415779us
I20260709 13:01:37.774035  4031 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:37.648254  4275 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:37.861848  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.005s
I20260709 13:01:37.926272  1555 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:01:37.937847  1559 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.721s	user 0.000s	sys 0.009s
W20260709 13:01:37.950764  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.008s	sys 0.005s
W20260709 13:01:37.951093  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 593151us
I20260709 13:01:37.947301  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:37.811515  3527 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.954159  1559 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:37.944352  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 229839us
I20260709 13:01:37.949968  4146 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:37.947124  4213 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:37.923893  4006 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.956043  4005 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:37.956190  4146 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:37.965633  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.935678  4197 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.968689  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 556) took 1365 ms. Trace:
I20260709 13:01:37.968480  1458 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 4, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 4 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.979210  4006 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:37.979959  1458 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 2, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 2 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.981352  4146 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.973029  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:37.967243  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:37.974370  1633 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:37.952052  4196 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:37.977062  2288 rpcz_store.cc:276] 0709 13:01:36.603680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.633903 (+ 30223us) service_pool.cc:224] Handling call
0709 13:01:36.689611 (+ 55708us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:36.689863 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.775518 (+ 85655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.845777 (+ 70259us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.851469 (+  5692us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.152255 (+300786us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:37.152353 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.208586 (+ 56233us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:37.208672 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.968484 (+759812us) raft_consensus.cc:1685] finished
0709 13:01:37.968506 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.968630 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1831,"raft.queue_time_us":25926,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":15413,"spinlock_wait_cycles":73577600,"thread_start_us":300311,"threads_started":1,"wal-append.queue_time_us":448485}
I20260709 13:01:37.931684  4119 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:37.989636  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:37.995018  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:37.937502  4117 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.992364  4096 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:37.993355  3870 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.120s	user 0.007s	sys 0.003s
I20260709 13:01:37.926254  4127 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:37.995973  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:37.996863  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:37.995572  1555 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:38.005267  1632 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:37.951426  4059 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:38.014429  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 297020us
I20260709 13:01:37.990760  4005 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:38.020951  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:38.021811  2297 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:38.023377  2288 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:38.000650  4096 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.018905  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:37.977063  4189 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:01:38.018602  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:38.037103  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 327372us
I20260709 13:01:38.039347  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:38.039568  4189 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:37.999311  3850 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.041765  4307 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:38.042202  4307 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.042537  4307 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:38.042862  3850 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:38.043393  3850 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.044621  1553 raft_consensus.cc:2368] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:38.011693  3870 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:38.015555  3928 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:38.066403  2297 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:38.068743  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:37.994077  4297 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:38.044788  4096 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:38.056222  3850 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:38.072973  1556 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:38.080770  3928 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:38.089179  3928 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:38.089321  4117 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.090045  4271 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.090163  3928 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.090615  4271 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.092334  4314 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.092995  4271 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.092882  4314 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:38.093159  4199 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:38.095144  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99283.6us
I20260709 13:01:38.095957  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:38.070082  4213 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.017392  4006 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:38.037192  4119 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.057552  4286 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:38.081314  4306 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:38.104494  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:38.080312  4277 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:38.097765  4314 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:38.104493  3845 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.971s	user 0.010s	sys 0.001s
I20260709 13:01:38.090642  4127 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.119601  4308 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:01:38.121563  1707 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:38.116629  4005 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.109011  4137 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:38.131698  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 134282us
I20260709 13:01:38.118096  4277 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:38.119267  4059 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.106249  4213 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.109869  4119 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.117010  4189 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:38.142949  4323 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 13:01:38.145998  4218 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.000s
I20260709 13:01:38.125296  3845 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:01:38.153929  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:38.154771  2292 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:38.111599  4286 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.169293  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:38.107730  4306 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.172511  4298 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:38.151489  4277 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.140300  4275 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:38.189530  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.148972  4059 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:01:38.171443  4316 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:38.231683  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 127259us
I20260709 13:01:38.232846  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:38.234463  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:38.224939  4323 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.235927  4298 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:38.252899  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:38.256404  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:38.266121  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:38.267375  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:38.270388  3944 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:38.270987  4031 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:38.228794  4196 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:38.256187  4304 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:01:38.268210  4315 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:01:38.268312  4311 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:01:38.298650  2292 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:38.202620  4277 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.281690  4031 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:38.287009  4306 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:38.257030  4213 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.279289  4117 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:38.244140  4323 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:38.321170  4247 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.004s	sys 0.001s
I20260709 13:01:38.313179  4319 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:38.318411  4117 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:38.329007  4247 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.004s	sys 0.001s
W20260709 13:01:38.332041  4153 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.192s	user 0.011s	sys 0.001s
I20260709 13:01:38.238670  4189 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.347848  4277 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.342137  4319 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:38.344571  4117 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:38.344871  4119 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.389037  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2029) took 1215 ms. Trace:
I20260709 13:01:38.401269  1553 rpcz_store.cc:276] 0709 13:01:37.173418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.233100 (+ 59682us) service_pool.cc:224] Handling call
0709 13:01:38.388838 (+1155738us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54002ba7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:38.388973 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863446912}
I20260709 13:01:38.437806  4208 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:38.453507  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 209958us
I20260709 13:01:38.359633  4153 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.389348  4319 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:38.454619  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:38.453121  4321 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 13:01:38.388557  4096 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.463022  1553 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:38.436444  4119 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.536164  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.043s	sys 0.071s
I20260709 13:01:38.471910  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:38.552659  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.045s	sys 0.074s
I20260709 13:01:38.552985  1553 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:01:38.513553  4175 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.752s	user 0.004s	sys 0.007s
I20260709 13:01:38.454480  4286 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.566351  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.571550  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:38.576342  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.577334  1557 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:38.588433  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.589116  1553 raft_consensus.cc:2376] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7. Re-sending same reply.
I20260709 13:01:38.590057  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.584997  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:38.597654  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.598435  1704 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:38.536473  4196 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.601712  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.548468  4253 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:38.513281  4031 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:38.604588  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.609565  1702 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:01:38.595592  4254 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 13:01:38.618965  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.619791  1704 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:38.508167  4119 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.600184  3924 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:38.622761  1704 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:38.623929  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.469163  4307 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.607374  4218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44858 (request call id 79) took 18581 ms. Trace:
I20260709 13:01:38.611312  4323 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.574096  4175 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.623710  3924 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:38.643227  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.644577  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.645321  1557 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:38.614015  4031 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.630458  4196 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.650250  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.633612  4096 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:38.622686  4254 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.755122  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.698472  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.757701  1557 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:38.655362  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:38.675635  4310 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:38.767920  4330 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:01:38.646709  3924 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:38.698513  4255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.570s	user 0.018s	sys 0.013s
I20260709 13:01:38.666577  4153 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:38.800983  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.787482  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.772497  4005 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:38.787721  4255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.665s	user 0.022s	sys 0.015s
I20260709 13:01:38.814173  1557 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:01:38.811106  4337 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:01:38.813372  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.822265  1558 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:38.666677  4196 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.800730  4323 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:38.816901  4340 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:38.768028  4310 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.814079  1709 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:38.820848  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.834093  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.836630  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:38.837455  1557 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:01:38.824899  4299 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.000s
I20260709 13:01:38.796118  4117 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:38.901911  1557 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
W20260709 13:01:38.962997  1524 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.846s (SENT)
I20260709 13:01:38.964046  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:38.964668  1557 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:38.965945  1709 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:38.968134  1708 raft_consensus.cc:2410] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:01:38.962913  4343 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:01:38.893482  4345 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.125s	user 0.000s	sys 0.002s
I20260709 13:01:38.982512  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:38.963718  3850 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.002s
I20260709 13:01:38.890797  4175 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:38.821389  4005 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:38.994627  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:38.995055  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:38.967808  4299 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:38.979676  4288 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.998771  1708 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:38.999132  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:38.838851  4323 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.001521  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:39.005807  1558 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:39.001551  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:39.007367  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.004082  1557 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:39.009614  1709 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:39.000875  4339 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.286s	user 0.001s	sys 0.001s
W20260709 13:01:39.014928  4341 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:01:39.015035  4336 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:01:38.997325  4310 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.017720  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:39.000471  4192 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:39.018951  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:39.006664  4275 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:39.024557  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:38.988988  3850 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.017194  1701 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:38.630815  4218 rpcz_store.cc:276] 0709 13:01:20.004603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.004726 (+   123us) service_pool.cc:224] Handling call
0709 13:01:20.070413 (+ 65687us) log.cc:844] Serialized 501 byte log entry
0709 13:01:22.124549 (+2054136us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b18003d41a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:38.568739 (+16444190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111270,"raft.queue_time_us":113632,"raft.run_cpu_time_us":2250,"raft.run_wall_time_us":351159,"spinlock_wait_cycles":19892096,"thread_start_us":258593,"threads_started":1,"wal-append.queue_time_us":422091,"wal-append.run_cpu_time_us":2108,"wal-append.run_wall_time_us":1002107}
I20260709 13:01:39.008636  4119 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.031013  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:39.033349  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.021205  4308 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:39.016147  4307 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.007033  4205 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:01:39.008293  4288 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.040144  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:39.041213  1704 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:39.025583  4192 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.029031  4275 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.043931  4117 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.064878  4086 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.044834  4281 raft_consensus.cc:993] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:01:39.039199  4308 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:39.039741  4098 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:39.060725  4275 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:39.054090  4255 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.042899  4175 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:39.059978  4149 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.060356  4192 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:39.078688  4307 raft_consensus.cc:1081] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:01:39.078423  4308 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:39.065726  4086 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:39.099413  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97935.9us
I20260709 13:01:39.066036  4331 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:39.084177  4196 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.087553  4254 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.113616  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:39.088788  3928 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:39.123852  4347 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.175s	user 0.000s	sys 0.002s
I20260709 13:01:39.093616  4255 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.095525  4175 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.099146  4149 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.087090  4117 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:01:39.123525  1555 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:39.216897  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 160076us
I20260709 13:01:39.220844  4310 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:01:39.227435  4086 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.229336  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 327) took 2111 ms. Trace:
I20260709 13:01:39.229700  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2018) took 1974 ms. Trace:
I20260709 13:01:39.229851  1556 rpcz_store.cc:276] 0709 13:01:37.109350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.109486 (+   136us) service_pool.cc:224] Handling call
0709 13:01:37.112138 (+  2652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.087043 (+974905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.220419 (+1133376us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b5400309dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.220610 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208629120}
I20260709 13:01:39.230121  1630 rpcz_store.cc:276] 0709 13:01:37.255653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.626918 (+371265us) service_pool.cc:224] Handling call
0709 13:01:37.651455 (+ 24537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.936611 (+285156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.936641 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.936685 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:37.936723 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.229464 (+1292741us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54001c34d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.229509 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.229642 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655326080}
I20260709 13:01:39.231510  4255 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.236469  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 561) took 2359 ms. Trace:
I20260709 13:01:39.229552  4175 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.236893  2293 rpcz_store.cc:276] 0709 13:01:36.877329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.889458 (+ 12129us) service_pool.cc:224] Handling call
0709 13:01:36.914426 (+ 24968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.261541 (+347115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.261571 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.261613 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:37.261661 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.236197 (+1974536us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.236258 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.236387 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797795712}
I20260709 13:01:39.221570  3928 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.240463  3928 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.216550  4119 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.241475  4119 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:39.244328  4319 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.000s
I20260709 13:01:39.248541  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:39.248210  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1361) took 1063 ms. Trace:
I20260709 13:01:39.250221  4175 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:39.250097  1555 rpcz_store.cc:276] 0709 13:01:38.184523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.119025 (+934502us) service_pool.cc:224] Handling call
0709 13:01:39.119500 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.247933 (+128433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.248141 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:39.250787  4175 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:39.251189  4175 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:39.220001  4279 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.454s	user 0.008s	sys 0.013s
I20260709 13:01:39.259858  4279 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.261320  1700 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:39.217278  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1307) took 2524 ms (client timeout 1550 ms). Trace:
W20260709 13:01:39.261817  1707 rpcz_store.cc:269] 0709 13:01:36.692930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.716452 (+ 23522us) service_pool.cc:224] Handling call
0709 13:01:39.148419 (+2431967us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540010e0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:39.217204 (+ 68785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3229177600}
I20260709 13:01:39.244798  4319 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.264560  4319 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:39.264648  4149 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.266799  4279 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:39.269745  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 560) took 2392 ms. Trace:
I20260709 13:01:39.274686  2291 rpcz_store.cc:276] 0709 13:01:36.877035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.886963 (+  9928us) service_pool.cc:224] Handling call
0709 13:01:37.090658 (+203695us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:37.092941 (+  2283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.130784 (+ 37843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.137381 (+  6597us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.270321 (+132940us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.873593 (+603272us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:37.881874 (+  8281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.922003 (+ 40129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.251421 (+1329418us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b54005e0ea0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95aaaa 00007f70ce959dd9 00007f70ce964930 00007f70ce9648d8 00007f70ce964858 00007f70ce964813 00007f70ce963a9c 00007f70d484f626 00007f70d4849dd8 00007f70ce9a07cb
0709 13:01:39.253590 (+  2169us) raft_consensus.cc:1685] finished
0709 13:01:39.253631 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.269659 (+ 16028us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4059,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":6477,"spinlock_wait_cycles":40474624,"thread_start_us":495978,"threads_started":1,"wal-append.queue_time_us":573471}
I20260709 13:01:39.278085  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2012) took 2117 ms. Trace:
I20260709 13:01:39.281994  4146 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:39.285754  3918 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.278584  1631 rpcz_store.cc:276] 0709 13:01:37.160807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.160924 (+   117us) service_pool.cc:224] Handling call
0709 13:01:37.213552 (+ 52628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.633088 (+419536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.633117 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.633662 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.731248 (+ 97586us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:37.731299 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.277658 (+1546359us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400297cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.277703 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.277763 (+    60us) raft_consensus.cc:1685] finished
0709 13:01:39.277779 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.278019 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":200214,"raft.queue_time_us":198210,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":21515,"spinlock_wait_cycles":539063040,"thread_start_us":97140,"threads_started":1,"wal-append.queue_time_us":196690}
W20260709 13:01:39.289613  4342 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.327s	user 0.005s	sys 0.005s
I20260709 13:01:39.292119  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:39.293079  3924 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.293948  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:39.294796  4208 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:39.295344  4208 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:39.295898  4208 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.297463  3918 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:39.302863  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.304629  4303 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.305172  4303 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.307312  4303 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.295090  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:39.216606  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 95519.8us
W20260709 13:01:39.313271  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63501.4us
I20260709 13:01:39.321296  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:39.327044  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:39.219259  4218 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:39.372196  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:39.330650  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 375) took 2121 ms. Trace:
I20260709 13:01:39.326936  4263 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:01:39.431828  1632 rpcz_store.cc:276] 0709 13:01:37.209484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.210378 (+   894us) service_pool.cc:224] Handling call
0709 13:01:37.234815 (+ 24437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.006689 (+771874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.330424 (+1323735us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54001af9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.330588 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771072768}
I20260709 13:01:39.327320  4005 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.375805  3850 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:39.376163  4205 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:39.328779  1701 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:39.409190  4140 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.312655  4119 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.380249  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:39.379280  1558 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:39.329668  2291 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:39.334357  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:39.327486  3918 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:39.382093  4029 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:39.412894  4275 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.379379  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 618) took 1371 ms. Trace:
I20260709 13:01:39.219323  4196 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:39.295192  3924 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.248210  4117 consensus_queue.cc:1243] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:39.333587  2295 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:39.426011  2297 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:39.346431  4323 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:39.432505  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60397us
I20260709 13:01:39.441913  2288 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:39.450529  2292 rpcz_store.cc:276] 0709 13:01:38.008260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.008860 (+   600us) service_pool.cc:224] Handling call
0709 13:01:39.379220 (+1370360us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400720954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:39.379323 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267782400}
I20260709 13:01:39.435128  4352 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:39.407646  4218 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:39.465143  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:39.449195  3918 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.467700  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.442011  4288 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.466516  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:39.470257  1628 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:39.469331  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.469960  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.473261  1629 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:39.452862  4196 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.477432  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.474238  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.474928  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.479107  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.480226  1628 raft_consensus.cc:2410] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:39.478818  1630 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:39.487434  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.488451  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:39.489738  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.490482  1707 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:39.491500  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.464200  4119 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:39.501852  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.504593  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:39.505182  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:39.506273  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.500083  4288 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.507118  1703 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:39.512172  4288 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.470593  3918 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:39.467712  4140 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.515098  4288 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.517217  4140 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:39.521842  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2015) took 2298 ms. Trace:
I20260709 13:01:39.522647  1626 rpcz_store.cc:276] 0709 13:01:37.223782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.303477 (+ 79695us) service_pool.cc:224] Handling call
0709 13:01:37.356597 (+ 53120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.363617 (+  7020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.363648 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.376238 (+ 12590us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.234394 (+1858156us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b18000af5a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:39.449761 (+215367us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:39.455149 (+  5388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.455266 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.521420 (+ 66154us) raft_consensus.cc:1685] finished
0709 13:01:39.521449 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.521755 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13314,"raft.queue_time_us":36349,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":15766,"spinlock_wait_cycles":4460928,"thread_start_us":215105,"threads_started":1,"wal-append.queue_time_us":2095219}
I20260709 13:01:39.524480  4308 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.525471  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.526304  1626 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:39.528118  1635 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:39.528319  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.508008  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.529309  1626 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 3.
I20260709 13:01:39.529634  1700 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:39.533542  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:39.541417  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:39.483209  4196 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:39.527583  4308 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:39.509732  4275 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.475389  3924 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:01:39.614787  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:39.595084  4356 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:01:39.477499  4192 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.538056  4254 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:39.628022  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.528713  4352 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:01:39.646735  2295 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:39.613806  4358 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.539229  4119 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.652406  1632 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:39.635345  4275 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.640851  4254 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.664063  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 328) took 2554 ms. Trace:
I20260709 13:01:39.664588  1552 rpcz_store.cc:276] 0709 13:01:37.137432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.137595 (+   163us) service_pool.cc:224] Handling call
0709 13:01:37.414812 (+277217us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:37.433846 (+ 19034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.663796 (+2229950us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002607d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:39.663985 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526999936}
I20260709 13:01:39.651376  4293 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.636227  4208 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.677673  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 330) took 2511 ms. Trace:
I20260709 13:01:39.678223  1554 rpcz_store.cc:276] 0709 13:01:37.166687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.166805 (+   118us) service_pool.cc:224] Handling call
0709 13:01:37.214670 (+ 47865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.867331 (+652661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.677329 (+1809998us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54002b98d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.677588 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299490176}
I20260709 13:01:39.680469  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1349) took 1513 ms. Trace:
I20260709 13:01:39.680948  1557 rpcz_store.cc:276] 0709 13:01:38.166862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.034110 (+867248us) service_pool.cc:224] Handling call
0709 13:01:39.034565 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.037500 (+  2935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.037532 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.038612 (+  1080us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.330884 (+292272us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:39.330924 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.330996 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.680175 (+349179us) raft_consensus.cc:1685] finished
0709 13:01:39.680203 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.680397 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530,"raft.queue_time_us":698,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3047,"thread_start_us":267774,"threads_started":1,"wal-append.queue_time_us":399705}
I20260709 13:01:39.598953  4218 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.663827  4254 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:39.690024  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 130303us
I20260709 13:01:39.676273  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:39.674263  4293 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.698663  1628 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.700140  1626 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:39.674223  3687 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:39.674578  4352 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:39.660545  3928 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:39.665540  4275 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.659224  4358 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.700573  4218 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:39.702644  3687 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:39.700997  4293 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.723075  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2008) took 2567 ms. Trace:
I20260709 13:01:39.711566  4358 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.723699  1633 rpcz_store.cc:276] 0709 13:01:37.155456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.156923 (+  1467us) service_pool.cc:224] Handling call
0709 13:01:37.179912 (+ 22989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.984887 (+804975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.722772 (+1737885us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54000dded4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:39.722997 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826288512}
I20260709 13:01:39.727540  2288 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:39.728017  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 186534us
I20260709 13:01:39.729676  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:39.736133  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:39.738749  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.736004  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:39.739961  1633 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:39.740674  4268 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 2.471s	user 0.008s	sys 0.028s
W20260709 13:01:39.809092  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67448.5us
I20260709 13:01:39.742060  4372 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:39.735432  4146 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:01:39.735025  3918 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.812633  1708 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:39.813211  1703 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:39.804786  4144 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.807624  4192 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.828527  1705 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:39.812736  3687 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.839144  1708 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:39.827976  4366 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:01:39.811496  4364 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:01:39.829600  1631 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:39.829254  3918 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:39.827752  4175 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.002s
I20260709 13:01:39.841439  1633 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:39.708259  4352 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.849418  4175 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:39.841773  4146 consensus_queue.cc:1243] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:39.853999  4352 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.855479  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:39.868341  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:39.869078  1633 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:39.869199  1557 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:01:39.843508  4372 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:39.855916  4208 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.840474  4144 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:39.963045  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:39.992129  1705 raft_consensus.cc:606] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:39.992516  1705 raft_consensus.cc:614] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:39.990648  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:39.995345  1703 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:01:39.990303  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.137039  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:39.995689  1703 pending_rounds.cc:85] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 3
I20260709 13:01:40.138062  1703 pending_rounds.cc:107] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:40.138587  1703 raft_consensus.cc:2889] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.035370  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.140796  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.128891  4254 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.141474  1555 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.144217  4254 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.989948  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.148840  1554 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:40.149875  1554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51928
I20260709 13:01:40.150617  1554 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:40.150300  4293 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.081367  1705 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:40.152760  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.153939  4293 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:40.154094  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:39.964015  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 606) took 3384 ms. Trace:
I20260709 13:01:40.225036  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.224141  2296 rpcz_store.cc:276] 0709 13:01:36.579894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.580053 (+   159us) service_pool.cc:224] Handling call
0709 13:01:36.592886 (+ 12833us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:36.593348 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.593386 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:36.593952 (+   566us) log.cc:844] Serialized 52 byte log entry
0709 13:01:36.898297 (+304345us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:36.898342 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.898418 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.675605 (+2777187us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:39.871812 (+196207us) raft_consensus.cc:1685] finished
0709 13:01:39.963676 (+ 91864us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.963948 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":76944640,"thread_start_us":303935,"threads_started":1,"wal-append.queue_time_us":560958}
I20260709 13:01:39.901878  3918 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:40.226351  3918 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:40.226464  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.227731  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:39.989516  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.159152  1633 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:40.090572  4196 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.284143  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.284883  1634 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:40.284993  1557 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:40.285974  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.286675  1557 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:01:39.968328  4370 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:01:40.286998  4196 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:39.980083  4275 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:40.287348  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1314) took 1737 ms (client timeout 2166 ms). Trace:
I20260709 13:01:39.979370  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.187306  1552 raft_consensus.cc:2368] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8.
I20260709 13:01:40.208214  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:40.287912  1701 rpcz_store.cc:269] 0709 13:01:38.550219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.550869 (+   650us) service_pool.cc:224] Handling call
0709 13:01:40.287179 (+1736310us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b5400132ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:40.287288 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971624192}
I20260709 13:01:40.287940  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.289005  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.289345  1705 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:40.289412  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.289837  1702 raft_consensus.cc:2410] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:40.290328  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.290643  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:40.290892  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:40.281951  4293 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.291950  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.292428  1702 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:40.292207  4293 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.293078  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.293388  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.293701  1559 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:40.151082  1554 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.294984  4293 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:40.295534  3850 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:40.295784  1557 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:40.295843  4252 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.296144  3850 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.297514  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.298255  3850 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:40.298517  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.299119  1557 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:40.227424  3918 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.300247  4031 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:40.233215  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:40.247223  2260 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.888s (SENT) [suppressed 20 similar messages]
W20260709 13:01:40.301093  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53930.8us
I20260709 13:01:40.034010  4144 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:40.302552  1700 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:40.304499  1629 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.284143  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.308223  1633 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.253640  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.308912  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.309787  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.310036  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.310560  1630 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:40.252897  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.015s	sys 0.015s
W20260709 13:01:40.311180  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.015s	sys 0.015s
I20260709 13:01:40.276283  1442 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c4b89801fe624f179701cbd4b98f1490 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:40.255512  3687 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.281045  4149 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:40.282011  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:40.314579  4350 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.231s	user 0.004s	sys 0.009s
I20260709 13:01:40.315114  4149 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:39.962873  4175 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:40.316560  4175 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:40.316916  1705 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:40.317190  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.317762  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.318084  4175 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:40.319074  4175 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:40.319531  1630 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:40.281170  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:40.332224  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.308095  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.333559  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.310760  1635 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:40.345242  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.345854  1635 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:40.345917  4355 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:40.346522  4355 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:40.294853  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:39.972380  4372 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.325055  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:40.253515  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 397469us
W20260709 13:01:40.326062  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:40.350099  1634 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:40.351322  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1970007900000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:01:40.353377  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.356288  1631 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:40.290812  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:40.264048  4208 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.338238  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.377025  1632 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:40.376911  4208 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.378022  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.378614  1632 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:40.378872  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.515s (SENT)
I20260709 13:01:40.346714  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.318531  1557 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:40.280387  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2022) took 3979 ms (client timeout 2037 ms). Trace:
W20260709 13:01:40.380189  1560 rpcz_store.cc:269] 0709 13:01:36.300855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.366335 (+ 65480us) service_pool.cc:224] Handling call
0709 13:01:40.280142 (+3913807us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002bb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:40.280271 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766583552}
W20260709 13:01:40.381196  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.347576  4355 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.381397  4208 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:40.359899  4197 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:40.384279  1701 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:40.385309  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.386785  1631 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:01:40.388613  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.389663  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.390458  1631 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:40.370368  1554 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.392222  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.395402  1627 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:40.397505  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2011) took 3237 ms. Trace:
I20260709 13:01:40.397951  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:40.397867  1627 rpcz_store.cc:276] 0709 13:01:37.160479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.160940 (+   461us) service_pool.cc:224] Handling call
0709 13:01:37.218061 (+ 57121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.395906 (+3177845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.397257 (+  1351us) spinlock_profiling.cc:243] Waited 3.18 s on lock 0x7b5400343ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:40.397446 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7306465024}
I20260709 13:01:40.399281  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.399394  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.400194  1627 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.400321  1554 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:40.401739  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.402426  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.403013  1631 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:01:40.403890  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.403792  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:40.362423  1633 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:40.381052  1635 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:40.404541  2288 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:40.404860  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:40.405089  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.405858  1635 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:40.405997  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:40.406505  2295 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.406865  1635 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:40.374441  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.356910  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.382073  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.382350  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.408113  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.408581  1557 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:40.408943  1635 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:40.409622  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.409904  4149 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:40.410738  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.411012  4149 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.411614  2294 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:40.412998  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.413641  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.408152  1553 raft_consensus.cc:2368] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:40.419305  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.416551  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.422351  1560 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:40.422519  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:40.382429  3850 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:40.417217  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 102522us
I20260709 13:01:40.423480  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.424192  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:40.423527  3850 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.425077  4175 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:40.425599  4175 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:40.426203  4175 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:40.426397  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.433346  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:40.422408  4023 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:40.433636  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:40.434115  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:40.424580  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.417141  1557 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:40.434214  4023 consensus_queue.cc:1243] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:40.434450  4308 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:40.437201  4308 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.437780  4308 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.438042  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.439093  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:40.439329  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.440626  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.426780  3850 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:40.430003  2296 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:40.441757  1560 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.431443  4300 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:40.442644  3937 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:40.443128  3937 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.444083  3937 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.445389  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.432170  4323 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:40.415462  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.437072  2291 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.408133  1559 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:01:40.447781  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:40.428036  1627 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:40.441632  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.448539  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.445483  3937 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.449738  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.450855  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.441394  1557 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:40.452406  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.420115  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.430909  1630 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:40.446370  4323 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.480659  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.481325  1553 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:40.481551  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.471022  3767 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.473290  1627 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:40.452970  1557 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:40.473516  4208 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:40.484599  1559 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:40.475708  2293 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:40.475836  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:40.488484  2295 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:40.490016  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:40.399227  1632 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:40.492496  1630 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:40.493043  1559 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:40.494015  1551 raft_consensus.cc:2410] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:01:40.494663  4285 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.495457  4285 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.491334  4284 raft_consensus.cc:2764] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:40.491433  4197 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:40.479422  4308 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.478227  4175 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.437295  4023 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:40.478657  3850 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:40.481168  4323 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.405941  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.497947  1559 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:40.382010  4355 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.482738  1635 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:40.490518  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:40.527688  1559 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.510854  1557 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:40.514182  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:40.518100  1634 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:40.516636  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:40.379317  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:40.504773  1627 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:01:40.531862  1627 pending_rounds.cc:85] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 6
I20260709 13:01:40.532145  1627 pending_rounds.cc:107] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:01:40.532647  1627 raft_consensus.cc:2889] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.532828  4031 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:40.533941  4395 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:40.534333  4395 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:40.535115  4395 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.523494  4023 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:40.511580  1709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.011s	sys 0.003s
W20260709 13:01:40.538779  1709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.011s	sys 0.003s
I20260709 13:01:40.540391  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:40.541669  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50455.8us
I20260709 13:01:40.525750  2291 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:40.482410  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.476894  4394 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:40.546381  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:40.547578  1553 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:40.522467  1560 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:40.532087  4308 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:40.512873  1632 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:40.538319  1559 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.434115  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.556438  1554 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:40.559047  2295 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:40.555886  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:40.564141  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:40.568168  1634 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:01:40.570051  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:40.590548  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:40.595993  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 249369us
I20260709 13:01:40.560904  1629 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:40.586125  1555 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:40.599506  1559 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:40.599471  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89815.9us
I20260709 13:01:40.538275  4252 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:40.544972  4394 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.633837  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.551415  4372 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:40.611209  1627 raft_consensus.cc:1217] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-8.8]   Dedup: 3.6->[6.7-8.8]
I20260709 13:01:40.561136  4293 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:40.637226  1559 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:01:40.643227  1627 pending_rounds.cc:85] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 6
I20260709 13:01:40.613844  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:40.635902  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:01:40.647985  1627 pending_rounds.cc:107] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:40.551108  4395 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.648353  1627 raft_consensus.cc:2889] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.648280  1635 raft_consensus.cc:606] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:40.598990  3944 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:40.671499  1559 pending_rounds.cc:85] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 3
I20260709 13:01:40.671916  1559 pending_rounds.cc:107] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 7.4
W20260709 13:01:40.671684  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57951.4us
I20260709 13:01:40.672380  1559 raft_consensus.cc:2889] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.607613  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:40.634279  3981 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:40.599875  4031 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:40.600005  4303 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:40.611579  3850 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:40.643371  4386 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:01:40.639008  4314 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:40.638783  4265 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.638931  4153 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:40.666397  4140 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:40.667400  3767 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:40.604575  4254 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:01:40.642980  4372 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:40.645778  4252 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.634202  4300 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
W20260709 13:01:40.704871  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 110685us
I20260709 13:01:40.715104  1525 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:40.705101  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64012.2us
I20260709 13:01:40.644269  4394 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.646528  4144 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 13:01:40.697413  4356 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:40.689870  4127 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:40.795032  4127 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.795645  4127 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:40.799081   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:40.700166  4303 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:40.760111  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.686533  3981 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:40.709362  4031 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:01:40.798883  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74747.8us
I20260709 13:01:40.808017  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2208) took 1326 ms. Trace:
I20260709 13:01:40.714351  4372 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:40.693331  3850 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.705895  4265 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:40.815387  4385 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:01:40.815135  4031 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:40.810812  1628 rpcz_store.cc:276] 0709 13:01:39.481505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.481670 (+   165us) service_pool.cc:224] Handling call
0709 13:01:39.482087 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.700401 (+218314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.807747 (+1107346us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54002751d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:40.807956 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497475968}
I20260709 13:01:40.679224  4023 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.872680  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2084) took 1014 ms. Trace:
I20260709 13:01:40.873232  1709 rpcz_store.cc:276] 0709 13:01:39.858432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.858565 (+   133us) service_pool.cc:224] Handling call
0709 13:01:39.868152 (+  9587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.874822 (+  6670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.874876 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.964216 (+ 89340us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.539879 (+575663us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:40.539950 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.540054 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.872369 (+332315us) raft_consensus.cc:1685] finished
0709 13:01:40.872399 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.872596 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28354,"raft.queue_time_us":1427,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":61954,"thread_start_us":575113,"threads_started":1,"wal-append.queue_time_us":669140}
I20260709 13:01:40.700121  3918 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:01:40.706329  4314 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:40.810820  4396 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:01:40.911721  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111229us
I20260709 13:01:40.807211  4277 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:40.918812  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.920120  1635 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:40.920792  4409 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:40.921077  4256 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:40.922643  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.939973  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:40.846310  4153 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:40.930639  4407 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:01:40.911945  4254 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:40.953548  4323 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:40.957090  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:40.957773  1557 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:40.920878  4144 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:40.938987  4409 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.944613  4175 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:41.000207  4140 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:41.000962  4140 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:41.001709  4401 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:01:41.002355  4140 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:41.005039  4406 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:01:41.005530  4140 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.941422  4300 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:41.006642  4300 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.007896  4303 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.008826  4300 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.009410  4300 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:41.009723  4300 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:40.991356  3981 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.010479  4300 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.989625  4144 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.015095  4144 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:41.022634  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:40.985626  4252 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.993265  4254 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:01:41.027494  4409 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:40.985950  3918 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:40.956537  4153 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.030911  4153 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.033607  4285 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:41.034279  4355 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:01:41.036268  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:41.050329  4119 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.040772  4265 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:41.040906  4314 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.011541  3981 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:41.041926  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2028) took 1532 ms. Trace:
I20260709 13:01:41.056682  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2050) took 1071 ms. Trace:
I20260709 13:01:41.057266  1552 rpcz_store.cc:276] 0709 13:01:39.984918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.136701 (+151783us) service_pool.cc:224] Handling call
0709 13:01:41.053000 (+916299us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b54002f9c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:41.056592 (+  3592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112991616}
W20260709 13:01:41.042564  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.045478  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:41.047508  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.057052  1633 rpcz_store.cc:276] 0709 13:01:39.509738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.446610 (+936872us) service_pool.cc:224] Handling call
0709 13:01:40.446886 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.453063 (+  6177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.453089 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.471257 (+ 18168us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.555004 (+ 83747us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:40.555067 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.555166 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.041715 (+486549us) raft_consensus.cc:1685] finished
0709 13:01:41.041734 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.041861 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5805,"raft.queue_time_us":6261,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":82814,"threads_started":1,"wal-append.queue_time_us":218005}
I20260709 13:01:41.061836  1557 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:41.060241  1635 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:41.065972  1557 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:01:41.066843  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:41.067826  1702 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 4: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:41.071653  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:41.073160  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:41.075409  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 344) took 2183 ms (client timeout 1786 ms). Trace:
W20260709 13:01:41.075834  1558 rpcz_store.cc:269] 0709 13:01:38.892395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.014016 (+121621us) service_pool.cc:224] Handling call
0709 13:01:41.075226 (+2061210us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540036f1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:41.075341 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":804945792}
I20260709 13:01:41.011541  4300 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.076937  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:41.076004  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158497us
I20260709 13:01:41.078859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:41.079851  1702 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:41.079893  1709 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.081686  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:41.089707  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 482178us
I20260709 13:01:41.095191  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:41.195696  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:41.196213  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 101118us
I20260709 13:01:41.197280  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:41.112660  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:41.147408  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.063511  4314 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.183003  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:41.185014  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:41.203982  1558 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:41.204373  1705 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:41.185742  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:41.205209  1704 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:01:41.207052  1558 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:41.209306  1704 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:01:41.191856  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.002s	sys 0.007s
W20260709 13:01:41.212795  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.003s	sys 0.007s
W20260709 13:01:41.192103  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.005s
I20260709 13:01:41.194195  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:41.088541  4410 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.199410  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 377569us
W20260709 13:01:41.215478  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.004s	sys 0.009s
I20260709 13:01:41.162187  4278 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:41.199402  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.222970  1705 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:41.226442  1705 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:41.214959  4410 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:41.240729  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 298166us
I20260709 13:01:41.213151  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:41.214115  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.006s	sys 0.005s
W20260709 13:01:41.241952  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 569089us
I20260709 13:01:41.243805  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:41.188875  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.006s
I20260709 13:01:41.190688  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:41.244702  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.004s	sys 0.006s
I20260709 13:01:41.244916  1701 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:41.245215  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 866297us
W20260709 13:01:41.245831  1675 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.596s (SENT)
I20260709 13:01:41.247403  4031 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:41.221450  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.004s	sys 0.009s
I20260709 13:01:41.250690  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.250921  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.090931  4394 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.199522  3527 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:41.233862  4395 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.003s
I20260709 13:01:41.211583  4285 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:41.208206  4119 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:01:41.237903  4377 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:01:41.233245  4355 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:41.264176  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:41.266662  3928 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:41.277665  2292 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:41.238996  4175 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:01:41.280989  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 560170us
I20260709 13:01:41.261739  4278 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:41.211359  4314 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.305833  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.299624  4415 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.033630  4140 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.269s
I20260709 13:01:41.295801  4422 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:41.303867  4355 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:41.281164  4285 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.268705  4127 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.329854  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.337401  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.323408  4425 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:41.345757  1554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.020s	sys 0.011s
I20260709 13:01:41.332530  4140 consensus_queue.cc:1243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:01:41.354723  1554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.020s	sys 0.011s
I20260709 13:01:41.337459  4256 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:41.326387  4415 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:41.372963  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 175800us
I20260709 13:01:41.370615  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.423729  1701 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:41.346546  4144 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:41.472362  1709 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:41.477526  2260 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.641s (SENT)
I20260709 13:01:41.489205  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:41.363950  4377 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:41.376621  4140 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:41.490144  4403 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:01:41.372049  4256 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:41.469729  4417 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:41.388753  4031 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 13:01:41.493047  4175 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.497068  4394 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.496666  4140 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:41.533923  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 287267us
W20260709 13:01:41.534744  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.733s (SENT)
I20260709 13:01:41.535125  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:41.535706  4144 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:41.536943  4144 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:41.537163  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.537643  4418 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:01:41.538436  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:41.538622  4256 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.539242  4285 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.381515  4278 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:41.485846  4410 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:41.549465  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 219715us
I20260709 13:01:41.550381  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.551573  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:41.555585  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.572021  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1388) took 1201 ms. Trace:
I20260709 13:01:41.572556  1554 rpcz_store.cc:276] 0709 13:01:40.370949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.636034 (+265085us) service_pool.cc:224] Handling call
0709 13:01:40.636324 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.636543 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:41.358167 (+721624us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.571555 (+213388us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.571617 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:41.571668 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.571737 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.571946 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7345,"prepare.queue_time_us":670552,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"thread_start_us":897369,"threads_started":2}
I20260709 13:01:41.488427  4355 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.541668  4140 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.546209  4416 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:41.548095  4303 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:41.521071  4377 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:41.580006  1551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2033) took 4221 ms (client timeout 1515 ms). Trace:
I20260709 13:01:41.573817  3928 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:41.588587  3928 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:41.586970  1551 rpcz_store.cc:269] 0709 13:01:37.351643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.351799 (+   156us) service_pool.cc:224] Handling call
0709 13:01:41.562279 (+4210480us) spinlock_profiling.cc:243] Waited 3.11 s on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:41.573079 (+ 10800us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7150349184}
W20260709 13:01:41.637243  4432 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:01:41.638614  4426 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:01:41.585891  4303 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:41.684161  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.687350  4410 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:41.688172  4372 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.786s	user 0.006s	sys 0.006s
I20260709 13:01:41.688951  3850 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.688746  4372 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:41.690158  3850 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:41.690258  4431 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:41.690829  3850 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.691250  3850 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:41.690977  4431 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.694873  4023 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.697351  4127 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.698516  4414 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:41.698984  4414 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:41.700734  4414 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.704614  4431 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.707784  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:41.708050  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:41.708225  4293 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:41.708484  1554 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:41.634907  4314 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.690081  1635 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:41.708556  1553 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:41.723353  4429 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:01:41.724186  4377 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.727686  4415 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.728205  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 356) took 1254 ms. Trace:
I20260709 13:01:41.728613  1552 rpcz_store.cc:276] 0709 13:01:40.473495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.058181 (+584686us) service_pool.cc:224] Handling call
0709 13:01:41.058564 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.064260 (+  5696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.064306 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.078286 (+ 13980us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.370328 (+292042us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:41.484486 (+114158us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.484709 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.484832 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.727989 (+243157us) raft_consensus.cc:1685] finished
0709 13:01:41.728010 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.728142 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5264,"prepare.queue_time_us":159512,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":282133,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21196,"spinlock_wait_cycles":11820928,"thread_start_us":175338,"threads_started":1,"wal-append.queue_time_us":468634}
I20260709 13:01:41.653716  4303 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:41.730224  1554 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:41.732738  2296 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:01:41.733172  2296 pending_rounds.cc:85] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 7
I20260709 13:01:41.733399  2296 pending_rounds.cc:107] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:01:41.733621  2296 raft_consensus.cc:2889] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:41.734690  4444 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:41.735144  4444 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.735476  4444 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:41.708791  4293 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:41.741372  4372 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:41.742416  4293 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:41.745218  3981 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.000s
I20260709 13:01:41.750221  1553 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
W20260709 13:01:41.710180  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 333334us
I20260709 13:01:41.711243  2297 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:41.713126  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:41.759557  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46545.5us
I20260709 13:01:41.762934  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.764719  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.765625  2288 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:41.767810  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.770561  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:41.772692  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.773559  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:41.722664  4421 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
W20260709 13:01:41.780418  1560 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.001s	sys 0.013s
W20260709 13:01:41.780941  1560 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.001s	sys 0.013s
I20260709 13:01:41.743716  4293 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:41.781818  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.759194  3981 raft_consensus.cc:2764] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:41.785840  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:41.784324  1557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.008s	sys 0.008s
I20260709 13:01:41.786931  1626 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:01:41.784266  1555 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.009s	sys 0.038s
I20260709 13:01:41.790323  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:41.801929  1628 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:41.802011  1557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.008s	sys 0.008s
I20260709 13:01:41.797516  1633 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:41.791707  4444 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.780658  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:41.782087  4300 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.697s	user 0.009s	sys 0.003s
I20260709 13:01:41.809481  4127 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:41.776995  4377 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:41.812276  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:41.827104  1555 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.009s	sys 0.038s
I20260709 13:01:41.851140  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1379) took 1482 ms. Trace:
I20260709 13:01:41.810173  4127 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:41.851613  1560 rpcz_store.cc:276] 0709 13:01:40.368682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.552958 (+184276us) service_pool.cc:224] Handling call
0709 13:01:40.553387 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.561406 (+  8019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.561439 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.595674 (+ 34235us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.213401 (+617727us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.781747 (+568346us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.850836 (+ 69089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.850914 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.850951 (+    37us) raft_consensus.cc:1685] finished
0709 13:01:41.850962 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.851078 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76284,"prepare.queue_time_us":277515,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":41613,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":19040,"thread_start_us":1083318,"threads_started":2,"wal-append.queue_time_us":580149}
I20260709 13:01:41.858436  4444 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:41.858670  3937 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:41.859685  4445 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:41.863549  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:41.848778  3767 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:41.858454  4408 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.917s	user 0.009s	sys 0.001s
W20260709 13:01:41.887831  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67870us
W20260709 13:01:41.860280  4418 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:01:41.886024  4416 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:41.801677  4442 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:01:41.892593  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.892939  1553 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:41.894367  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:41.895385  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:41.908883  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.014s	sys 0.019s
W20260709 13:01:41.909956  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.014s	sys 0.019s
W20260709 13:01:41.897682  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 342121us
I20260709 13:01:41.905506  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:41.852249  4440 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:01:41.914855  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2037) took 2059 ms. Trace:
I20260709 13:01:41.915290  1630 rpcz_store.cc:276] 0709 13:01:39.855637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.493311 (+637674us) service_pool.cc:224] Handling call
0709 13:01:40.507051 (+ 13740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.507228 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:40.507519 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.835179 (+1327660us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.914489 (+ 79310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.914543 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:41.914606 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.914672 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.914796 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100091,"prepare.queue_time_us":48316,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":85,"raft.queue_time_us":352813,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":1784,"thread_start_us":1305764,"threads_started":3,"wal-append.queue_time_us":840558}
W20260709 13:01:41.918380  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.915882  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.916666  3527 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.000s
I20260709 13:01:41.922317  2289 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:41.922256  3767 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:41.925546  4127 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:41.926678  3767 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:41.930285  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 651) took 1424 ms. Trace:
I20260709 13:01:41.935530  2297 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:41.883375  4300 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:41.908099  4005 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.934769  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:41.936914  2290 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:41.912132  4252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.006s	sys 0.011s
I20260709 13:01:41.930809  2293 rpcz_store.cc:276] 0709 13:01:40.506288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.506412 (+   124us) service_pool.cc:224] Handling call
0709 13:01:40.506855 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:40.507229 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.507256 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:40.531422 (+ 24166us) log.cc:844] Serialized 52 byte log entry
0709 13:01:40.606781 (+ 75359us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:40.606815 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.606876 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.857748 (+1250872us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:41.929992 (+ 72244us) raft_consensus.cc:1685] finished
0709 13:01:41.930014 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.930204 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745,"raft.queue_time_us":3338,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":17854,"spinlock_wait_cycles":6814592,"thread_start_us":74977,"threads_started":1,"wal-append.queue_time_us":272712}
W20260709 13:01:41.956712  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 417074us
I20260709 13:01:41.963129  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 355) took 1489 ms. Trace:
I20260709 13:01:41.963850  1555 rpcz_store.cc:276] 0709 13:01:40.473236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.049948 (+576712us) service_pool.cc:224] Handling call
0709 13:01:41.050270 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.050479 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:41.827932 (+777453us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.962697 (+134765us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.962747 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.962818 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.962910 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.963054 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":309606,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4875,"thread_start_us":911251,"threads_started":2}
I20260709 13:01:41.924041  4117 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.935456  3527 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:41.975019  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58642.8us
W20260709 13:01:41.976073  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 139513us
I20260709 13:01:41.976436  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:41.960301  4252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.006s	sys 0.014s
I20260709 13:01:41.936168  4031 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:41.957041  4449 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.996527  4447 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.943400  4355 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:41.997670  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:41.998541  4447 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.001354  4416 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:42.016712  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53922.2us
I20260709 13:01:42.041217  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:42.048978  1675 leader_election.cc:400] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:42.049516  1675 leader_election.cc:403] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:01:42.026551  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:41.998891  4175 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.050859  4175 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.051981  4140 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.052865  4213 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:41.995129  4457 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.040495  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.053315  4213 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:42.062918  4431 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.041232  4395 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.069190  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:42.059396  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 121681us
I20260709 13:01:41.993388  4117 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.043561  4308 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:42.079070  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.087742  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:41.995235  3527 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.061095  4436 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:42.016475  4355 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:42.148409  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 107979us
W20260709 13:01:42.116819  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.076220  4252 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.083981  4314 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:42.004406  4031 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.085742  4444 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:42.091061  4437 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:42.061131  4446 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:01:42.068560  4461 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:42.004678  4449 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.158325  3928 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.157006  4218 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.075943  4431 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.175978  4475 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.164601  4458 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.062913  4457 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.176574  4023 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.094548  4454 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.096s
W20260709 13:01:42.201140  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 180744us
W20260709 13:01:42.215322  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 136431us
I20260709 13:01:42.092597  4395 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.170053  4314 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:42.224267  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.224334  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.179877  4452 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:42.229836  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.236246  2297 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.175237  4252 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.202894  4140 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:42.247381  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.219818  4437 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.235385  4314 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.216336  4475 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.246567  4472 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:01:42.256512  4197 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:42.235155  4440 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:42.165582  4213 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.265336  4437 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.243620  4452 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:42.274011  4470 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 13:01:42.274647  4467 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:01:42.208102  4218 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.280112  4175 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:42.234516  4023 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.257771  3767 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:42.281205  4140 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:42.281291  4459 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:42.282346  3527 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:42.282673  4252 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.282929  4457 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:42.284233  4466 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:01:42.216512  4455 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:01:42.285068  4395 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:42.269887  2295 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:42.286638  4449 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:42.289615  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.303784  2293 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 13:01:42.403561  3850 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.003s
I20260709 13:01:42.305234  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.160955  4436 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:42.315966  4425 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.953s	user 0.008s	sys 0.004s
I20260709 13:01:42.315271  4213 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.258128  4478 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:42.224054  4458 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:42.342016  4303 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.001s
W20260709 13:01:42.399022  4293 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.002s
I20260709 13:01:42.371726  4300 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.400805  4355 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:42.404489  4481 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:42.406982  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.430965  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 579) took 1094 ms. Trace:
I20260709 13:01:42.427798  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 652) took 1919 ms. Trace:
W20260709 13:01:42.641515  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 553918us
I20260709 13:01:42.645161  2293 pending_rounds.cc:85] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 7
I20260709 13:01:42.648718  2293 pending_rounds.cc:107] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 6.8
W20260709 13:01:42.649958  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.016s
I20260709 13:01:42.641062  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 358) took 2167 ms. Trace:
I20260709 13:01:42.650087  2293 raft_consensus.cc:2889] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:42.650429  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.000s	sys 0.016s
W20260709 13:01:42.650959  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 656513us
I20260709 13:01:42.650502  1557 rpcz_store.cc:276] 0709 13:01:40.474042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.069124 (+595082us) service_pool.cc:224] Handling call
0709 13:01:41.156901 (+ 87777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.808074 (+651173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.808141 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.850155 (+ 42014us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.007999 (+157844us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.008034 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.008096 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.362405 (+354309us) raft_consensus.cc:1685] finished
0709 13:01:42.362431 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.640979 (+278548us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143338,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":32875,"thread_start_us":763356,"threads_started":2,"wal-append.queue_time_us":347514}
I20260709 13:01:42.640178  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:42.527534  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 211712us
W20260709 13:01:42.583011  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.647799  2290 rpcz_store.cc:276] 0709 13:01:41.336541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.336667 (+   126us) service_pool.cc:224] Handling call
0709 13:01:41.337068 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.952588 (+615520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.430680 (+478092us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400644bd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:42.430885 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378920832}
I20260709 13:01:42.647373  2294 rpcz_store.cc:276] 0709 13:01:40.508816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.509074 (+   258us) service_pool.cc:224] Handling call
0709 13:01:40.509432 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.510692 (+  1260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.510725 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.513880 (+  3155us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.034718 (+520838us) spinlock_profiling.cc:243] Waited 9.13 ms on lock 0x7b480024e1d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:01:41.700175 (+665457us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b18003948a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:41.933129 (+232954us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.933177 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.933267 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.427504 (+494237us) raft_consensus.cc:1685] finished
0709 13:01:42.427525 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.427708 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":836,"raft.queue_time_us":4091,"raft.run_cpu_time_us":1830,"raft.run_wall_time_us":511293,"spinlock_wait_cycles":42888704,"thread_start_us":232737,"threads_started":1,"wal-append.queue_time_us":1812694}
I20260709 13:01:42.354229  4456 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:42.664211  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 359048us
W20260709 13:01:42.428814  4402 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.221s	user 0.000s	sys 0.012s
W20260709 13:01:42.431738  4468 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 13:01:42.669759  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.674742  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.425590  4023 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.627956  4254 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:42.425303  4377 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.420938  4437 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:42.660218  4218 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.659650  4252 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.649721  4461 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:42.665362  4425 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.528462  4285 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.665083  4447 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.664848  4444 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:42.676425  4416 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.693467  1598 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:42.694295  4474 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:01:42.694391  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2025) took 3185 ms. Trace:
I20260709 13:01:42.694942  4449 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.695895  4456 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:42.670352  4031 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:42.670166  4477 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.693194  4452 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.733114  4436 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:42.677870  4355 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:42.711931  4416 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.663590  4481 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.738695  4481 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:42.740983  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47605us
I20260709 13:01:42.696283  4252 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.743386  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:42.746316  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 365) took 1033 ms. Trace:
I20260709 13:01:42.748395  4372 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.749373  4372 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:42.677615  3527 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:42.707654  1631 rpcz_store.cc:276] 0709 13:01:39.508998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.412733 (+903735us) service_pool.cc:224] Handling call
0709 13:01:40.413006 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.417667 (+  4661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.417698 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.418125 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.679962 (+261837us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:40.680002 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.680069 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.428147 (+1748078us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:42.694034 (+265887us) raft_consensus.cc:1685] finished
0709 13:01:42.694060 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.694315 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2620,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"spinlock_wait_cycles":3977984,"thread_start_us":261438,"threads_started":1,"wal-append.queue_time_us":658068}
I20260709 13:01:42.666347  4300 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.663285  4303 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:42.717481  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:42.690316  4423 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:01:42.746903  1558 rpcz_store.cc:276] 0709 13:01:41.712700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.722384 (+  9684us) service_pool.cc:224] Handling call
0709 13:01:41.744447 (+ 22063us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:41.744846 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.744875 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:41.745509 (+   634us) log.cc:844] Serialized 52 byte log entry
0709 13:01:41.923382 (+177873us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.989497 (+ 66115us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.002707 (+ 13210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.002798 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.746030 (+743232us) raft_consensus.cc:1685] finished
0709 13:01:42.746057 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.746235 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47857,"prepare.queue_time_us":65247,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":26202,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":42026,"thread_start_us":177428,"threads_started":1,"wal-append.queue_time_us":579229}
I20260709 13:01:42.766758  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:42.665549  3850 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:42.761829  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 100048us
I20260709 13:01:42.688024  4254 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.697067  4377 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:42.783489  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108827us
I20260709 13:01:42.785373  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.785985  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.786422  1701 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:42.787214  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.788069  1709 raft_consensus.cc:2410] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:01:42.786602  2260 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.640s (SENT)
I20260709 13:01:42.787936  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.788791  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:42.788938  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.704107  4285 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.791698  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.791908  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.796090  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.796937  1701 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:42.797438  1701 pending_rounds.cc:85] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 7
I20260709 13:01:42.797719  1701 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:42.797995  1701 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.792428  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.793115  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:42.799897  1703 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:01:42.793406  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.794049  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.799046  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.801216  1709 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.801385  1705 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:42.802093  1707 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:42.802556  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.803256  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.804423  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.813458  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.831247  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.803347  1705 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:42.824028  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.833264  1700 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:42.836609  1701 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:42.837082  1701 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:42.863930  1703 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:42.863905  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.873056  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.837992  1701 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:42.804531  4023 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.763135  4308 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:01:42.818401  1707 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:42.897630  1701 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:42.905324  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:42.817888  4314 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 13:01:42.901314  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
W20260709 13:01:42.909361  2260 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.749s (SENT)
I20260709 13:01:42.914077  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:42.904044  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.788838  4452 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:01:42.834837  4409 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:42.834532  4127 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:42.917635  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.867003  4408 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:01:42.806564  4486 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.866413  4252 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.877477  4449 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.929272  4323 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:42.932432  1701 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:01:42.834494  4416 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.940424  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.901333  4489 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:42.942484  1701 pending_rounds.cc:85] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:01:42.945694  1701 pending_rounds.cc:107] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:42.946025  1701 raft_consensus.cc:2889] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.946429  1701 pending_rounds.cc:107] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:42.946703  1701 raft_consensus.cc:2889] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.915964  3767 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:43.007066  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 358138us
I20260709 13:01:42.943189  4323 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:42.837188  4444 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:43.018496  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 225705us
I20260709 13:01:42.922345  4314 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:43.006441  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 230925us
I20260709 13:01:43.016265  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:42.914458  4023 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.019639  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:43.011904  1675 leader_election.cc:400] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8.
I20260709 13:01:43.020255  1704 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:01:43.070711  2289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 566) took 3752 ms (client timeout 2137 ms). Trace:
I20260709 13:01:43.077651  4437 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:43.084956  4285 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:42.945312  4486 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:43.089003  4459 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.683s	user 0.005s	sys 0.006s
I20260709 13:01:42.875864  4461 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.011130  4254 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.091862  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:43.165727  4140 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.812s	user 0.011s	sys 0.000s
W20260709 13:01:43.089071  4487 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:01:43.082422  4323 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.074653  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:43.164074  4329 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.331s	user 0.023s	sys 0.040s
I20260709 13:01:43.092156  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:43.076082  4252 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:43.071803  1702 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:01:43.070641  1523 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.795s (SENT)
I20260709 13:01:43.081324  3850 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:43.162139  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.018476  4491 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:43.017089  4489 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:43.096189  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:42.941298  4456 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:43.747968  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 736175us
W20260709 13:01:43.748138  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 677490us
I20260709 13:01:43.257807  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2035) took 4867 ms. Trace:
W20260709 13:01:43.471269  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.258860  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 575) took 2798 ms. Trace:
I20260709 13:01:43.754730  2292 rpcz_store.cc:276] 0709 13:01:40.460341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.470999 (+ 10658us) service_pool.cc:224] Handling call
0709 13:01:40.471363 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.303179 (+831816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.258547 (+1955368us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54005d02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:43.258777 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840174464}
I20260709 13:01:43.756040  1707 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:43.753310  1556 rpcz_store.cc:276] 0709 13:01:38.390740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.231285 (+840545us) service_pool.cc:224] Handling call
0709 13:01:41.075492 (+1844207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.080740 (+  5248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.080899 (+   159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.080973 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:41.081038 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.081138 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.257542 (+2176404us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540036f1d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:43.257708 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4894,"raft.queue_time_us":220624,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":5994,"spinlock_wait_cycles":4240416512}
W20260709 13:01:43.247452  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 217377us
I20260709 13:01:43.762465  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:43.747370  2289 rpcz_store.cc:269] 0709 13:01:39.318080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.326778 (+  8698us) service_pool.cc:224] Handling call
0709 13:01:43.056823 (+3730045us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b5400538ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:43.070601 (+ 13778us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5966161792}
I20260709 13:01:43.764323  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.765484  1707 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:43.070662  4500 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.776146  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.777957  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:43.319399  4413 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.104s	user 0.000s	sys 0.011s
W20260709 13:01:43.748172  4329 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.940s	user 0.032s	sys 0.056s
I20260709 13:01:43.795264  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.782972  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.803300  1707 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:43.158381  3767 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:43.158421  4452 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.803750  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:43.811347  1709 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:43.811798  1709 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:43.167456  4023 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.223932  4504 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.798825  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:43.749502  1700 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:43.805199  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.819974  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.820690  1709 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.812709  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.821406  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:43.170060  4213 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 13:01:43.597904  4498 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:43.822621  1709 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.816887  4323 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:43.772492  4459 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:43.177211  4458 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.779964  4117 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:43.828099  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.829025  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 657) took 1602 ms. Trace:
W20260709 13:01:43.862825  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.829263  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.829879  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2047) took 2482 ms. Trace:
I20260709 13:01:43.864542  1635 rpcz_store.cc:276] 0709 13:01:41.347696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.347948 (+   252us) service_pool.cc:224] Handling call
0709 13:01:41.348351 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.721830 (+373479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.829648 (+2107818us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001e8a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:43.829818 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785427328}
I20260709 13:01:43.864028  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.866801  1707 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:43.830011  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:43.753511  4314 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.773941  4140 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:43.838541  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.868530  1702 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:43.869413  1552 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:43.869874  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.790031  4254 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.838352  4504 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.854748  4480 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:43.763247  4252 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.856419  4489 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:43.857132  4459 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.870664  1552 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:43.871341  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.896112  1553 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:43.896780  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.897080  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.897589  1552 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:43.897908  1553 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6.
I20260709 13:01:43.898661  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.898900  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:43.899398  1552 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:43.900300  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:43.862198  2295 rpcz_store.cc:276] 0709 13:01:42.226734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.226904 (+   170us) service_pool.cc:224] Handling call
0709 13:01:42.227221 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.419369 (+192148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.828759 (+1409390us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b540040b5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:43.828955 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97357696}
I20260709 13:01:43.878314  1702 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:43.774430  4285 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:43.918128  4285 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.881219  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.921077  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.922180  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.922681  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:43.838752  4505 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.926720  4505 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.923364  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:43.928779  3767 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:43.933813  4409 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.994s	user 0.010s	sys 0.000s
I20260709 13:01:43.934358  4441 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.934984  4441 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.935954  4486 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.936455  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2055) took 2045 ms. Trace:
I20260709 13:01:43.937774  4504 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.889751  1709 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:01:43.938539  4459 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.940208  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1397) took 2045 ms. Trace:
I20260709 13:01:43.940905  4459 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:43.877415  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:43.941275  4501 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.920s	user 0.003s	sys 0.000s
I20260709 13:01:43.941911  1556 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:43.169013  4449 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.674s
I20260709 13:01:43.942445  4449 consensus_queue.cc:1243] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:01:43.942906  2262 connection.cc:256] RPC call timeout handler was delayed by -1.1958565319999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:01:43.901414  1702 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:43.772272  4437 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:43.943888  4425 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:43.905490  1553 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:43.944612  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 665) took 1655 ms. Trace:
I20260709 13:01:43.904985  4480 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.945143  4437 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:43.945533  1553 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:43.943898  1702 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.946285  1702 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:43.823980  4452 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.946712  4480 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:43.912595  4475 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:43.913811  1700 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:43.886840  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:43.948367  4452 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:43.805972  4500 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.949643  4476 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:43.950081  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1391) took 3563 ms. Trace:
I20260709 13:01:43.914953  4472 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:43.942987  1556 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:43.944787  4425 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:43.917631  1705 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:01:43.936887  1632 rpcz_store.cc:276] 0709 13:01:41.890955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.891090 (+   135us) service_pool.cc:224] Handling call
0709 13:01:41.891404 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.916128 (+ 24724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.916157 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.916619 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.934172 (+1017553us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b18000450a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:43.230137 (+295965us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:43.230198 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.230300 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.936140 (+705840us) raft_consensus.cc:1685] finished
0709 13:01:43.936162 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.936387 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24330,"raft.queue_time_us":24938,"spinlock_wait_cycles":28609664,"thread_start_us":295676,"threads_started":1,"wal-append.queue_time_us":1957432}
I20260709 13:01:43.950556  1559 rpcz_store.cc:276] 0709 13:01:40.386652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.691945 (+305293us) service_pool.cc:224] Handling call
0709 13:01:40.692825 (+   880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.062638 (+1369813us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:42.062667 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.062710 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:42.062751 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.949869 (+1887118us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540030a7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:43.949908 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.950014 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45757,"raft.queue_time_us":47572,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"spinlock_wait_cycles":3044028800}
I20260709 13:01:43.929421  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.901136  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:43.951579  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:43.956809  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.341922  2292 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:44.345263  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:44.349167  2292 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:44.014448  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:44.107393  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 356794us
W20260709 13:01:44.352308  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.281553  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 97206us
I20260709 13:01:43.994103  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:43.978124  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.357708  2295 raft_consensus.cc:2410] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:01:44.279845  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.966001  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.360508  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:43.951800  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:43.952309  1704 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:44.033555  1702 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:44.375319  1702 pending_rounds.cc:85] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 1
I20260709 13:01:44.375710  1702 pending_rounds.cc:107] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:44.376085  1702 raft_consensus.cc:2889] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.377338  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2105) took 1282 ms. Trace:
I20260709 13:01:44.377835  1702 rpcz_store.cc:276] 0709 13:01:43.094703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.877930 (+783227us) service_pool.cc:224] Handling call
0709 13:01:43.878196 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.376979 (+498783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.377260 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:44.151124  1701 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.001s	sys 0.018s
W20260709 13:01:44.407409  1701 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.011s	sys 0.020s
I20260709 13:01:44.329559  2289 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:43.940624  1554 rpcz_store.cc:276] 0709 13:01:41.894541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.895001 (+   460us) service_pool.cc:224] Handling call
0709 13:01:41.895340 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:41.907491 (+ 12151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.907520 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:41.910022 (+  2502us) log.cc:844] Serialized 52 byte log entry
0709 13:01:42.069994 (+159972us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:42.097215 (+ 27221us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.097531 (+   316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.097626 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.931955 (+1834329us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b18002f0620. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:43.939886 (+  7931us) raft_consensus.cc:1685] finished
0709 13:01:43.939989 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.940133 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11736,"prepare.queue_time_us":27707,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":56623,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":20498,"spinlock_wait_cycles":11199232,"thread_start_us":126345,"threads_started":1,"wal-append.queue_time_us":230271}
W20260709 13:01:44.446424  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.446753  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.447598  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:01:44.068220  4423 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.018s	sys 0.041s
W20260709 13:01:44.454217  4423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.024s	sys 0.053s
I20260709 13:01:44.455549  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:44.456379  1523 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.511s (SENT)
I20260709 13:01:44.456869  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:44.457711  1702 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:44.461203  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:44.462082  1702 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:44.462917  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:44.464399  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.465157  2291 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:43.944988  2297 rpcz_store.cc:276] 0709 13:01:42.288715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.288905 (+   190us) service_pool.cc:224] Handling call
0709 13:01:42.289207 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.699902 (+410695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.700483 (+   581us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.701142 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.869842 (+168700us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:42.869892 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.869975 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.935505 (+1065530us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:43.944314 (+  8809us) raft_consensus.cc:1685] finished
0709 13:01:43.944335 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.944550 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114643,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":126,"spinlock_wait_cycles":53263744,"thread_start_us":566569,"threads_started":2,"wal-append.queue_time_us":246913}
I20260709 13:01:44.470469  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.470690  1702 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:44.471199  1702 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:44.472208  1702 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.473259  1702 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:44.479357  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2089) took 1522 ms. Trace:
W20260709 13:01:44.095259  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 317330us
I20260709 13:01:44.479923  1701 rpcz_store.cc:276] 0709 13:01:42.945056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.016370 (+ 71314us) service_pool.cc:224] Handling call
0709 13:01:43.016949 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.044780 (+ 27831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.074903 (+ 30123us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.087787 (+ 12884us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.267255 (+179468us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:44.445343 (+1178088us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.464627 (+ 19284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.464738 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.464799 (+    61us) raft_consensus.cc:1685] finished
0709 13:01:44.464818 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.464975 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30779,"raft.queue_time_us":72562,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1203271,"threads_started":2,"wal-append.queue_time_us":824252}
I20260709 13:01:43.960408  4117 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:43.978343  4491 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:44.158998  4005 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:44.184834  4434 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.476s	user 0.006s	sys 0.005s
W20260709 13:01:44.512566  2290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.000s	sys 0.038s
W20260709 13:01:44.513679  2290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.000s	sys 0.039s
I20260709 13:01:44.487931  1707 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:01:44.488834  1557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.019s	sys 0.013s
I20260709 13:01:44.489504  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:44.515954  1557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.019s	sys 0.013s
W20260709 13:01:44.490036  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.482242  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.46256e+06us
I20260709 13:01:44.516860  1707 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:44.492411  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2104) took 1398 ms. Trace:
I20260709 13:01:43.871110  4498 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.493158  4425 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:43.994381  4409 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:44.241645  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.894014  4254 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:43.951720  1708 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:44.067162  4452 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.448058  4285 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:44.159175  4442 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:43.978966  4475 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:44.448130  4437 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.506484  2292 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:44.511744  1523 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.582s (SENT)
I20260709 13:01:44.560675  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:44.562302  2289 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:44.517395  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:44.517632  1705 rpcz_store.cc:276] 0709 13:01:43.094381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.877726 (+783345us) service_pool.cc:224] Handling call
0709 13:01:43.917387 (+ 39661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.492036 (+574649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.492335 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.950173  4476 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.565104  1709 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:43.960675  4252 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:44.515420  4472 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:44.573856  4442 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.066890  4300 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:44.584491  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.010s	sys 0.036s
W20260709 13:01:44.585175  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.010s	sys 0.036s
I20260709 13:01:44.545604  4500 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:44.591811  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.625746  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.618408  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2111) took 1327 ms. Trace:
I20260709 13:01:44.627589  1702 rpcz_store.cc:276] 0709 13:01:43.290993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.470034 (+1179041us) service_pool.cc:224] Handling call
0709 13:01:44.470439 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.617893 (+147454us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:44.617938 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.617998 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:44.618051 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.618148 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.618326 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":100162,"threads_started":1}
I20260709 13:01:44.539476  4208 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.519099  4005 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.512439  4117 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.503680  4449 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:01:44.572414  4285 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:44.519630  4491 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:44.651297  4461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.004s	sys 0.027s
I20260709 13:01:44.588339  4252 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:44.549660  4409 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:44.714216  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87650us
I20260709 13:01:44.564795  4475 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.753011  2293 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:44.577278  4437 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:44.727133  4461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.004s	sys 0.027s
I20260709 13:01:44.712617  4520 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.572026  4452 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.558267  4425 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.747383  4300 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.766294  4520 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:44.771369  4514 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:01:44.795497  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 284876us
I20260709 13:01:44.713351  4208 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.799098  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.716862  4285 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:44.816293  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:44.817520  2295 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:44.797374  4461 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:44.797767  4300 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.826480  4461 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.828740  4425 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:44.841774  4475 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:44.843035  4517 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:44.792424  4472 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.852370  4425 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:44.853848  4476 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:44.854413  4498 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:44.854823  4498 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:44.855041  4498 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:44.855593  4498 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.856822  4254 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:44.829569  4536 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:44.838382  4461 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:44.876710  4515 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:01:44.877997  4525 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:44.862860  4536 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:44.886148  4536 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:44.922802  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 410752us
I20260709 13:01:44.923167  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1403) took 1763 ms. Trace:
I20260709 13:01:44.935286  1555 rpcz_store.cc:276] 0709 13:01:43.159506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.161326 (+  1820us) service_pool.cc:224] Handling call
0709 13:01:43.161730 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:43.162178 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.162218 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:43.703105 (+540887us) log.cc:844] Serialized 52 byte log entry
0709 13:01:44.108215 (+405110us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:44.108297 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.108403 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.922303 (+813900us) raft_consensus.cc:1685] finished
0709 13:01:44.922392 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.922890 (+   498us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108303,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5235,"thread_start_us":315012,"threads_started":1,"wal-append.queue_time_us":857988}
I20260709 13:01:44.924552  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1401) took 1793 ms. Trace:
I20260709 13:01:44.938490  1557 rpcz_store.cc:276] 0709 13:01:43.130814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.130939 (+   125us) service_pool.cc:224] Handling call
0709 13:01:43.163539 (+ 32600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.289150 (+125611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.289193 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.289730 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.516360 (+1226630us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:44.516397 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.516460 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.923957 (+407497us) raft_consensus.cc:1685] finished
0709 13:01:44.924009 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.924366 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":250842,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":132147,"thread_start_us":935888,"threads_started":2,"wal-append.queue_time_us":1198822}
W20260709 13:01:44.923015  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 405799us
W20260709 13:01:44.924511  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.928267  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 446492us
W20260709 13:01:44.934324  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 416847us
I20260709 13:01:44.955658  4491 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:44.955726  4005 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:44.956133  4409 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:44.963416  4117 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:44.969512  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.972592  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 137868us
I20260709 13:01:44.974524  4452 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:44.961529  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 226439us
I20260709 13:01:44.986011  4442 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:44.989164  4536 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:44.990707  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.990315  4536 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:44.991283  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.991110  4285 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:44.997341  4539 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:44.998353  4437 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:44.999362  4437 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:45.013818  4442 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.014562  4442 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.003896  4437 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:45.017107  4437 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.017781  4536 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.017796  4442 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.018438  4300 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.018958  4437 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:45.029961  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:45.033607  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:45.034200  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2124) took 1124 ms. Trace:
I20260709 13:01:45.034679  1709 rpcz_store.cc:276] 0709 13:01:43.909469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.939279 (+ 29810us) service_pool.cc:224] Handling call
0709 13:01:45.033798 (+1094519us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54001342d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:45.033983 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467399936}
I20260709 13:01:45.035164  4538 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:45.035868  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:45.036810  1705 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:45.031327  1702 raft_consensus.cc:2376] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8. Re-sending same reply.
I20260709 13:01:45.038370  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:45.039256  1705 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:45.035903  4538 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:45.041281  4538 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:45.042652  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.043623  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.044008  4545 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:45.045017  4541 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:45.045003  4545 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.046123  4541 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:45.045291  2260 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.786s (SENT)
I20260709 13:01:45.062960  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 674) took 2018 ms. Trace:
I20260709 13:01:45.062842  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:45.060238  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:45.067216  4452 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.086078  4558 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.086941  4558 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.089507  4558 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.090059  4558 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:45.090803  4558 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:45.093698  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48613.1us
I20260709 13:01:45.063496  2290 rpcz_store.cc:276] 0709 13:01:43.044458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.070886 (+ 26428us) service_pool.cc:224] Handling call
0709 13:01:43.079397 (+  8511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.860005 (+780608us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:43.860273 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.828528 (+968255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.828995 (+   467us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:44.829068 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.057747 (+228679us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54002c8dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:45.060658 (+  2911us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.062782 (+  2124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":327642,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":1794641792,"thread_start_us":967824,"threads_started":2,"wal-append.queue_time_us":901519}
W20260709 13:01:45.107966  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.101070  1675 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6.
I20260709 13:01:45.109661  4500 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.110476  4500 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.112462  4265 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:45.113049  4265 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:45.113511  4265 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:45.113451  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2128) took 1159 ms. Trace:
I20260709 13:01:45.121469  4500 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.103611  4117 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:45.147611  4265 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:45.151121  4457 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 13:01:45.161616  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 85198.3us
W20260709 13:01:45.162755  4493 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 2.243s	user 0.013s	sys 0.005s
I20260709 13:01:45.165573  1555 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:45.172745  4541 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.176151  4254 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:45.177138  4254 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.171432  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.222476  4422 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:45.236852  4554 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.237876  1626 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:45.238765  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:45.263958  4477 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.265683  4532 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:45.216809  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 108501us
I20260709 13:01:45.264770  4477 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.223466  4422 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.301888  4505 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.302292  4422 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.121933  4452 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.115573  1707 rpcz_store.cc:276] 0709 13:01:43.910560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.517747 (+607187us) service_pool.cc:224] Handling call
0709 13:01:44.518037 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.518289 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:44.948415 (+430126us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.001339 (+ 52924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.020609 (+ 19270us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:45.050251 (+ 29642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.059352 (+  9101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.070078 (+ 10726us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":226909,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":24509,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":333,"thread_start_us":413191,"threads_started":2,"wal-append.queue_time_us":73996}
I20260709 13:01:45.302558  4532 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.326522  4461 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.328938  4461 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.331845  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.332821  1556 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:45.334460  4553 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:45.237942  4554 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.410820  1705 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:45.342092  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1399) took 2212 ms. Trace:
I20260709 13:01:45.394439  1703 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.418133  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.419379  1556 raft_consensus.cc:2410] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:45.419868  4554 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:45.390955  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2106) took 2251 ms (client timeout 2138 ms). Trace:
W20260709 13:01:45.420509  1700 rpcz_store.cc:269] 0709 13:01:43.095007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.827835 (+732828us) service_pool.cc:224] Handling call
0709 13:01:45.346452 (+1518617us) spinlock_profiling.cc:243] Waited 51.7 ms on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:45.346642 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118901632}
I20260709 13:01:45.421959  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:45.393826  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.391976  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.425547  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.426594  1556 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:45.423558  1634 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:45.345588  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:45.429297  4452 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.344249  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.435530  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:45.436112  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.390662  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.215991  4005 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.302304  4477 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.441641  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.442560  1626 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:45.443907  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.444895  1626 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:45.446081  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.446991  1626 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:45.447186  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 355880us
I20260709 13:01:45.423507  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:45.448944  4452 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:45.450802  4452 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:45.451359  4452 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:45.449774  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:45.448777  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.413451  1551 rpcz_store.cc:276] 0709 13:01:43.129826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.130009 (+   183us) service_pool.cc:224] Handling call
0709 13:01:43.171669 (+ 41660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.944063 (+772394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.944094 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.944135 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:43.944192 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.341548 (+1397356us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54002bbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:45.341618 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.341887 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775819520}
I20260709 13:01:45.342077  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.448562  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:45.457023  1632 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:45.335234  4558 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.277760  4545 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.465951  4441 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:45.470378  4495 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 2.448s	user 0.014s	sys 0.011s
I20260709 13:01:45.470247  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.472766  2290 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:45.475168  4536 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:45.477300  4536 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:45.477667  4536 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:45.478636  4536 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:45.480026  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.482671  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 842) took 3096 ms. Trace:
I20260709 13:01:45.485630  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.485852  4005 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.488696  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.486385  2289 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:45.490228  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:45.490787  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.491587  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.492025  4441 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:45.492308  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.492846  4441 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:45.490960  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:45.493916  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.494498  2289 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:45.495153  4498 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.493098  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.497107  2292 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:45.494837  4441 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.499274  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.500234  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.500231  1551 raft_consensus.cc:2410] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:45.482580  4372 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.501741  2260 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.859s (SENT)
I20260709 13:01:45.501895  4372 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.502328  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:45.484505  4127 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:45.503311  4254 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.491413  1626 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:01:45.504341  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2085) took 2700 ms (client timeout 1743 ms). Trace:
I20260709 13:01:45.504372  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1400) took 2373 ms. Trace:
W20260709 13:01:45.504834  1708 rpcz_store.cc:269] 0709 13:01:42.803835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.805351 (+  1516us) service_pool.cc:224] Handling call
0709 13:01:45.503854 (+2698503us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b5400097254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:45.504075 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265895680}
I20260709 13:01:45.506299  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.506373  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:45.505352  1558 rpcz_store.cc:276] 0709 13:01:43.130347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.130508 (+   161us) service_pool.cc:224] Handling call
0709 13:01:43.171231 (+ 40723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.951844 (+780613us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:44.465233 (+513389us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.718396 (+253163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.718467 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:44.718550 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.503820 (+785270us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54000e9a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:45.503914 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.504159 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6858,"prepare.queue_time_us":622691,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":48679,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5136,"spinlock_wait_cycles":1794575616,"thread_start_us":729223,"threads_started":2,"wal-append.queue_time_us":363704}
I20260709 13:01:45.507201  1555 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:45.484476  4520 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.503121  4127 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.508733  1701 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:45.495464  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.501818  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:45.510802  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.509675  2291 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:45.513603  1556 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:45.479600  4414 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:45.447317  4538 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.515144  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:45.517369  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.506417  4377 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.520314  4127 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.527951  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.485397  2296 rpcz_store.cc:276] 0709 13:01:42.386575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.408603 (+ 22028us) service_pool.cc:224] Handling call
0709 13:01:42.409130 (+   527us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:42.424034 (+ 14904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.424080 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:42.424863 (+   783us) log.cc:844] Serialized 78 byte log entry
0709 13:01:42.830415 (+405552us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.830472 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.830577 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.469034 (+2638457us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:45.482081 (+ 13047us) raft_consensus.cc:1685] finished
0709 13:01:45.482102 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.482477 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19805,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":37909888,"thread_start_us":186520,"threads_started":1,"wal-append.queue_time_us":517623}
I20260709 13:01:45.521821  4372 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.516137  4117 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:45.529207  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:45.530058  1708 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:45.529844  4117 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.531414  4208 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.522469  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2090) took 2509 ms. Trace:
I20260709 13:01:45.522770  4355 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:45.483543  4265 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:45.527577  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.193s (SENT) [suppressed 48 similar messages]
W20260709 13:01:45.525900  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.335333  4553 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.532752  1703 rpcz_store.cc:276] 0709 13:01:43.013450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.013649 (+   199us) service_pool.cc:224] Handling call
0709 13:01:43.014130 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.394098 (+2379968us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54004838a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:01:45.458389 (+ 64291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.522114 (+ 63725us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b54004838a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e7a02 00007f70ce9e3dd6 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:01:45.522387 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839332864}
W20260709 13:01:45.560966  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47058.5us
W20260709 13:01:45.561586  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.575090  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.561561  4117 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.608078  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:01:45.570758  4539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.004s	sys 0.008s
I20260709 13:01:45.596156  1700 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
W20260709 13:01:45.613572  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.615736  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:45.616453  1632 raft_consensus.cc:2376] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9. Re-sending same reply.
I20260709 13:01:45.599797  4536 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:45.633841  4553 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.634509  4560 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:45.635605  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.622225  4539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.005s	sys 0.009s
W20260709 13:01:45.637502  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.676474  4308 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.680176  4308 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:45.680758  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.682370  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.682885  4308 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.638334  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.683887  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.684755  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.683758  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.689195  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 187505us
I20260709 13:01:45.600180  4461 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:01:45.692296  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.695555  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.703186  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.709349  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:45.654429  4492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.754s	user 0.038s	sys 0.024s
I20260709 13:01:45.672127  4481 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:45.701339  1523 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.671s (SENT)
I20260709 13:01:45.715464  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.625244  4532 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:45.718132  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 269821us
I20260709 13:01:45.722597  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:45.726532  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.712321  4492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.819s	user 0.041s	sys 0.031s
W20260709 13:01:45.751838  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44058.3us
I20260709 13:01:45.752020  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.752867  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.753172  1552 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:01:45.752895  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67418.3us
I20260709 13:01:45.755834  2289 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:45.756258  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.724277  3918 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:45.753808  2296 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:45.759500  1558 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:45.755298  4410 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:45.762149  4410 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:01:45.762228  1523 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.743s (SENT)
I20260709 13:01:45.762956  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.765759  1551 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:01:45.766441  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45641.1us
I20260709 13:01:45.767356  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.770774  2294 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:45.772197  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:45.774410  1630 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:45.775867  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.780939  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:45.784088  1556 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:45.787494  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.788197  1632 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:45.788292  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.789680  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.790012  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.790702  1553 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:45.790947  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.791793  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.792018  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.792167  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.792589  1553 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:45.792999  1558 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:45.794370  1555 pending_rounds.cc:85] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 2
I20260709 13:01:45.785658  1559 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:01:45.785349  2260 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.888s (SENT)
W20260709 13:01:45.794365  1523 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.800s (SENT)
I20260709 13:01:45.795959  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.796756  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:45.777268  4308 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:45.779995  4538 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:45.801883  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.785797  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.794759  1555 pending_rounds.cc:107] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:01:45.803385  1555 raft_consensus.cc:2889] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:45.804303  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.805510  1560 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:45.779554  3918 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:45.810045  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.821887  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.821916  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.840421  4449 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:45.841123  4449 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:45.842326  4572 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:45.837587  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.800117  4308 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.849664  1634 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:45.852412  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 160455us
I20260709 13:01:45.845214  1553 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:45.838930  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:45.859004  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107692us
I20260709 13:01:45.853075  1551 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
W20260709 13:01:45.843731  4568 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:45.862843  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.848711  4574 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:01:45.806733  4117 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:45.861443  4573 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:45.853360  4308 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.881175  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.849494  3918 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:45.887251  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 92948.2us
W20260709 13:01:45.864385  4569 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:01:45.889533  1703 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:45.876380  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.889607  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.895051  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.898411  4431 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:45.899523  4449 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:45.903940  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:45.929044  1634 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:45.873227  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.895085  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:45.913326  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.913693  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2989575880000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 13:01:45.932639  1674 connection.cc:256] RPC call timeout handler was delayed by -1.231457588000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 13:01:45.917130  4452 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:45.919360  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2102) took 2825 ms (client timeout 1944 ms). Trace:
I20260709 13:01:45.923312   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:45.923630  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:45.924572  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:45.930579  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.904426  1709 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:01:45.905764  1523 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.842s (SENT)
I20260709 13:01:45.931895  1628 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:45.931614  1551 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
W20260709 13:01:45.933095  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2026575879999655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:01:45.939608  1674 connection.cc:256] RPC call timeout handler was delayed by -1.1756575879999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:01:45.940021  1674 connection.cc:256] RPC call timeout handler was delayed by -1.0586575880000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
I20260709 13:01:45.939286  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:45.936751  1704 rpcz_store.cc:269] 0709 13:01:43.093765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.777307 (+683542us) service_pool.cc:224] Handling call
0709 13:01:45.919008 (+2141701us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001b8354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:45.919166 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299760000}
I20260709 13:01:45.938629  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:45.963618  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.966694  1556 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
W20260709 13:01:45.967094  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 188742us
W20260709 13:01:45.974225  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.976692  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.977257  4562 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:45.978029  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.979305  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:45.980224  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 85304us
I20260709 13:01:45.980522  4431 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:45.982234  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58721.9us
I20260709 13:01:45.978081  1632 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:45.983438  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.976931  4117 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:45.967555  4449 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:46.022939  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.984143  1631 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:45.993474  1554 pending_rounds.cc:85] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:01:46.024987  1554 pending_rounds.cc:107] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:01:46.025352  1554 raft_consensus.cc:2889] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:45.995411  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:45.997916  1706 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:46.004258  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:46.008313  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.027057  1633 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.013836  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:46.015242  4565 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.250s	user 0.002s	sys 0.000s
W20260709 13:01:46.030514  2260 connection.cc:256] RPC call timeout handler was delayed by -2.0235477849999484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
W20260709 13:01:46.031168  2260 connection.cc:256] RPC call timeout handler was delayed by -2.0226477849999469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260709 13:01:46.033877  1634 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:46.040869  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:46.048971  1703 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:46.034520  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.054186  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:45.885704  4410 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.056093  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.060549  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.053683  2260 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.961s (SENT)
I20260709 13:01:46.060267  4372 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:46.048112  4322 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:46.073196  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.066766  4372 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.051649  4117 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:46.080348  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.058920  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.080410  1702 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 3.
I20260709 13:01:46.072499  4322 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:46.082315  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 176625us
I20260709 13:01:46.082512  4322 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:46.082930  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.085430  4377 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:46.086606  4441 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:46.087157  4441 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:46.088591  4441 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:46.090022  4117 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.090535  4441 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.094414  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:46.096316  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:46.100073  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56570.3us
I20260709 13:01:46.100430  4529 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:46.100863  2260 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.969s (SENT)
I20260709 13:01:46.101091  4529 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:46.101382  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.101709  4529 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.102345  4394 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:46.103734  2260 connection.cc:256] RPC call timeout handler was delayed by -1.6824477849999653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26230000000000003s for a response
W20260709 13:01:46.104115  2260 connection.cc:256] RPC call timeout handler was delayed by -1.6646354619999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
I20260709 13:01:46.104177  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:46.104507  2260 connection.cc:256] RPC call timeout handler was delayed by -1.6345357769999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:01:46.104841  2260 connection.cc:256] RPC call timeout handler was delayed by -1.1034050549999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27740000000000004s for a response
I20260709 13:01:46.104923  1631 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:46.105185  2260 connection.cc:256] RPC call timeout handler was delayed by -1.0568403839999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
I20260709 13:01:46.106853  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:46.108458  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:46.080412  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.086645  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.082922  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:46.129339  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.133106  1634 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:46.137187  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.138003  1703 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:46.141173  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.141851  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:46.143018  4437 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:46.143172  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.143515  4437 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.144186  4394 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:46.144766  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.144896  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.145823  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87023.6us
I20260709 13:01:46.146234  4377 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.146310  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.147184  4263 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:46.149500  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.156131  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:46.159279  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 847) took 2165 ms (client timeout 1691 ms). Trace:
I20260709 13:01:46.160617  4452 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:46.181878  2293 rpcz_store.cc:269] 0709 13:01:43.993630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.993794 (+   164us) service_pool.cc:224] Handling call
0709 13:01:46.158849 (+2165055us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54005a0fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.159027 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910610816}
I20260709 13:01:46.161911  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:46.162707  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87659us
I20260709 13:01:46.177347  1633 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:46.179082  1631 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:46.077065  4372 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.186467  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.188661  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.193363  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.159415  4322 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:46.194505  1632 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:46.176026  4437 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.180094  4377 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.204330  2290 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:01:46.207103  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.208076  1631 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.208508  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.251506  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.210260  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.267200  1702 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:01:46.230967  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.272119  1628 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:01:46.273519  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 92738.2us
I20260709 13:01:46.209867  4368 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:46.284587  4368 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:46.286288  4368 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:46.320842  4535 consensus_queue.cc:1175] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" responder_term: 6 status { last_received { term: 4 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: [5.5-5.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:46.320762  4492 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:46.320538  4368 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.208014  4377 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.288343  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.290733  4440 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.293383  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.293968  4265 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.295624  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 188864us
I20260709 13:01:46.296233  4545 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.348502  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.288179  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 142007us
I20260709 13:01:46.304706  4562 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.305027  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.307906  4456 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.313529  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.316828  4308 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.288136  4459 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.332686  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.341811  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.342334  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.344604  1634 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:46.343700  1703 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:46.376457  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.203393  4431 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:46.351436  4265 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.352581  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:46.348361  4440 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:46.355901  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 386) took 3567 ms (client timeout 1961 ms). Trace:
I20260709 13:01:46.360441  1702 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:46.206561  4394 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.367939  1629 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
W20260709 13:01:46.378960  1706 rpcz_store.cc:269] 0709 13:01:42.788277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.788621 (+   344us) service_pool.cc:224] Handling call
0709 13:01:46.355500 (+3566879us) spinlock_profiling.cc:243] Waited 3.21 s on lock 0x7b5400165154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.355800 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7376639744}
I20260709 13:01:46.386509  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.391638  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.398833  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.400101  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.403143  1633 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:46.414209  1630 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:46.414888  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.418363  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:46.419663  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.421536  1629 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:46.423656  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.424938  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:46.425482  1551 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.428458  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:46.428983  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 154521us
I20260709 13:01:46.429733  4265 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.430310  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.430331  4140 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:46.431342  4494 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:46.431552  4440 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:46.432221  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.420424  1702 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:46.451045  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.384055  4394 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:46.440431  1703 raft_consensus.cc:2376] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
W20260709 13:01:46.480444  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50706.8us
I20260709 13:01:46.487439  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.494366  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.483706  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.499372  1703 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:46.472942  4394 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:46.503957  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 151430us
I20260709 13:01:46.500969  1557 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:46.397946  4545 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.507932  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.509800  1631 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:46.510846  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.514904  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:46.517877  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.398448  4456 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.515679  1702 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:46.468977  4431 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:46.419353  4562 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.390128  4459 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.527072  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.524740  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.528493  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.531111  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:46.479743  4372 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.539070  4322 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:46.542610  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:46.539409  4395 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:46.545991  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:46.545957  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 279259us
W20260709 13:01:46.548427  1558 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.036s	sys 0.011s
W20260709 13:01:46.549607  1558 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.036s	sys 0.011s
I20260709 13:01:46.549728  1706 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:46.554947  1634 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:01:46.554761  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.560073  1702 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:46.558838  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:46.576905  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2056) took 1176 ms. Trace:
I20260709 13:01:46.578616  1631 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:46.579291  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2057) took 1185 ms. Trace:
I20260709 13:01:46.577368  1633 rpcz_store.cc:276] 0709 13:01:45.393659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.543925 (+1150266us) service_pool.cc:224] Handling call
0709 13:01:46.554486 (+ 10561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.563427 (+  8941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.563453 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:46.569423 (+  5970us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:46.569487 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.569562 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.569685 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.589933  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.579684  1631 rpcz_store.cc:276] 0709 13:01:45.393977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.564174 (+1170197us) service_pool.cc:224] Handling call
0709 13:01:46.578447 (+ 14273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.579043 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.579226 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.596235  1551 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:46.597496  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:46.597954  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:46.600214  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 280682us
I20260709 13:01:46.554759  4127 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:46.580180  4480 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:46.603479  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:46.607245  1631 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:46.607899  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2059) took 1213 ms. Trace:
I20260709 13:01:46.608220  1631 rpcz_store.cc:276] 0709 13:01:45.394538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.594684 (+1200146us) service_pool.cc:224] Handling call
0709 13:01:46.604486 (+  9802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.607653 (+  3167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.607847 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.609987  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.612205  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:46.612792  1551 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.573493  4529 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.617059  1633 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:46.622306  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71176.9us
I20260709 13:01:46.626441  4175 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.627058  4175 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:46.621989  4263 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:01:46.626585  4127 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.630489  4127 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:46.631137  4127 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.603011  4308 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.607398  4480 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:46.632149  4480 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.606693  4410 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:46.634321  4410 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:46.635428  4410 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:46.627552  4175 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.636525  4252 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.637181  4252 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:46.605060  4494 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:46.646678  1633 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:46.591460  4437 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:46.659858  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.695712  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 405394us
W20260709 13:01:46.708102  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.645843  4494 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.748558  2297 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:46.792240  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.792999  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55227.6us
W20260709 13:01:46.792385  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.793061  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60531us
I20260709 13:01:46.748844  4494 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.797186  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.799422  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:46.812227  4585 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:01:46.813102  4562 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.813269  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 673) took 3784 ms. Trace:
I20260709 13:01:46.814206  4598 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:46.813719  2288 rpcz_store.cc:276] 0709 13:01:43.028731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.028857 (+   126us) service_pool.cc:224] Handling call
0709 13:01:43.044683 (+ 15826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.075633 (+ 30950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.075684 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.076636 (+   952us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.133415 (+3056779us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b1800320c20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:01:46.538990 (+405575us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:46.539074 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.539733 (+   659us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.812859 (+273126us) raft_consensus.cc:1685] finished
0709 13:01:46.812882 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.813109 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":502,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":101471488,"thread_start_us":401116,"threads_started":1,"wal-append.queue_time_us":3733995}
I20260709 13:01:46.813591  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2112) took 3442 ms. Trace:
I20260709 13:01:46.816052  1701 rpcz_store.cc:276] 0709 13:01:43.371045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.485238 (+1114193us) service_pool.cc:224] Handling call
0709 13:01:45.507967 (+1022729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.509100 (+  1133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.813363 (+1304263us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400097250. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.813528 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350741248}
I20260709 13:01:46.818830  1552 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:46.813913  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:46.833451  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:46.844616  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.849020  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.847462  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:46.848080  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:46.806442  4372 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.858870  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.859815  1555 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:46.860375  4005 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:46.849192  4117 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.861078  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.861514  4117 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:46.861975  1555 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:46.848940  4602 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.862839  4602 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:46.863117  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.861977  4117 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.865649  4602 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.866241  4303 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:46.867601  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 411) took 1524 ms. Trace:
I20260709 13:01:46.868016  1630 rpcz_store.cc:276] 0709 13:01:45.343179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.966291 (+623112us) service_pool.cc:224] Handling call
0709 13:01:45.977033 (+ 10742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.414727 (+437694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.867369 (+452642us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:46.867543 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976079104}
I20260709 13:01:46.869735  1630 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:01:46.873769  1551 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:46.874868  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.876480  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:46.877729  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.879719  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65890.2us
I20260709 13:01:46.881229  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:46.882125  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2073) took 1448 ms (client timeout 1572 ms). Trace:
I20260709 13:01:46.882419  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:46.882682  1626 rpcz_store.cc:269] 0709 13:01:45.434216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.448314 (+ 14098us) service_pool.cc:224] Handling call
0709 13:01:46.881895 (+1433581us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b54002751d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.882024 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86991616}
I20260709 13:01:46.883986  4535 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.884476  4535 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.884848  4535 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:46.884958  2291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 595) took 1396 ms (client timeout 1607 ms). Trace:
W20260709 13:01:46.885408  2291 rpcz_store.cc:269] 0709 13:01:45.488272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.489892 (+  1620us) service_pool.cc:224] Handling call
0709 13:01:46.884750 (+1394858us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b54002c8dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.884878 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42998144}
I20260709 13:01:46.885797  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1366) took 1827 ms. Trace:
I20260709 13:01:46.886111  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1410) took 1190 ms. Trace:
I20260709 13:01:46.887168  4368 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.887578  4368 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.887935  4368 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.888365  4444 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.888600  4308 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:46.889333  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1374) took 1749 ms (client timeout 1702 ms). Trace:
I20260709 13:01:46.877674  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.879783  1632 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:46.889706  1700 rpcz_store.cc:269] 0709 13:01:45.139694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.421574 (+281880us) service_pool.cc:224] Handling call
0709 13:01:46.889157 (+1467583us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:46.889263 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398683520}
I20260709 13:01:46.891774  1632 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:46.888944  4544 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:46.886150  1709 rpcz_store.cc:276] 0709 13:01:45.058581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.074152 (+ 15571us) service_pool.cc:224] Handling call
0709 13:01:45.100003 (+ 25851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.938217 (+838214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.885558 (+947341us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:46.885711 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848825728}
W20260709 13:01:46.902894  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54904.5us
I20260709 13:01:46.905551  1552 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.904139  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:46.909292  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.909852  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:46.911837  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.886507  1554 rpcz_store.cc:276] 0709 13:01:45.695622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.734916 (+ 39294us) service_pool.cc:224] Handling call
0709 13:01:45.992787 (+257871us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:01:45.993043 (+   256us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:46.072999 (+ 79956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.073031 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:46.109350 (+ 36319us) log.cc:844] Serialized 52 byte log entry
0709 13:01:46.512426 (+403076us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:46.548789 (+ 36363us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.551571 (+  2782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.551661 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.885836 (+334175us) raft_consensus.cc:1685] finished
0709 13:01:46.885927 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.886054 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38764,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":271564,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":31000,"spinlock_wait_cycles":12166528,"thread_start_us":402630,"threads_started":1,"wal-append.queue_time_us":637086}
I20260709 13:01:46.920188  1560 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:46.925809  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.930099  1634 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:46.931421  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.934453  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1420) took 1143 ms. Trace:
I20260709 13:01:46.934866  1558 rpcz_store.cc:276] 0709 13:01:45.791399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.800529 (+  9130us) service_pool.cc:224] Handling call
0709 13:01:45.836327 (+ 35798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.928950 (+ 92623us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:46.551414 (+622464us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.926580 (+375166us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.934099 (+  7519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:46.934161 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:46.934210 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.934292 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.934401 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7301,"prepare.queue_time_us":169745,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":25421,"thread_start_us":909590,"threads_started":2}
I20260709 13:01:46.925936  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:46.937096  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.937228  1632 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:46.939630  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:46.941015  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:46.942888  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:46.944446  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 334405us
I20260709 13:01:46.921500  4545 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.888972  4444 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:46.949067  1556 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:47.008230  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71222.5us
I20260709 13:01:47.056262  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:46.981441  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:47.036140  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.059448  1632 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:47.012127  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:47.018101  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.979689  1558 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:47.025722  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:47.030498  1560 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:47.065441  1553 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:47.012451  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:46.946606  1554 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:47.015931  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 94865.3us
I20260709 13:01:47.067811  2297 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
W20260709 13:01:46.992300  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86510.8us
I20260709 13:01:46.926802  4539 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.943539  1629 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:47.068703  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:47.071323  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.072592  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47000.9us
I20260709 13:01:46.940995  4598 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:47.083003  4598 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:47.083495  4598 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.084172  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.085043  1554 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:47.085326  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.073318  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.075598  2297 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:46.939502  4459 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:46.987149  4423 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 13:01:47.086009  1560 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:46.935233  4252 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:47.076370  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 506430us
I20260709 13:01:47.070053  4263 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.077901  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.022476  4416 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.094707  4416 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.075026  4545 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.096194  4545 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.016314  4599 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.082218  4444 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.097715  4444 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:47.098124  4444 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.928369  4544 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.082587  1559 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:47.098307  4545 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:47.003726  4441 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.912s	user 0.012s	sys 0.001s
W20260709 13:01:47.004675  4601 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:47.099861  4544 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.100277  1551 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:47.090054  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.100838  4444 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.991904  4208 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:47.102509  3767 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:47.104492  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.105594  1702 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:01:47.107192  4394 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.001s
I20260709 13:01:47.107688  4394 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.108832  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2084) took 1303 ms. Trace:
I20260709 13:01:47.109231  1631 rpcz_store.cc:276] 0709 13:01:45.807024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.609084 (+802060us) service_pool.cc:224] Handling call
0709 13:01:46.622568 (+ 13484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.626481 (+  3913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.626513 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.627044 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.928413 (+301369us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:46.928653 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.928852 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.108594 (+179742us) raft_consensus.cc:1685] finished
0709 13:01:47.108617 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.108770 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":300382,"threads_started":1,"wal-append.queue_time_us":476782}
I20260709 13:01:47.070369  4539 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.110149  1626 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:47.110525  4539 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.113617  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.114414  1707 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:47.113159  4444 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.091751  4562 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.116073  4456 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.116593  2294 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:47.117756  4416 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.090580  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.092625  1556 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:47.119895  4459 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.121222  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.121974  4263 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:47.122730  1709 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:47.122730  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44904.9us
I20260709 13:01:47.024168  4437 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.124543  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2234) took 1023 ms. Trace:
I20260709 13:01:47.133291  1626 rpcz_store.cc:276] 0709 13:01:46.100984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.883968 (+782984us) service_pool.cc:224] Handling call
0709 13:01:47.109972 (+226004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.110637 (+   665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.124334 (+ 13697us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400140ad0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:47.124476 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518699264}
I20260709 13:01:47.124975  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:46.943836  4481 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.066805  4529 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:47.135386  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59935.5us
I20260709 13:01:46.989344  4395 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:47.131266  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.110055  4394 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.135874  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.136188  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.138437  1705 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:47.141834  1707 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:47.132071  4263 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.144021  4441 raft_consensus.cc:2764] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:47.144225  4263 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.145185  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.145725  4263 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.146488  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.147679  4303 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.148075  4303 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.148424  4303 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:47.149391  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56926.9us
I20260709 13:01:47.150178  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:47.151353  4431 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.151532  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.127523  4208 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:47.073951  4117 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.140192  4423 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:47.142249  4395 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:47.158208  4127 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:01:47.172852  1552 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:47.136317  4481 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:47.160054  4480 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.002s
I20260709 13:01:47.169508  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:47.175591  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71228.4us
I20260709 13:01:47.154956  4599 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:47.160171  4431 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.156665  1457 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 6 to 7, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:47.162691  4602 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.167347  4394 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:47.169847  4449 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:47.156208  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.187512  1709 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:47.178162  4481 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.180123  4599 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.191955  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.194648  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.192127  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:47.169103  4529 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:01:47.179715  4127 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.196488  4602 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:47.197289  4117 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.198030  4117 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.198566  4117 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:47.199057  4117 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:47.187244  4377 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
I20260709 13:01:47.200065  4377 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.201675  1557 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:47.198710  2288 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:47.202303  4377 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.198069  4127 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.204135  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.204753  1702 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:47.204573  4394 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.205653  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.206214  1702 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:47.206759  4394 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.207144  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.185554  4449 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:47.208143  4449 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.208899  1704 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:47.187258  4437 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.210001  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:47.210582  1704 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.211805  3767 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.212215  4494 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.212543  4208 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.213657  4368 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.214448  4494 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.214512  4431 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:47.212229  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.215479  4431 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:47.215855  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:47.215991  1634 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:47.213893  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.218776  4609 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:47.219238  4609 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:47.219398  4603 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:47.219822  4603 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:47.176218  4598 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.220301  4609 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.220722  4603 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.221627  4609 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.221951  4603 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.222061  4598 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.223717  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.223937  4252 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:47.224346  4252 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:47.224907  4252 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.225675  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.226305  4535 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.218060  4599 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.175956  4459 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:01:47.227474  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.228993  1705 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:47.143812   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:47.232448  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.234927  1709 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:47.181041  4480 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:47.229480  4175 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.001s
I20260709 13:01:47.248383  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:47.229019  4368 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.212241  3767 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:47.261126  1707 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:47.254848  4175 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.266507  1457 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 2 to 3, term changed from 6 to 7, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to <none>, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 444b494bb53842b59408a7230884398a (127.0.107.193), VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194), VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195), VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:47.268808  1457 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 3 to 8, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.233345  4604 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.273901  4480 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.277361  4252 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.270933  1457 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 8, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.266530  4175 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.284965  1457 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 6 to 7, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 7 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.280293  4449 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.250758  4377 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.281086  4395 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.250901  4598 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:47.289476  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 94917.5us
I20260709 13:01:47.302999  1708 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:47.305357  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77925.2us
I20260709 13:01:47.306468  1704 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:47.286895  1457 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:47.317577  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.322325  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.326262  1705 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:47.329658  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.253258  4437 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.253811  4444 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.333956  1457 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 4 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:47.291613  3767 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.347072  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:01:47.354547  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 215262us
W20260709 13:01:47.365038  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.374023  4431 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.337881  4395 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.337186  4598 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:47.377400  4431 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.354542  4377 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:47.379739  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 295625us
I20260709 13:01:47.355082  4535 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.370278  4604 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.388988  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.378870  4598 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:47.389672  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 316405us
I20260709 13:01:47.378958  4431 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.392879  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.386176  4377 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.358788  4303 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.397311  4444 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.426514  1701 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:47.427997  1628 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:47.397482  4377 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:47.430905  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45284.7us
I20260709 13:01:47.432384  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:47.429472  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2093) took 1454 ms. Trace:
I20260709 13:01:47.430994  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.402908  4140 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:47.401440  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.430289  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:01:47.435869  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 360549us
I20260709 13:01:47.440789  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.434540  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.446555  2291 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:47.448863  2297 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:47.448213  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 241976us
I20260709 13:01:47.451013  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.431111  4529 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:47.431180  4604 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:47.402976  4459 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:47.463162  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.460915  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 8, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:47.438722  4410 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.466166  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.466634  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2228) took 1431 ms. Trace:
I20260709 13:01:47.439504  4423 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.470667  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:47.471518  1552 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.475759  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.476078  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.464046  4535 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:01:47.480329  1554 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.464581  4140 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:47.471482  1628 rpcz_store.cc:276] 0709 13:01:46.040050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.874732 (+834682us) service_pool.cc:224] Handling call
0709 13:01:47.427712 (+552980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.433199 (+  5487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.466376 (+ 33177us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54003444d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:47.466557 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1270997376}
I20260709 13:01:47.484988  4140 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.488087  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 424) took 1553 ms. Trace:
I20260709 13:01:47.488564  1633 rpcz_store.cc:276] 0709 13:01:45.934541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.843653 (+909112us) service_pool.cc:224] Handling call
0709 13:01:46.844125 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.925772 (+ 81647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.925807 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.979814 (+ 54007us) log.cc:844] Serialized 26 byte log entry
0709 13:01:47.246662 (+266848us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:47.246721 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.246836 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.487840 (+241004us) raft_consensus.cc:1685] finished
0709 13:01:47.487862 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.488021 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10988,"raft.queue_time_us":28417,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10193,"thread_start_us":259601,"threads_started":1,"wal-append.queue_time_us":410991}
I20260709 13:01:47.429435  4539 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.490433  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.447858  4252 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.450840  4480 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.428926  4368 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:01:47.503400  1554 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:47.504347  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2226) took 1527 ms. Trace:
I20260709 13:01:47.505630  4440 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.504894  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.506115  4480 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:47.510376  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.434397  4492 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:01:47.513983  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44345.4us
I20260709 13:01:47.517727  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.514955  1525 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:01:47.518290  1556 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.518461  1525 leader_election.cc:403] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:47.470034  4416 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.518290  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.507864  1630 rpcz_store.cc:276] 0709 13:01:45.976481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.870633 (+894152us) service_pool.cc:224] Handling call
0709 13:01:46.870995 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.871309 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:47.323186 (+451877us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.502861 (+179675us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.504007 (+  1146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.504058 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:47.504109 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.504175 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.504285 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45106,"prepare.queue_time_us":534823,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":14577,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"thread_start_us":544427,"threads_started":2}
I20260709 13:01:47.519985  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.470151  4622 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.520862  4622 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:47.521226  4622 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.516395  4449 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.464399  4456 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:01:47.523939  4368 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:47.524209  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:47.517341  4480 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.524518  4619 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:47.525009  4619 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:47.525774  4619 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.527036  4619 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.509498  4535 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.492434  4410 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:47.465718  4604 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.530921  1675 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:47.531975  1552 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:47.532742  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.533342  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.534266  1555 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:47.535085  1675 leader_election.cc:403] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:01:47.539175  1554 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:47.534027  4410 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.469584  4598 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.566156  4423 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.531107  4494 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:47.567799  4494 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.542927  4252 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.543095  4303 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.531354  4322 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:47.532068  4441 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.582916  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.509913  4459 consensus_queue.cc:1243] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:47.467264  4529 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.488921  4117 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:47.557585  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 127398us
I20260709 13:01:47.601846  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.607474  1560 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
W20260709 13:01:47.559298  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44408.7us
I20260709 13:01:47.549818  4618 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.571859  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:47.580926  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.573356  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.551548  4539 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.611142  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:47.583674  1552 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.561546  4444 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.588222  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.593094  1559 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:47.595973  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 288) took 14603 ms (client timeout 15000 ms). Trace:
I20260709 13:01:47.600261  4627 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 13:01:47.603350  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70080.9us
I20260709 13:01:47.604079  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.605394  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.528853  4555 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:47.644647  4555 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:47.645583  4555 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.646981  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.648015  4395 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:47.648267  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.648764  1629 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.649088  4394 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:47.649573  4394 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.650434  4394 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.651697  4394 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:47.611145  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78473.5us
I20260709 13:01:47.654117  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.654393  4494 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.534548  4624 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:47.655735  4624 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.656195  4624 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.656450  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.657181  1558 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:47.657927  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.612113  4322 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.660293  4638 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:47.659979  4545 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:47.660732  4638 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.661140  4638 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.662264  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.604149  4459 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.663506  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.664316  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.623553  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:47.625838  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.572041  4603 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:47.585652  4456 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.630860  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.632567  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.673589  1628 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.633769  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.674368  1634 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:47.634609  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.636754  1457 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 4 to 9, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.638669  4604 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.644075  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:47.664089  4459 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.664903  1560 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.669893  1552 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:01:47.621881  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.664884  4610 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:47.671646  2291 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:47.676138  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 185798us
I20260709 13:01:47.612322  4618 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.675662  1457 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 7, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 7 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.617595  4368 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.696660  4368 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.697105  4368 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:47.678241  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47451.6us
I20260709 13:01:47.701174  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.701566  4621 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:47.702273  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.702497  4621 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:47.702834  4621 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:01:47.680138  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.703758  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.703836  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.704617  2288 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:47.677196  4623 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:47.705523  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.706111  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.706614  4535 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.706910  4562 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:47.707204  4459 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.707382  4562 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:47.707413  4449 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:47.708431  4562 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.709071  1551 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:47.709415  4459 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:47.709705  4562 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.709798  4449 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:47.710343  4449 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.710220  4377 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.710775  4449 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.711290  4437 raft_consensus.cc:2764] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:01:47.680225  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.693207  4208 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:47.712134  4208 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.712298  4618 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:47.712539  4208 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.712911  4494 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:47.713559  4494 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.682747  4603 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.714403  4494 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:47.715049  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.681031  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:47.678912  4631 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:47.601919  4602 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:01:47.717152  4529 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.689828  4117 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:47.703598  1598 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.719545  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:47.707019  4535 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.721673  4535 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.724967  4377 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.690812  4481 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.728642  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.729035  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.725801  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.687920  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.677909  4545 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:47.670609  4492 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:01:47.673738  4444 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:47.712977  4555 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:47.624756  1458 rpcz_store.cc:269] 0709 13:01:33.018583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.031848 (+ 13265us) service_pool.cc:224] Handling call
0709 13:01:34.438051 (+1406203us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5400002210. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8c00b2 00007f70d4839a89 00007f70d47d1da6 00007f70d47a3088 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc
0709 13:01:47.595699 (+13157648us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:38.293486 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:38.371090 (+ 77604us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:38.371138 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:01:38.371157 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:38.374717 (+  3560us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:38.374822 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:38.374849 (+    27us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:01:38.376493 (+  1644us) tablet.cc:701] Row locks acquired
0709 13:01:38.376541 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:01:38.376717 (+   176us) write_op.cc:270] Start()
0709 13:01:38.376873 (+   156us) write_op.cc:276] Timestamp: P: 1783602098376643 usec, L: 0
0709 13:01:38.376895 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:01:38.457262 (+ 80367us) log.cc:844] Serialized 8817 byte log entry
0709 13:01:39.232534 (+775272us) op_driver.cc:464] REPLICATION: finished
0709 13:01:39.528231 (+295697us) write_op.cc:301] APPLY: starting
0709 13:01:39.592747 (+ 64516us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:39.697651 (+104904us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:39.708084 (+ 10433us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:39.712455 (+  4371us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:39.799333 (+ 86878us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:01:39.805163 (+  5830us) write_op.cc:312] APPLY: finished
0709 13:01:39.981405 (+176242us) log.cc:844] Serialized 145 byte log entry
0709 13:01:39.981594 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:39.981772 (+   178us) write_op.cc:454] Released schema lock
0709 13:01:39.981844 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:39.982257 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:39.982287 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":314486,"reported_tablets":41,"spinlock_wait_cycles":47286912,"child_traces":[["op",{"apply.queue_time_us":295380,"apply.run_cpu_time_us":6931,"apply.run_wall_time_us":453999,"num_ops":21,"prepare.queue_time_us":184375,"prepare.run_cpu_time_us":19270,"prepare.run_wall_time_us":427588,"raft.queue_time_us":186142,"raft.run_cpu_time_us":110177,"raft.run_wall_time_us":243474,"replication_time_us":855489,"thread_start_us":571372,"threads_started":4,"wal-append.queue_time_us":305096,"wal-append.run_cpu_time_us":54928,"wal-append.run_wall_time_us":2222709}]]}
I20260709 13:01:47.730122  1552 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:47.710147  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.734401  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.734614  1628 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:47.737715  1556 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:47.766004  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:47.770823  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.771752  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:47.772228  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43712.2us
I20260709 13:01:47.774564  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.775849  2297 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:47.774271  2290 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:47.777436  2293 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:47.779013  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.794946  1629 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:47.434983  1635 rpcz_store.cc:276] 0709 13:01:45.974912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.849630 (+874718us) service_pool.cc:224] Handling call
0709 13:01:46.858883 (+  9253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.859122 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:47.186535 (+327413us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.427400 (+240865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.427455 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:47.427504 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.427570 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.429409 (+  1839us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":508669,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":1412,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3729,"thread_start_us":536050,"threads_started":2}
I20260709 13:01:47.797127  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.798184  1556 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:47.739749  4602 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.734303  4545 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.803956  4603 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.805508  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.734804  4610 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.739269  4444 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.774212  1457 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.764359  4252 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:01:47.766144  4416 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:47.819234  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:47.827073  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 240577us
I20260709 13:01:47.833214  1551 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:47.822386  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.819425  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.836393  1552 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:47.709084  4480 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:47.838095  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 103772us
W20260709 13:01:47.839087  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 155167us
I20260709 13:01:47.839088  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.839972  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.808375  4494 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:47.844511  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:47.832340  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.835485  1457 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 5 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:47.828004  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158844us
I20260709 13:01:47.704617  4622 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.816170  4545 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:47.849953  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71034.4us
I20260709 13:01:47.855016  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:47.852021  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.814210  4322 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.815537  4619 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:47.856775  4619 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:47.857867  4322 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:47.857705  4619 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.858282  4322 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.858649  4322 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.820645  4598 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.863401  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:47.869594  1552 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:47.865450  4252 raft_consensus.cc:2764] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:47.866735  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:47.847318  4599 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.003s
I20260709 13:01:47.833000  4444 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:47.793910  4410 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.870326  4416 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 13:01:47.917510  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77601us
I20260709 13:01:47.705550  4621 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.876104  4633 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:47.922526  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:47.877803  4639 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:01:47.887560  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:47.862762  4372 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.888201  4610 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.879187  4638 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:47.894716  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59196us
I20260709 13:01:47.896075  1560 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:47.901731  4456 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
W20260709 13:01:47.871291  3767 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.001s
W20260709 13:01:47.890417  4612 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:47.916070  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:47.890054  4494 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:47.952872  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:47.935629  4635 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:01:47.934794  4634 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:01:47.935581  4368 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.937201  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:47.938529  4603 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:47.864650  4208 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:47.942301  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:47.946954  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:47.953615  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:47.914152  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 80098.7us
W20260709 13:01:47.949625  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 284686us
I20260709 13:01:47.925402  4372 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:47.974489  4372 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.931950  4377 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:47.951577  4431 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
I20260709 13:01:47.976137  4431 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.978385  4431 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.953195  4645 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:47.948887  3767 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.981719  3767 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:47.982117  1630 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:47.900735  4599 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:47.949421  4642 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.959824  4598 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:47.990273  2295 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:47.985435  1552 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:47.991631  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54519.6us
I20260709 13:01:47.993897  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:47.975441  4368 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.006278  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:47.989464  1632 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:47.965288  4621 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:47.997576  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:47.975493  4208 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:47.999642  2292 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:47.998852  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:47.966176  4646 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.011819  1553 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:47.995723  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 69788.4us
W20260709 13:01:48.023480  1524 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.691s (SENT)
I20260709 13:01:48.024302  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:48.007503  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83322.7us
I20260709 13:01:47.994561  4492 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.998684  4609 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:48.031546  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.027541  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:48.033004  1629 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:48.033509  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:48.009444  4481 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:48.030375  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 191369us
I20260709 13:01:48.037883  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.034790  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.039597  1557 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:01:48.009225  4598 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:47.995553  4456 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:48.045442  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.046247  1557 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.047179  4602 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.042330  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.048050  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.048902  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.049788  1633 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:48.035795  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.050817  4442 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:48.038697  4452 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.051234  4442 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:48.031538  4650 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:48.029587  4603 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:01:48.011322  4494 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.090687  2291 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:48.075971  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.091259  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.068154  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.077165  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:48.102830  2294 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:48.103859  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.054565  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2081) took 1681 ms. Trace:
I20260709 13:01:48.104933  1555 rpcz_store.cc:276] 0709 13:01:46.372859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.901137 (+528278us) service_pool.cc:224] Handling call
0709 13:01:48.054366 (+1153229us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54003faf54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:48.054486 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393857920}
W20260709 13:01:48.101783  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63903.9us
I20260709 13:01:48.076030  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.107298  1702 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:01:48.100936  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 66215.4us
I20260709 13:01:48.083941  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.108886  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.032618  4609 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.114733  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:48.145750  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.103645  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:48.149194  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.151185  1701 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9.
I20260709 13:01:48.151484  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.154111  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.154469  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.155679  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.039042  4431 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.160368  1627 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:01:48.161005  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46497.8us
I20260709 13:01:48.162014  1705 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:48.162379  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.163865  1708 raft_consensus.cc:2368] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6.
I20260709 13:01:48.164568  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.166842  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.068347  4410 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.168859  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.169574  2296 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:48.169490  1633 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.172261  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.048420  4647 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:48.052596  4140 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:48.079384  4619 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:48.054971  4005 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:48.063937  4480 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.061235  4599 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.099426  4456 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.150604  4609 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:48.144754  4441 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:48.061838  4638 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.043669  4208 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 13:01:48.159677  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:48.184371  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.204005  1705 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:48.156857  4539 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:48.190418  1704 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:48.036571  4492 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.208985  1628 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:01:48.206700  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.212028  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.212405  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.153909  4655 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:01:48.171636  4555 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:01:48.216784  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54899.7us
I20260709 13:01:48.083374  4618 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.161151  4368 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:01:48.184916  4005 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:48.227690  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.197122  4456 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.227416  1700 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:48.072856  4598 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.238580  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.180303  4322 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.192286  4645 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:48.253979  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.256105  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:48.225576  4562 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 13:01:48.264339  1628 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:48.258438  1633 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:48.206851  4431 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:48.261137  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:48.269490  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 271966us
I20260709 13:01:48.261217  1635 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:48.269663  2260 leader_election.cc:400] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9.
I20260709 13:01:48.268278  1555 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:48.270927  1626 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:48.271158  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74987.4us
I20260709 13:01:48.263044  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.288609  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.289985  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.295102  4658 raft_consensus.cc:2764] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:48.302294  1705 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:48.304579  4653 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:48.307147  2260 leader_election.cc:400] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6.
I20260709 13:01:48.307608  2260 leader_election.cc:403] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.202883  4442 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:48.289927  4449 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 13:01:48.328019  4449 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.205638  4656 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:01:48.332758  4656 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:48.333046  1631 raft_consensus.cc:2368] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7.
I20260709 13:01:48.334846  4372 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.337011  4618 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.298291  4555 consensus_queue.cc:1243] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:48.337734  3767 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:48.341122  4535 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.000s
I20260709 13:01:48.341567  4535 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.342876  1634 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:48.345973  4492 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.343719  4372 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.352294  4645 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.313803  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.314608  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.315464  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.285089  4263 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.324589  4442 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.287693  4431 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:48.365271  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58336.6us
I20260709 13:01:48.366253  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:48.358938  2294 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:48.370937  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:48.330659  4449 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.376314  1628 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:48.379707  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:48.368649  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 209075us
I20260709 13:01:48.281213  4610 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:01:48.333101  4656 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:48.311345  4465 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.240s
I20260709 13:01:48.369629  4127 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6.
I20260709 13:01:48.377154  4618 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:48.425953  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 246612us
W20260709 13:01:48.426240  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 128943us
I20260709 13:01:48.427377  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.385439  4621 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.412181  4423 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:48.434777  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:48.436748  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50017us
I20260709 13:01:48.421908  4610 consensus_queue.cc:1243] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:48.444607  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.361359  4444 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.316056  4005 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.448483  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.436367  1555 raft_consensus.cc:2955] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:01:48.452520  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:48.460815  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.453722  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.439097  4140 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.439836  4562 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:48.448207  4624 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.792s	user 0.009s	sys 0.002s
W20260709 13:01:48.448796  4545 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.001s
I20260709 13:01:48.466914  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:48.502084  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74819.3us
I20260709 13:01:48.505041  4544 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:48.508599  4662 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:48.510380  1627 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:48.518276  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 716) took 1045 ms. Trace:
I20260709 13:01:48.522022  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.451418  4640 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:01:48.565191  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.564460  2297 rpcz_store.cc:276] 0709 13:01:47.473335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.473542 (+   207us) service_pool.cc:224] Handling call
0709 13:01:47.508279 (+ 34737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.796218 (+287939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.518048 (+721830us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54004d49d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:48.518214 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614888832}
I20260709 13:01:48.566104  1628 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.567032  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.500746  4545 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.527582  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:48.543048  1556 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:48.468709  4431 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.545111  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:48.545336  1702 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:48.522099  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.550681  4444 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.551986  4535 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.470700  4667 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:48.586848  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.553050  4649 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:48.555403  4456 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.546865  4618 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:48.462834  4610 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.449518  4652 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:48.472440  4624 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.566660  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.568434  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:48.500916  4005 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:48.597028  4456 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.580533  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:48.599510  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.599843  4624 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.580364  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.470082  4140 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.576824  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:48.583983  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56512us
I20260709 13:01:48.584543  4602 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.446820  4665 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:01:48.462970  4647 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:48.587664  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:48.621930  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.622483  1703 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.591941  1709 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:48.592553  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.565609  4544 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:48.594893  1628 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:48.600020  1707 raft_consensus.cc:2376] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7. Re-sending same reply.
I20260709 13:01:48.602378  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.574225  4322 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.607827  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.600073  4494 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:48.610383  2288 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:48.601060  4618 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.617499  4545 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.618036  1559 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.618593  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.638082  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.620194  4140 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.625608  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.630614  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:48.647871  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.648550  1557 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:48.635658  1552 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:48.654125  1552 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:01:48.636624  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 70034.7us
I20260709 13:01:48.655362  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:48.656754  4642 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.005s
I20260709 13:01:48.657191  4642 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.658088  4416 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:48.658670  4416 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:48.660578  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.660749  4423 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.657199  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.638651  1707 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.663106  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.664230  4459 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:48.667500  4545 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.658128  1632 raft_consensus.cc:2410] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:48.637537  1701 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:48.461043  4656 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.600826  4667 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.623375  4624 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.639477  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:48.642112  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:48.669765  4624 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.642670  4494 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.671123  4667 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.638084  4623 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.644438  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.672896  1559 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.673908  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:48.674443  4544 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.675671  4544 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:48.676017  4263 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:48.676131  4618 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:48.676496  4263 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:48.645968  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.661576  2291 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:48.664268  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:48.640784  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:48.679273  1706 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:48.679371  2296 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:48.680785  2295 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:48.677389  4263 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.682278  1704 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.682821  1707 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:48.593786  4535 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
W20260709 13:01:48.783349  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 119141us
I20260709 13:01:48.802924  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:48.804071  1555 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.713076  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:48.821568  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:48.708395  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.768245  4609 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260709 13:01:48.677389  4494 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.684250  4444 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.701763  4624 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.659562  4416 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:48.853195  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52833.3us
I20260709 13:01:48.887688  1704 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 4.
I20260709 13:01:48.875046  2289 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:48.878721  1557 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:48.639245  4647 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:48.872715  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:48.673928  4652 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.923220  4640 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:48.923245  1552 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.921645  2289 pending_rounds.cc:85] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 7
I20260709 13:01:48.686136  4140 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.841562  4562 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.924455  4140 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.924795  2289 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:48.925328  2289 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.841670  4665 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.927309  4602 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:48.927443  4667 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:48.920168  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122593us
I20260709 13:01:48.928517  4667 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:48.928890  4667 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:48.868752  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.929960  1631 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.930132  4667 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.847756  4539 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:48.932466  4667 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:48.932703  4539 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:48.857033  4609 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.936558  1554 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:48.937798  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:48.939646  4535 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:48.872468  4649 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:48.942272  1555 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:48.711301  1707 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:48.944957  1707 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:01:48.661568  4423 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.944751  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:48.950567  1552 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:48.955916  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:48.957839  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.918131  4529 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.965328  4263 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:48.965704  4539 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:48.965477  4529 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:48.979336  1631 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:48.981748  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.983902  1631 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:48.986299  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1435) took 1351 ms (client timeout 1647 ms). Trace:
I20260709 13:01:48.986493  1560 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:01:48.986747  1558 rpcz_store.cc:269] 0709 13:01:47.634517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.657674 (+ 23157us) service_pool.cc:224] Handling call
0709 13:01:48.986120 (+1328446us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002bbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:48.986242 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446477568}
I20260709 13:01:48.993741  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.994810  1631 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:48.995816  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:49.008555  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:49.009873  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:48.981369  4642 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:49.015029  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.987756  4416 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:49.022017  4642 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:49.027134  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.047605  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.048815  1705 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:49.028537  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:49.032948  4609 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:49.043231  1631 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:49.051633  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:49.056139  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:49.057124  1631 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:49.058856  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90126.1us
I20260709 13:01:49.060767  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:49.065781  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:48.941769  4649 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.070016  4649 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:48.926744  4665 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.062310  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:49.086848  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:48.923067  4647 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:49.096769  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 144105us
W20260709 13:01:49.097340  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.098536  1631 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:49.101223  2290 raft_consensus.cc:2410] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:49.102380  1706 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:49.104007  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.105108  1705 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:49.104869  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.108135  1703 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:49.109246  1631 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:49.109133  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.111660  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:49.112839  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:49.114154  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64102.5us
I20260709 13:01:49.096602  4609 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:49.122911  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.125856  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:49.129434  1701 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:49.108992  4459 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:49.132788  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.134536  1705 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:49.134938  1706 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:49.136497  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.117862  4431 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.121470  4672 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:49.140928  1706 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:49.125778  4665 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.128706  4535 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:49.128037  4539 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:49.108086  4692 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:49.152486  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.153800  1706 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:49.154794  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.154973  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:49.155567  1706 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:49.108711  4642 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.157833  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:49.158136  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 101818us
I20260709 13:01:49.156358  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.159967  1706 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:49.154949  4539 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.169256  1706 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:49.172112  4696 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:49.169390  4539 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.175669  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.176694  1705 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:49.177101  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111851us
I20260709 13:01:49.173621  4609 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:49.180691  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:49.179682  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70095us
I20260709 13:01:49.142657  4431 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.190801  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.192659  1705 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:49.184514  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:49.193776  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:49.194648  1551 raft_consensus.cc:2376] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10. Re-sending same reply.
I20260709 13:01:49.200341  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:49.190815  4696 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:49.202453  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.215230  1705 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:49.203133  4649 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:49.217387  4649 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:49.217796  4622 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:49.218573  4649 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.201078  4665 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:49.205081  4431 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.234341  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.235090  1705 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.202273  4624 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:49.202279  4696 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:49.221381  4692 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:49.246006  4667 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.245532  4696 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:49.260744  4667 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.264675  4667 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:49.265291  4648 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:49.265738  4648 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:49.266551  4648 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.269162  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:49.269872  2297 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:49.271044  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:49.271823  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:49.272712  1555 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:49.271845  4699 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:49.274474  4699 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.274889  4699 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:49.276335  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2268) took 1185 ms. Trace:
I20260709 13:01:49.276770  1629 rpcz_store.cc:276] 0709 13:01:48.090430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.103596 (+ 13166us) service_pool.cc:224] Handling call
0709 13:01:49.276067 (+1172471us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:49.276266 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201124736}
I20260709 13:01:49.278057  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:49.278800  1629 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:49.271808  4696 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:49.281468  2288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 609) took 1575 ms (client timeout 2001 ms). Trace:
I20260709 13:01:49.248879  4624 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:49.281908  2288 rpcz_store.cc:269] 0709 13:01:47.705745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.705875 (+   130us) service_pool.cc:224] Handling call
0709 13:01:49.281274 (+1575399us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:49.281408 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055421184}
I20260709 13:01:49.279615  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:49.284324  1629 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:49.294175  1705 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:01:49.295553  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:49.267668  4648 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:49.299103  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:49.256093  4692 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.302213  4696 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.300014  1629 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:49.303650  4692 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.304909  4431 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.305097  4368 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:49.305563  4368 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:49.306643  4368 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.305464  4431 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:49.319609  4431 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:49.319737  4368 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:49.322166  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:49.328851  4658 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:49.330931  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:49.329298  4658 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:49.361631  4699 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:49.364390  4699 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.423467  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:50.546325  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 427) took 1255 ms. Trace:
I20260709 13:01:50.546664  1705 rpcz_store.cc:276] 0709 13:01:49.290761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.296972 (+  6211us) service_pool.cc:224] Handling call
0709 13:01:50.546266 (+1249294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.547809  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.548566  1705 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:50.549000  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 429) took 1233 ms. Trace:
I20260709 13:01:50.549322  1705 rpcz_store.cc:276] 0709 13:01:49.316197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.547493 (+1231296us) service_pool.cc:224] Handling call
0709 13:01:50.548945 (+  1452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.550654  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.551254  1705 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:50.551730  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 423) took 1587 ms. Trace:
I20260709 13:01:50.552034  1705 rpcz_store.cc:276] 0709 13:01:48.964799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.550378 (+1585579us) service_pool.cc:224] Handling call
0709 13:01:50.551677 (+  1299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.552935  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:50.553938  1705 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:50.560005  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.560797  1708 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:50.561283  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2174) took 1355 ms. Trace:
I20260709 13:01:50.561571  1708 rpcz_store.cc:276] 0709 13:01:49.205823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.559681 (+1353858us) service_pool.cc:224] Handling call
0709 13:01:50.561223 (+  1542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.562557  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.562777  1705 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:50.563235  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:50.563354  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 428) took 1247 ms. Trace:
I20260709 13:01:50.563791  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2171) took 1537 ms. Trace:
I20260709 13:01:50.563738  1705 rpcz_store.cc:276] 0709 13:01:49.315609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.552644 (+1237035us) service_pool.cc:224] Handling call
0709 13:01:50.563295 (+ 10651us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.564311  1708 rpcz_store.cc:276] 0709 13:01:49.026147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.562309 (+1536162us) service_pool.cc:224] Handling call
0709 13:01:50.563717 (+  1408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.566061  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:50.566277  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:50.566817  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2169) took 1541 ms. Trace:
I20260709 13:01:50.567217  1708 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:50.567157  1705 rpcz_store.cc:276] 0709 13:01:49.025484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.565806 (+1540322us) service_pool.cc:224] Handling call
0709 13:01:50.566730 (+   924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.567920  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 421) took 1604 ms. Trace:
I20260709 13:01:50.568236  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.568372  1708 rpcz_store.cc:276] 0709 13:01:48.963597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.566013 (+1602416us) service_pool.cc:224] Handling call
0709 13:01:50.567861 (+  1848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.568930  1705 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:50.569475  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 424) took 1604 ms. Trace:
I20260709 13:01:50.569777  1705 rpcz_store.cc:276] 0709 13:01:48.965144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.567994 (+1602850us) service_pool.cc:224] Handling call
0709 13:01:50.569424 (+  1430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.570063  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44858 (request call id 89) took 1963 ms. Trace:
I20260709 13:01:50.570504  1708 rpcz_store.cc:276] 0709 13:01:48.627835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.569053 (+1941218us) service_pool.cc:224] Handling call
0709 13:01:50.569998 (+   945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.570863  1705 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:50.571509  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 422) took 1607 ms. Trace:
I20260709 13:01:50.571849  1708 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:50.571836  1705 rpcz_store.cc:276] 0709 13:01:48.963866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.570415 (+1606549us) service_pool.cc:224] Handling call
0709 13:01:50.570685 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.571286 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.571462 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.572592  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 425) took 1599 ms. Trace:
I20260709 13:01:50.572922  1708 rpcz_store.cc:276] 0709 13:01:48.973616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.571278 (+1597662us) service_pool.cc:224] Handling call
0709 13:01:50.571495 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.572335 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.572537 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.573391  1705 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:50.574031  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2152) took 1599 ms. Trace:
I20260709 13:01:50.574520  1708 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:50.574332  1705 rpcz_store.cc:276] 0709 13:01:48.974948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.572824 (+1597876us) service_pool.cc:224] Handling call
0709 13:01:50.573069 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.573801 (+   732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.573997 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.575484  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2155) took 1594 ms. Trace:
I20260709 13:01:50.575871  1705 pending_rounds.cc:85] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 3
I20260709 13:01:50.576303  1705 pending_rounds.cc:107] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:01:50.575942  1708 rpcz_store.cc:276] 0709 13:01:48.980936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.573760 (+1592824us) service_pool.cc:224] Handling call
0709 13:01:50.574071 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.575223 (+  1152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.575434 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.576792  1705 raft_consensus.cc:2889] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:50.577425  1708 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:50.577859  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 395) took 2842 ms. Trace:
I20260709 13:01:50.577934  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2165) took 1553 ms. Trace:
I20260709 13:01:50.578481  1560 rpcz_store.cc:276] 0709 13:01:47.735535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.965571 (+230036us) service_pool.cc:224] Handling call
0709 13:01:48.986242 (+1020671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.992624 (+  6382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.577694 (+1585070us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54002bbbd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:50.577807 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282726144}
I20260709 13:01:50.578590  1708 rpcz_store.cc:276] 0709 13:01:49.024184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.576978 (+1552794us) service_pool.cc:224] Handling call
0709 13:01:50.577266 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.577753 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.577899 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.580896  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2159) took 1582 ms. Trace:
I20260709 13:01:50.581282  1705 rpcz_store.cc:276] 0709 13:01:48.998188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.575403 (+1577215us) service_pool.cc:224] Handling call
0709 13:01:50.575670 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.577397 (+  1727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.577427 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.577829 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.578786 (+   957us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:50.578818 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.578882 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.580677 (+  1795us) raft_consensus.cc:1685] finished
0709 13:01:50.580698 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.580819 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":622,"threads_started":1,"wal-append.queue_time_us":990}
I20260709 13:01:50.582775  1705 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:50.583295  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2168) took 1558 ms. Trace:
I20260709 13:01:50.583570  1705 rpcz_store.cc:276] 0709 13:01:49.025163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.582358 (+1557195us) service_pool.cc:224] Handling call
0709 13:01:50.582623 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.583095 (+   472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.583252 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.584769  1705 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:50.585232  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2170) took 1559 ms. Trace:
I20260709 13:01:50.585512  1705 rpcz_store.cc:276] 0709 13:01:49.025836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.584226 (+1558390us) service_pool.cc:224] Handling call
0709 13:01:50.584480 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.585025 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.585189 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.586627  1705 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:50.587148  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2175) took 1369 ms. Trace:
I20260709 13:01:50.587435  1705 rpcz_store.cc:276] 0709 13:01:49.217414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.586104 (+1368690us) service_pool.cc:224] Handling call
0709 13:01:50.586364 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.586933 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.587093 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:50.624490  1551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.370s	user 0.011s	sys 0.021s
W20260709 13:01:50.625180  1551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.012s	sys 0.022s
I20260709 13:01:50.627044  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1463) took 1413 ms. Trace:
I20260709 13:01:50.627441  1551 rpcz_store.cc:276] 0709 13:01:49.213336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.213494 (+   158us) service_pool.cc:224] Handling call
0709 13:01:49.234468 (+ 20974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.234652 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:50.625635 (+1390983us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.626664 (+  1029us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.626719 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:50.626770 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.626843 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.626972 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131004,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":1380226,"threads_started":2,"wal-append.queue_time_us":992}
W20260709 13:01:50.639986  4685 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.581s	user 0.007s	sys 0.008s
I20260709 13:01:50.687225  4303 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:50.687824  4303 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:50.688952  4670 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.688683  4303 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.689406  4670 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.689993  4303 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.690975  4670 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.691382  4670 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.691658  4670 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.694109  4670 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.696777  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:50.698755  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:50.700232  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:50.700744  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.701819  4658 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.702986  4658 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:50.705265  4670 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:50.705837  4670 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.706712  4670 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.707700  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:50.702956  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:50.714656  1706 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
W20260709 13:01:50.715224  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2163) took 1691 ms (client timeout 1992 ms). Trace:
I20260709 13:01:50.715351  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:50.715716  1706 rpcz_store.cc:269] 0709 13:01:49.023505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.170094 (+146589us) service_pool.cc:224] Handling call
0709 13:01:50.707484 (+1537390us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:50.715169 (+  7685us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12881664}
I20260709 13:01:50.713907  4670 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:50.716609  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:50.710228  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1466) took 1404 ms. Trace:
I20260709 13:01:50.717604  4624 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.725605  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:50.724725  4535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.489s	user 0.021s	sys 0.029s
W20260709 13:01:50.728016  4535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.492s	user 0.022s	sys 0.029s
I20260709 13:01:50.728637  4535 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:50.728863  4670 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.729187  4535 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.731237  4535 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.733315  1708 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:50.735209  4658 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:50.735528  4670 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.735668  4658 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.736667  4442 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:50.737136  4442 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.737721  4535 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:50.738160  4535 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.738021  4442 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.739051  4535 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.729812  1706 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:50.745273  1706 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:50.725741  1555 rpcz_store.cc:276] 0709 13:01:49.305746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.709103 (+1403357us) service_pool.cc:224] Handling call
0709 13:01:50.709550 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.709785 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.709823 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.709889 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:50.709946 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.710019 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.710159 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:50.729390  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1413) took 2281 ms (client timeout 1564 ms). Trace:
W20260709 13:01:50.746784  1704 rpcz_store.cc:269] 0709 13:01:48.448193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.521865 (+ 73672us) service_pool.cc:224] Handling call
0709 13:01:50.729169 (+2207304us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:50.729302 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248184064}
I20260709 13:01:50.740474  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2167) took 1715 ms. Trace:
I20260709 13:01:50.747469  4535 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:50.747736  1708 rpcz_store.cc:276] 0709 13:01:49.024839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.579820 (+1554981us) service_pool.cc:224] Handling call
0709 13:01:50.733125 (+153305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.733764 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.735672 (+  1908us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001b6a50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:50.740391 (+  4719us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351798016}
I20260709 13:01:50.739522  4442 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:50.749222  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.749788  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1566896209999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
I20260709 13:01:50.751336  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:50.752710  4303 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:50.752970  4715 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:50.753432  4303 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.754294  4715 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:50.755393  4303 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.756274  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:50.758154  4303 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.758678  4303 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.759724  4602 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:50.760243  4602 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:50.760875  4303 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.761380  4303 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.761153  4602 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.761796  4303 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.763703  4303 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.765403  4303 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:50.766121  4688 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.678s	user 0.013s	sys 0.000s
I20260709 13:01:50.766050  4303 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.769587  4718 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:50.736702  4658 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.775740  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 436) took 2249 ms. Trace:
I20260709 13:01:50.776532  1629 rpcz_store.cc:276] 0709 13:01:48.525969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.285375 (+759406us) service_pool.cc:224] Handling call
0709 13:01:49.285760 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.303201 (+ 17441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.775479 (+1472278us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54001432d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:50.775647 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32050432}
W20260709 13:01:50.777398  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.778213  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 438) took 2251 ms. Trace:
I20260709 13:01:50.778538  1629 rpcz_store.cc:276] 0709 13:01:48.526514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.777483 (+2250969us) service_pool.cc:224] Handling call
0709 13:01:50.777762 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.777919 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.777942 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.777977 (+    35us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:50.778009 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.778058 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.778140 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.736297  4670 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.781394  1629 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:50.771448  4718 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.782120  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 440) took 2108 ms. Trace:
I20260709 13:01:50.782917  1629 rpcz_store.cc:276] 0709 13:01:48.673244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.779363 (+2106119us) service_pool.cc:224] Handling call
0709 13:01:50.781229 (+  1866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.781825 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.782048 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.783842  4718 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.784382  4718 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:50.784680  4140 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.784878  4718 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.785441  1629 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:50.785998  4395 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.786561  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2114) took 2105 ms. Trace:
I20260709 13:01:50.787011  1629 rpcz_store.cc:276] 0709 13:01:48.681115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.784396 (+2103281us) service_pool.cc:224] Handling call
0709 13:01:50.784660 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.786332 (+  1672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.786508 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.770756  4303 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.788623  4672 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:50.788998  4672 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:50.790030  4718 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.789866  4672 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.790993  4672 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:50.785832  4690 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:50.792632  4690 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.793509  4529 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.794025  4529 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.794745  4690 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:50.785647  4624 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.795696  4624 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.794478  4529 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.797761  4624 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.798532  4529 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.785214  4140 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:50.799554  4140 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.786623  4395 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.802894  4395 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:50.805146  4140 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:50.818993  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1438) took 3183 ms. Trace:
I20260709 13:01:50.819494  1556 rpcz_store.cc:276] 0709 13:01:47.635249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.814999 (+179750us) service_pool.cc:224] Handling call
0709 13:01:47.834465 (+ 19466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.573146 (+738681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.818715 (+2245569us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400309dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:50.818923 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1573270912}
W20260709 13:01:50.830133  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2262) took 2784 ms (client timeout 2855 ms). Trace:
W20260709 13:01:50.830722  1632 rpcz_store.cc:269] 0709 13:01:48.075472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.075684 (+   212us) service_pool.cc:224] Handling call
0709 13:01:50.824318 (+2748634us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400343fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:50.828832 (+  4514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279223552}
I20260709 13:01:50.832692  1632 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:50.833461  4368 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.512s	user 0.011s	sys 0.000s
I20260709 13:01:50.833413  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 442) took 1784 ms. Trace:
I20260709 13:01:50.834611  1629 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:50.835111  1629 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:01:50.835366  1629 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:50.835628  1629 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:50.829419  4604 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:50.834412  1632 rpcz_store.cc:276] 0709 13:01:49.049429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.831866 (+1782437us) service_pool.cc:224] Handling call
0709 13:01:50.832255 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.833126 (+   871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.833348 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.836817  4604 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:50.838012  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2119) took 1532 ms. Trace:
I20260709 13:01:50.838421  1632 rpcz_store.cc:276] 0709 13:01:49.306076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.837150 (+1531074us) service_pool.cc:224] Handling call
0709 13:01:50.837512 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.837677 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.837699 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.837739 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:50.837791 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.837845 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.837955 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:50.838968  4604 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:50.848579  4140 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.849195  4140 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:50.851274  4140 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:50.860229  4648 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.532s	user 0.005s	sys 0.014s
W20260709 13:01:50.907112  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.574s (SENT) [suppressed 28 similar messages]
I20260709 13:01:51.106187  4127 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.107120  4127 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.110070  4127 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:51.170639  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.226193  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.263160  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.341497  1674 connection.cc:256] RPC call timeout handler was delayed by -1.6818630000000212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:01:51.341980  1674 connection.cc:256] RPC call timeout handler was delayed by -1.6197630000000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 13:01:51.342276  1674 connection.cc:256] RPC call timeout handler was delayed by -1.3802595579999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:01:51.342573  1674 connection.cc:256] RPC call timeout handler was delayed by -1.3388595579999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 13:01:51.342852  1674 connection.cc:256] RPC call timeout handler was delayed by -1.3316595579999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
W20260709 13:01:51.356841  4678 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 2.088s	user 0.011s	sys 0.011s
W20260709 13:01:51.387459  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 234176us
W20260709 13:01:51.388284  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.398473  1674 connection.cc:256] RPC call timeout handler was delayed by -1.0209463580000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:01:51.387107  4539 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:51.389209  4456 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.405570  4431 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.405438  4539 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.406107  4431 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.405952  4456 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.408507  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.409523  1705 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:51.408951  4539 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.411688  4539 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.412012  4456 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.412252  4539 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.404834  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.419027  1708 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:51.412942  4456 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.416433  4544 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.420187  4544 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.422881  4544 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.423446  4544 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.424021  4544 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.424379  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.426107  4544 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.426671  4456 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.431283  4494 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.431855  4494 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.432925  4456 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.427621  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.429380  4539 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.429576  1708 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:51.433758  4494 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.430197  4609 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.433873  1705 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:51.430699  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.434906  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.435711  1708 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:51.436669  4609 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:51.438812  4609 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.439922  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.435148  4494 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:51.404574  4692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.013s	sys 0.005s
I20260709 13:01:51.452927  4494 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.454134  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:51.454958  2292 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:51.455142  4692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.014s	sys 0.005s
I20260709 13:01:51.449255  4691 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.468393  4691 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.471194  4691 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:51.452414  4697 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:51.472397  4697 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.490736  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2284) took 2463 ms. Trace:
I20260709 13:01:51.491233  1629 rpcz_store.cc:276] 0709 13:01:49.027065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.787811 (+1760746us) service_pool.cc:224] Handling call
0709 13:01:50.788083 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.836227 (+ 48144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.490464 (+654237us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b54002756d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:51.490668 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105881216}
I20260709 13:01:51.492018  2292 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:51.493309  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:51.505088  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 405) took 3402 ms (client timeout 1676 ms). Trace:
W20260709 13:01:51.505630  1700 rpcz_store.cc:269] 0709 13:01:48.102965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.103081 (+   116us) service_pool.cc:224] Handling call
0709 13:01:51.504846 (+3401765us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540010d6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.505007 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165865344}
I20260709 13:01:51.509965  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.511094  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.512334  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.513213  1628 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:01:51.511642  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.515506  1633 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:51.516350  4539 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.512614  4665 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:51.518108  4726 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.519011  4726 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:51.510876  1629 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:51.521800  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.522717  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.523258  1628 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:51.516558  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.520013  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.518390  4665 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:51.527395  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.520942  4726 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:51.531795  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 414) took 3365 ms. Trace:
I20260709 13:01:51.532521  4726 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.533711  4726 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:51.535965  4431 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.536072  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.536963  1630 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:51.532274  1709 rpcz_store.cc:276] 0709 13:01:48.166140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.166256 (+   116us) service_pool.cc:224] Handling call
0709 13:01:48.181085 (+ 14829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.623469 (+442384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.531511 (+2908042us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b5400042754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:51.531715 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944220032}
W20260709 13:01:51.539566  1673 connection.cc:256] RPC call timeout handler was delayed by -2.3181570670000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 13:01:51.548815  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.551744  4697 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:51.554880  1673 connection.cc:256] RPC call timeout handler was delayed by -1.2817142819999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 13:01:51.551594  1442 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 00f3fbc60b884078a3c219909b44f64c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:01:51.559365  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63319.1us
W20260709 13:01:51.567654  1674 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.145s (SENT)
I20260709 13:01:51.554505  4416 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.568266  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:51.568743  4416 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.570786  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:51.558939  4544 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.572371  4544 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.574388  4544 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.575989  4544 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.576171  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63135.4us
I20260709 13:01:51.554083  4562 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.577878  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.579944  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:51.554564  4456 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.583930  4697 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:01:51.598783  4431 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.599469  4431 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.588058  4539 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:51.576495  4544 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.562863  4494 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:51.603590  4544 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.604193  4544 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:51.525902  4665 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.604918  4539 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.607000  4555 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:51.598960  4728 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:51.609182  1674 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.191s (SENT)
I20260709 13:01:51.607095  4539 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.630362  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:51.597301  4416 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:51.641363  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.629927  4555 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:51.646692  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69171us
I20260709 13:01:51.631104  4728 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.648779  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:51.648912  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:51.651520  4555 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.654551  4656 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:51.655997  4431 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.656577  4431 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:51.657043  4431 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.643819  4539 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:51.598976  4697 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:51.661741  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:51.588721  4562 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.647500  4416 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:51.647976  4731 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.607589  4456 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.681109  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.703253  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:51.697149  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.679080  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:51.706230  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.707197  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:51.707937  1629 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:51.709771  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:51.710892  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:51.711679  1628 raft_consensus.cc:2376] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 8. Re-sending same reply.
I20260709 13:01:51.659235  4728 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:51.663753  4729 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:51.718223  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:51.722136  4735 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:51.722625  4735 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.722968  4735 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:51.728047  4431 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.728780  4431 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:51.729168  4431 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.729565  4431 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:51.731122  4735 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.662305  4539 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.739068  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 619) took 3288 ms. Trace:
I20260709 13:01:51.741222  2291 rpcz_store.cc:276] 0709 13:01:48.451170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.451303 (+   133us) service_pool.cc:224] Handling call
0709 13:01:48.451584 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.678243 (+226659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.738768 (+3060525us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540073c3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:51.738998 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482510592}
I20260709 13:01:51.669306  4697 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:51.674258  4562 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:51.742624  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63673.9us
I20260709 13:01:51.720749  4544 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:51.747370  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800318da0 after lost signal to thread 434
I20260709 13:01:51.714305  4730 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:01:51.752262  4730 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:51.752840  4730 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:51.754309  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:51.719906  4731 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.751439  4562 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:01:51.753949  4730 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.756942  4731 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:51.758080  4697 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.757002  4730 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:51.758153  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.765579  4539 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:51.756955  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 731) took 3121 ms. Trace:
W20260709 13:01:51.765599  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2255) took 3734 ms (client timeout 2199 ms). Trace:
I20260709 13:01:51.768972  2296 rpcz_store.cc:276] 0709 13:01:48.635488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.635717 (+   229us) service_pool.cc:224] Handling call
0709 13:01:48.643693 (+  7976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.684910 (+ 41217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.756672 (+3071762us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:51.756883 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81450240}
W20260709 13:01:51.769773  1635 rpcz_store.cc:269] 0709 13:01:48.030812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.030935 (+   123us) service_pool.cc:224] Handling call
0709 13:01:51.765386 (+3734451us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54002972d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.765523 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495297280}
I20260709 13:01:51.773486  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:51.780937  2290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 614) took 3963 ms (client timeout 2188 ms). Trace:
W20260709 13:01:51.781421  2290 rpcz_store.cc:269] 0709 13:01:47.817483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.826651 (+  9168us) service_pool.cc:224] Handling call
0709 13:01:49.028341 (+1201690us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:01:51.780698 (+2752357us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54005d02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.780855 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121946112}
W20260709 13:01:51.783017  2262 connection.cc:256] RPC call timeout handler was delayed by -2.3655571399999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 13:01:51.774154  1701 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:51.784703  4555 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:51.787303  1626 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:51.787734  1626 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:51.788014  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.788465  1626 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.770435  4732 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.790339  4481 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.791255  4733 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:01:51.790956  4481 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.791940  4621 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.792520  4621 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:51.788825  2262 connection.cc:256] RPC call timeout handler was delayed by -1.8773252439999624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 13:01:51.797972  2262 connection.cc:256] RPC call timeout handler was delayed by -1.3615732590000107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:01:51.798446  2262 connection.cc:256] RPC call timeout handler was delayed by -1.1092571130000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 13:01:51.800040  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:51.801081  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 410) took 3650 ms (client timeout 1702 ms). Trace:
I20260709 13:01:51.801791  4610 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.790125  4732 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:51.802405  4610 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:51.804230  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2108) took 4075 ms (client timeout 2072 ms). Trace:
W20260709 13:01:51.804713  1634 rpcz_store.cc:269] 0709 13:01:47.728672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.728811 (+   139us) service_pool.cc:224] Handling call
0709 13:01:51.803980 (+4075169us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54002751d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.804141 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409710848}
I20260709 13:01:51.807017  4610 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:51.807849  1627 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:51.811468  4732 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:51.801594  1702 rpcz_store.cc:269] 0709 13:01:48.151118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.151251 (+   133us) service_pool.cc:224] Handling call
0709 13:01:51.800882 (+3649631us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540010d1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.800999 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302722304}
I20260709 13:01:51.813843  1700 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:51.814644  1708 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:51.815011  4481 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:51.815078  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:51.816300  1704 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:51.816987  1553 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:51.815528  1706 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:01:51.819263  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800357000 after lost signal to thread 1492
I20260709 13:01:51.819556  1705 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:51.819376  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1452) took 3196 ms (client timeout 1560 ms). Trace:
W20260709 13:01:51.820747  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.820771  1701 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:51.821942  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 622) took 3183 ms (client timeout 1809 ms). Trace:
I20260709 13:01:51.819866  1552 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:51.822623  2293 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:51.824015  4621 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:51.827486  2260 connection.cc:256] RPC call timeout handler was delayed by -1.0350162319999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
I20260709 13:01:51.789620  1626 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.452455  4730 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:51.844869  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 128422us
I20260709 13:01:52.461304  4730 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.461681  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:52.461917  1709 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:01:52.462131  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.239475  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.462952  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.463433  1557 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:52.417833  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.464016  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.464589  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:52.464911  1709 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:52.464993  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.465392  4726 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.833391  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.479151  2290 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.478857  4726 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.463469  4730 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:52.480044  4730 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.862653  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1415) took 3396 ms. Trace:
I20260709 13:01:52.480172  4456 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.480535  4730 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.480598  1707 rpcz_store.cc:276] 0709 13:01:48.465928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.958955 (+493027us) service_pool.cc:224] Handling call
0709 13:01:51.506505 (+2547550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.506708 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.506733 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:51.506777 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:51.506815 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.506866 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.862389 (+355523us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b540010d6d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.862567 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5858189184}
I20260709 13:01:52.480186  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:52.466362  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.483418  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8624713940000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:01:51.861649  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.484462  1675 connection.cc:256] RPC call timeout handler was delayed by -3.21124619699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.268s for a response
W20260709 13:01:52.484977  1675 connection.cc:256] RPC call timeout handler was delayed by -2.3221292899999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:01:52.485383  1675 connection.cc:256] RPC call timeout handler was delayed by -2.2367776469999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
W20260709 13:01:52.485764  1675 connection.cc:256] RPC call timeout handler was delayed by -2.1908918689999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:01:52.468672  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080030ec40 after lost signal to thread 1584
W20260709 13:01:52.486578  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3aa0 after lost signal to thread 1585
I20260709 13:01:52.486677  4656 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.487180  4656 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:52.487907  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080032d9a0 after lost signal to thread 1659
W20260709 13:01:52.488268  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce200 after lost signal to thread 1660
I20260709 13:01:52.489136  4656 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:52.489408  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800297a40 after lost signal to thread 1733
I20260709 13:01:52.489673  4656 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.490156  4656 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:52.491076  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800312700 after lost signal to thread 2321
W20260709 13:01:52.491449  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800302b40 after lost signal to thread 2322
W20260709 13:01:52.491858  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080024b900 after lost signal to thread 4322
I20260709 13:01:52.469316  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.493508  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.494246  1700 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:52.494498  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.495262  2291 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 4: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.495965  4642 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:52.496102  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.496488  4642 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.497162  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.497851  2291 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.498550  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.499083  2291 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:51.759584  4697 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:52.173195  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.083491  4728 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.501791  4728 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.461926  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.503257  1705 raft_consensus.cc:2410] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:52.156633  4539 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:52.504074  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.504711  1705 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:52.504467  4539 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.505577  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.506408  1705 raft_consensus.cc:2376] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 11. Re-sending same reply.
I20260709 13:01:52.452811  4737 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.239969  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.492851  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:51.943266  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.508045  1704 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:52.508219  1703 raft_consensus.cc:2376] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 7. Re-sending same reply.
I20260709 13:01:52.110685  4735 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.507519  1559 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:52.225792  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.514461  1560 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
W20260709 13:01:52.468742  2260 connection.cc:256] RPC call timeout handler was delayed by -1.0116162319999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
I20260709 13:01:52.476975  1706 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:52.477285  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.515882  1702 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:52.516712  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.517285  1702 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:52.226781  1551 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:52.460269  4736 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.482715  4730 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:52.482848  4726 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.246706  4252 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:51.894625  4431 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:51.914083  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 739) took 2674 ms. Trace:
I20260709 13:01:52.532698  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:51.960054  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68553.2us
I20260709 13:01:52.491892  3767 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:52.492408  4555 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.534325  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 407) took 3295 ms. Trace:
I20260709 13:01:52.534258  3767 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.492525  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.535696  1707 raft_consensus.cc:2376] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 8. Re-sending same reply.
I20260709 13:01:52.492830  4649 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.493211  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.495100  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.536666  1701 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:01:51.832470  2296 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:52.001511  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 162756us
W20260709 13:01:52.537549  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.533485  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71898.4us
W20260709 13:01:52.539089  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.539201  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.534896  1552 rpcz_store.cc:276] 0709 13:01:49.238557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.238776 (+   219us) service_pool.cc:224] Handling call
0709 13:01:51.819677 (+2580901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.473203 (+653526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.534154 (+ 60951us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b54000771d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:52.534272 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5934377216}
I20260709 13:01:52.539431  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:52.163036  4731 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:51.868541  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.542490  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.542981  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.543093  1707 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:51.887787  4117 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.545579  1555 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 9. Ops: []
I20260709 13:01:52.461688  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.546388  1708 raft_consensus.cc:2376] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:01:52.509701  1558 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:52.510499  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08001aecc0 after lost signal to thread 4618
I20260709 13:01:52.550904  1702 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:52.510893  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.552186  2291 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:01:52.511435  1675 connection.cc:256] RPC call timeout handler was delayed by -1.2000152909999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28330000000000005s for a response
W20260709 13:01:52.463792  1554 rpcz_store.cc:269] 0709 13:01:48.623037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.630312 (+  7275us) service_pool.cc:224] Handling call
0709 13:01:51.819144 (+3188832us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000771d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.819275 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664986368}
I20260709 13:01:52.472947  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 867) took 3422 ms. Trace:
I20260709 13:01:52.553787  2293 rpcz_store.cc:276] 0709 13:01:49.050247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.050354 (+   107us) service_pool.cc:224] Handling call
0709 13:01:51.822317 (+2771963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.466789 (+644472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.472693 (+  5904us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54004c1650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:52.472878 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6348226432}
I20260709 13:01:52.465823  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:52.519099  2260 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.676s (SENT)
I20260709 13:01:52.521615  4539 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.522529  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.556743  3767 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.557098  1709 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:01:52.522716  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 437) took 3996 ms (client timeout 2049 ms). Trace:
I20260709 13:01:52.532433  2289 rpcz_store.cc:276] 0709 13:01:49.239287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.239564 (+   277us) service_pool.cc:224] Handling call
0709 13:01:49.255248 (+ 15684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.300644 (+ 45396us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:50.817368 (+1516724us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.856464 (+ 39096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.856527 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:50.856571 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.913740 (+1057169us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b540040a954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:51.913817 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.914003 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1520562,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":110,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":103721728,"thread_start_us":38670,"threads_started":1,"wal-append.queue_time_us":40589,"wal-append.run_cpu_time_us":1109,"wal-append.run_wall_time_us":1002817}
W20260709 13:01:52.557865  1627 rpcz_store.cc:269] 0709 13:01:48.526235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.645725 (+119490us) service_pool.cc:224] Handling call
0709 13:01:52.522525 (+3876800us) spinlock_profiling.cc:243] Waited 3.09 s on lock 0x7b54001428d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:52.522646 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7115332992}
I20260709 13:01:52.524866  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 405) took 3364 ms. Trace:
I20260709 13:01:52.525112  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.526541  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.560038  1705 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 8. Re-sending same reply.
I20260709 13:01:52.560314  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.561076  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.561425  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.559029  1553 rpcz_store.cc:276] 0709 13:01:49.160336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.160546 (+   210us) service_pool.cc:224] Handling call
0709 13:01:49.160845 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.465257 (+3304412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.524567 (+ 59310us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b54000ea454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:52.524796 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6107842304}
I20260709 13:01:52.528225  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:52.557682  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:52.562511  1704 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:52.530014  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.530534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.533530  1560 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:52.563000  1703 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:52.540983  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.563300  1559 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:52.563658  1552 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:01:52.564766  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.565841  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:52.541000  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.566632  2296 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:52.566987  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.567934  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.568413  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.568845  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.569049  1554 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:52.554930  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:52.555670  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.527189  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.561529  1551 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:52.571712  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.572508  1551 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:52.572870  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.573485  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.573975  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:52.474757  2295 rpcz_store.cc:269] 0709 13:01:48.638939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.639221 (+   282us) service_pool.cc:224] Handling call
0709 13:01:51.821765 (+3182544us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54004c1654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:51.821875 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24895232}
I20260709 13:01:52.554930  1557 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:52.575423  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:52.569907  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50843.8us
I20260709 13:01:52.570725  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.576610  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:52.576792  2260 leader_election.cc:336] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.702s (SENT)
I20260709 13:01:52.576841  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.577337  1709 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:52.577407  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.577746  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.578334  1634 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:52.580392  1709 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:52.580971  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:52.507092  4737 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.578539  1551 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:52.592379  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.592964  2293 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:52.593842  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.594275  1628 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:52.578181  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.589819  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.582876  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.596196  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.596601  1558 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:52.583973  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.596748  1559 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:52.597342  1709 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:52.584898  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.598512  1706 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:52.540187  4656 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.585858  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.557282  4735 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:52.612752  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.532097  4252 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.616127  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.588063  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.616737  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:52.587946  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.565207  4726 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:52.591477  4739 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:52.542047  4481 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:01:52.545389  4610 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:01:52.603137  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.592577  4732 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:52.595283  1553 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:52.604380  2294 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:52.640322  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.640928  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.604970  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.642441  1704 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:52.607275  4642 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.608726  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.608851  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:52.586381  1631 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:01:52.654659  1631 pending_rounds.cc:85] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:01:52.654851  1551 raft_consensus.cc:2410] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:52.654923  1631 pending_rounds.cc:107] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:01:52.655308  1631 raft_consensus.cc:2889] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:52.641744  2294 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:52.593730  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.565044  4623 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:01:52.610575  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.558357  4731 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.611999  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.615020  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.664476  1628 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:52.610390  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:52.665777  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 55451.9us
I20260709 13:01:52.666499  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:52.616634  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.617285  1627 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:52.588426  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.617414  2291 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:52.660573  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81274.7us
I20260709 13:01:52.618379  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.669687  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.670410  1634 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:52.670531  1704 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:52.588924  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:52.670769  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44051us
I20260709 13:01:52.616709  1630 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:52.619910  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.600919  2295 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:52.673418  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 56839.5us
I20260709 13:01:52.580572  4494 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:52.587806   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:52.628465  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.532299  4736 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.586138  4730 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:52.630481  1560 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:01:52.684404  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.594832  4456 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:52.637147  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.653445  4640 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.639750  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.688387  1553 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:52.656495  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.689333  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:52.542016  4555 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.595645  1629 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:52.692416  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.656495  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.693251  1705 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 3.
I20260709 13:01:52.610677  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:52.560993  4117 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.660923  1558 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:52.697047  1558 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:52.698138  1558 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:52.700419  1553 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:01:52.701231  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90616.8us
I20260709 13:01:52.660562  1557 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:52.703792  1555 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:52.704579  1558 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:01:52.660899  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84572.1us
I20260709 13:01:52.662125  1559 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:52.613484  1635 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 4.
I20260709 13:01:52.675061  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.707392  1704 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:52.676879  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.708588  1706 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:52.678457  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.709720  1709 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:52.636098  4621 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:52.681911  2293 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.682458  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.682458  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:52.712427  1556 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:52.685619  2292 raft_consensus.cc:2368] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:52.546432  4649 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.718751  4649 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.657423  4656 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.719689  4656 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.722620  4656 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.639899  4735 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:52.691212  4746 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:52.736301  4746 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.736759  4746 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:52.736630  4735 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.692195  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.694561  1702 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:52.695039  1554 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:52.686909  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.741498  1633 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:52.750578  1556 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:52.751868  1631 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:01:52.672585  4700 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.779224  1552 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:52.753074  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51159.3us
I20260709 13:01:52.780463  2296 raft_consensus.cc:2410] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:52.753966  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:52.764178  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88258.4us
I20260709 13:01:52.781253  2288 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.672806  4642 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.768962  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57276.6us
I20260709 13:01:52.783128  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.770990  1634 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:52.773694  1560 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:52.690371  4368 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.685927  4728 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:52.781904  1626 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:52.781145  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 111072us
I20260709 13:01:52.787578  3767 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:52.797855  3767 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.798324  3767 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.799809  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.772436  4730 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:52.811057  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2277) took 4538 ms. Trace:
I20260709 13:01:52.765403  4655 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:52.771883  4745 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:52.783346  4649 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.816345  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 208) took 13793 ms (client timeout 15000 ms). Trace:
I20260709 13:01:52.815008  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:52.810878  4653 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:01:52.805820  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:52.809578  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 122897us
I20260709 13:01:52.804481  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.800451  1629 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:52.768385  4456 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:52.811504  1626 rpcz_store.cc:276] 0709 13:01:48.258917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.296606 (+ 37689us) service_pool.cc:224] Handling call
0709 13:01:48.304431 (+  7825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.787184 (+3482753us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b54001e3fa0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:01:52.796578 (+1009394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.796771 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119783808}
I20260709 13:01:52.697793  4640 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.696873  4740 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:01:52.696646  4609 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 13:01:52.800629  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.826887  3767 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.827416  3767 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:52.828886  4731 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.814334  4745 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:52.821036  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:52.816965  1457 rpcz_store.cc:269] 0709 13:01:39.026854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.027116 (+   262us) service_pool.cc:224] Handling call
0709 13:01:52.812714 (+13785598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:48.384835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:48.524257 (+139422us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:48.524309 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:01:48.524327 (+    18us) tablet.cc:662] Decoding operations
0709 13:01:48.525373 (+  1046us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:48.525474 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:48.525498 (+    24us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:01:48.526540 (+  1042us) tablet.cc:701] Row locks acquired
0709 13:01:48.526567 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:01:48.526725 (+   158us) write_op.cc:270] Start()
0709 13:01:48.526853 (+   128us) write_op.cc:276] Timestamp: P: 1783602108526658 usec, L: 0
0709 13:01:48.526874 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:01:48.527512 (+   638us) log.cc:844] Serialized 4838 byte log entry
0709 13:01:49.120701 (+593189us) op_driver.cc:464] REPLICATION: finished
0709 13:01:49.286704 (+166003us) write_op.cc:301] APPLY: starting
0709 13:01:49.291862 (+  5158us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:49.292158 (+   296us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:49.292296 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:49.317901 (+ 25605us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:49.318222 (+   321us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:01:49.318293 (+    71us) write_op.cc:312] APPLY: finished
0709 13:01:49.318697 (+   404us) log.cc:844] Serialized 89 byte log entry
0709 13:01:49.318896 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.319030 (+   134us) write_op.cc:454] Released schema lock
0709 13:01:49.319078 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:49.355336 (+ 36258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.355376 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":434828,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":132746,"apply.run_cpu_time_us":7321,"apply.run_wall_time_us":55620,"num_ops":12,"prepare.queue_time_us":166422,"prepare.run_cpu_time_us":30979,"prepare.run_wall_time_us":494697,"raft.queue_time_us":190586,"raft.run_cpu_time_us":14757,"raft.run_wall_time_us":104233,"replication_time_us":583207,"thread_start_us":498430,"threads_started":4,"wal-append.queue_time_us":368010,"wal-append.run_cpu_time_us":21991,"wal-append.run_wall_time_us":1423973}]]}
I20260709 13:01:52.810864  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.835748  3767 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:52.778987  4621 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:01:52.812705  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.822710  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.822810  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.806473  4656 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:52.779224  4686 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:52.811463  4730 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.786550  4700 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:52.851567  4730 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:52.813277  4655 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:52.830561  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.827615  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.831728  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.833491  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.837486  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.810997  4494 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:01:52.829640  4735 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:01:52.815362  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.846709  4117 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.847379  4748 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:52.799986  4750 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:01:52.821870  4431 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.820534  4649 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.848407  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.847874  1455 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 6, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 6 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:52.838469  4539 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:52.839731  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:52.849555  4640 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.805864  4642 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.850497  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.850184  4656 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.851020  4686 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.812355  4368 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:52.854882  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.856624  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.864403  1628 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:01:52.863780  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 254951us
I20260709 13:01:52.853272  4655 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:52.863986  4609 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:52.868670  1560 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
W20260709 13:01:52.985908  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 129402us
I20260709 13:01:52.853044  4700 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:52.873093  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.873624  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:52.868248  1551 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:52.875937  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:52.876832  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:52.829195  4743 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:01:52.989202  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 149527us
I20260709 13:01:52.853518  1635 raft_consensus.cc:2376] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6. Re-sending same reply.
I20260709 13:01:52.993319  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.993976  1626 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:52.994287  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.994791  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.994975  1635 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:52.995359  1626 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:52.888163  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.995862  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:52.989207  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:52.996337  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108305us
I20260709 13:01:52.996809  1560 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:52.997174  1629 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:52.900962  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:52.997759  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 124740us
I20260709 13:01:52.910768  1552 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:52.966269  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:52.966797  1709 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:52.866654  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:52.949394  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 282898us
I20260709 13:01:52.999197  1634 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:52.967614  1703 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:52.999222  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:52.852996  4555 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.898692  4748 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:52.865659  4732 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:52.912940  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:52.854025  1631 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:01:52.986960  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:52.960387  4753 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:01:52.903045  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:52.998485  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.867389  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:53.078895  1633 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:52.861506  4736 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:52.990392  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:53.001345  2296 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:53.082660  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84246.2us
I20260709 13:01:52.991648  2292 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:53.002306  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.082943  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.083434  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.083536  1626 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:52.967247  4539 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:52.994621  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.008419  4117 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:53.009531  4656 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.009364  4610 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:52.851718  4252 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:53.012470  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.038440  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.077533  4732 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:53.032346  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 249459us
I20260709 13:01:53.010201  4762 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.033150  2290 raft_consensus.cc:2410] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:01:53.022384  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221539us
I20260709 13:01:53.076045  2288 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:53.019217  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.080009  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.984037  4686 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.011441  1632 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:53.084677  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:52.993352  4494 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.158301  1552 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:52.903996  4752 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.235s
I20260709 13:01:53.094934  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.955946  4431 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 13:01:53.032634  4731 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:53.162923  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:52.861857  4456 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:53.110818  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 111716us
I20260709 13:01:53.041539  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.095559  4737 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:53.131990  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 128060us
I20260709 13:01:53.173533  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:53.173970  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.128898  4769 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.134445  2294 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:01:53.120611  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87316.5us
I20260709 13:01:53.103719  4759 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:53.139483  4762 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.100020  1455 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.135905  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.032823  4623 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:01:53.089838  2297 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:53.071676  4697 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:01:53.159111  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.194916  1555 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:53.094067  4609 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.087873  4726 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:53.151988  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:53.201102  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49243.1us
I20260709 13:01:53.202263  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.010880  4754 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:52.988449  4751 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:53.098896  4700 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.160034  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.211068  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:53.178572  4769 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.213986  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:53.075104  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88240.4us
I20260709 13:01:53.121132  4768 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.169327  1552 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:53.163710  1560 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:53.177727  3767 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:53.124014  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 129422us
I20260709 13:01:52.961000  4745 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.180820  4766 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:53.158619  4494 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.163208  4758 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.129s
I20260709 13:01:53.184655  4767 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:53.164995  4731 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:53.161734  4736 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:01:53.182008  4759 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.095095  4117 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.181728  4621 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:01:53.215430  1455 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 5, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.224895  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:53.219669  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:53.239935  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.235752  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.282909  4776 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:53.164696  4744 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:01:53.284265  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70396.4us
I20260709 13:01:53.172330  4252 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:01:53.284893  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:53.220073  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60119.9us
I20260709 13:01:53.190433  4762 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.279673  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.198351  4609 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:53.218958  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:53.314239  4767 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:53.349958  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 131080us
I20260709 13:01:53.350101  4767 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:53.350283  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 130756us
I20260709 13:01:53.351115  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:53.351177  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:53.285383  4656 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:53.352013  4656 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.352383  4656 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.286691  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.251237  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:53.288406  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 82210.5us
I20260709 13:01:53.353572  1631 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:53.353873  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:53.354427  4775 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.354938  4609 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.355361  1626 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:53.354708  1455 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 3 to 7, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.356930  4621 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.354998  1627 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:53.356679  1455 catalog_manager.cc:5697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 7, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.183691  1554 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:53.310290  4494 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.204813  4555 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:01:53.303042  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.279031  3767 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:53.208415  4431 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:53.359407  4431 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:53.290014  4368 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.359911  4431 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:53.360160  4117 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.360272  3767 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:53.298482  2290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 876) took 1441 ms (client timeout 1519 ms). Trace:
W20260709 13:01:53.360968  2290 rpcz_store.cc:269] 0709 13:01:51.857481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.479876 (+622395us) service_pool.cc:224] Handling call
0709 13:01:53.298198 (+818322us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:53.298403 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213372672}
I20260709 13:01:53.361590  4759 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:53.301926  2295 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:53.161151  4746 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:53.312251  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 227618us
I20260709 13:01:53.362540  2295 pending_rounds.cc:85] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 8
I20260709 13:01:53.362879  2295 pending_rounds.cc:107] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:53.363108  2295 raft_consensus.cc:2889] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:53.363406  1524 leader_election.cc:400] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:53.363703  1524 leader_election.cc:403] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:53.364111  4746 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.285392  4769 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.365834  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.366817  1629 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:53.139334  4748 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:53.371382  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 84739.2us
I20260709 13:01:53.208586  4751 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:53.357379  4621 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.368614  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 89015.4us
I20260709 13:01:53.286275  4768 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.306340  2288 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:53.218923  4700 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.372670  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.383285  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.288599  4726 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:53.287634  4766 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:53.301983  4610 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.418815  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:53.468386  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82017.5us
I20260709 13:01:53.418358  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.313834  4539 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.417730  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:53.481478  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 98269.7us
I20260709 13:01:53.342799  4456 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:53.368058  4775 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.388104  4621 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:53.494078  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 283058us
I20260709 13:01:53.210958  4774 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.176s
I20260709 13:01:53.384637  4640 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:01:53.508392  3767 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:01:53.376344  4750 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:53.469065  4700 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.478365  1632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:01:53.479673  4765 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.469749  4697 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:53.484513  4368 consensus_queue.cc:1175] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "444b494bb53842b59408a7230884398a" responder_term: 8 status { last_received { term: 6 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:53.519352  4775 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.417905  4755 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.138s
W20260709 13:01:53.567826  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 195163us
I20260709 13:01:53.570255  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:53.522334  4539 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.520735  4770 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:53.570958  1630 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8. Re-sending same reply.
I20260709 13:01:53.558153  4649 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.386675  4653 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.511711  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.509447  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:53.371556  4746 raft_consensus.cc:2764] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:53.289214  4754 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.201s
I20260709 13:01:53.566871  4776 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.522799  1455 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.415501  4728 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:53.490360  4766 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:53.537444  1629 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:53.373488  4745 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:53.490736  4610 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:53.520750  4758 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.570595  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.558998  4456 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.508374  4782 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:53.582202  4749 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:53.578320  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66693.2us
I20260709 13:01:53.644611  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.646091  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:53.646570  4778 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:53.565775  4481 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:53.647451  4778 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:53.647959  4481 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:53.648953  4481 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.648911  4778 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:53.650550  4481 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:53.652251  4777 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:53.653790  4777 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:53.564922  4771 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.611651  4740 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:53.578868  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 97738.1us
I20260709 13:01:53.556815  1632 raft_consensus.cc:606] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:53.385741  4764 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:53.397231  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:53.661603  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.566355  4697 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.574139  4770 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.591756  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.597640  4621 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.583900  4653 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.584087  4728 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.654798  4771 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:53.611433  1455 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.572716  4763 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:01:53.638988  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 463831us
I20260709 13:01:53.684060  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.643260  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.685231  2295 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:53.590699  4758 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:53.469570  4748 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:53.655913  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 290158us
W20260709 13:01:53.656865  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 303038us
I20260709 13:01:53.694159  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.673048  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:53.399641  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.566653  4765 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:53.662482  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 265304us
I20260709 13:01:53.595494  4431 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:53.667970  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76315.2us
I20260709 13:01:53.581799  4776 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.590533  4610 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:53.700404  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 300802us
I20260709 13:01:53.617918  4456 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:53.665540  4697 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.694173  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.695032  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.661724  1632 raft_consensus.cc:614] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:53.696599  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.419497  4768 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.734496  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.664201  4764 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.734184  1455 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 1 to 4, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.701006  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:53.661360  4749 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.702349  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.699556  4555 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.704058  4765 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.704519  1635 raft_consensus.cc:2376] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5. Re-sending same reply.
W20260709 13:01:53.705984  4686 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.538s	user 0.001s	sys 0.007s
I20260709 13:01:53.704864  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.704993  4752 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:53.705401  4773 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:01:53.573091  4751 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.708195  4780 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:53.667302  4770 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.709672  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.712232  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:53.667191  4740 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:01:53.716548  4730 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.864s	user 0.009s	sys 0.001s
I20260709 13:01:53.772318  4735 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:53.671110  4728 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.699836  4754 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:53.724174  4782 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:53.701954  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:53.804299  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102402us
I20260709 13:01:53.805238  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:53.666021  4746 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.739440  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.692732  4758 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.745951  1555 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:53.687390  4789 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:53.773136  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111073us
I20260709 13:01:53.826443  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:53.728113  4776 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.670748  4786 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
W20260709 13:01:53.776748  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82650.1us
W20260709 13:01:53.845973  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.846210  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:53.777411  4796 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:53.778579  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.783058  1556 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.727988  4621 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:53.742377  4749 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.741472  4649 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.742507  4736 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:53.721714  4653 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.824723  4368 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:53.789109  4794 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:53.792954  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97988.5us
I20260709 13:01:53.790335  1455 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 5 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:01:53.747645  4781 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:53.768426  4731 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:53.795934  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.806980  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.781819  4780 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.809203  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69874.7us
I20260709 13:01:53.790836  4788 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.738133  4769 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:53.825358  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:53.829938  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.789391  4791 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:01:53.795811  4730 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.794024  4735 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:53.846729  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:53.846879  1555 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:01:53.848781  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.005s	sys 0.012s
I20260709 13:01:53.772898  4686 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.741066  4764 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:53.821046  4783 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:53.850467  4655 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.850s	user 0.009s	sys 0.002s
I20260709 13:01:53.783659  4759 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.841048  4456 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:53.888230  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 92406.6us
I20260709 13:01:53.925946  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:53.927964  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 149439us
I20260709 13:01:53.929374  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:53.931728  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.828197  4754 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:53.835992  4555 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:53.890389  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:53.892213  1455 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 6, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.892911  4768 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.836859  4431 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.835738  4771 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:53.901203  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75951.6us
I20260709 13:01:53.848738  4789 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:53.913722  4756 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:53.905658  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.908778  1556 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:53.935573  4656 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.005s
I20260709 13:01:53.974166  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:53.911937  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.005s	sys 0.012s
I20260709 13:01:53.874737  4368 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.808404  4785 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:01:53.881884  4772 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.942287  4623 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.824731  4746 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.882249  4794 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.843024  4494 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.894729  4788 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.924474  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 77804.2us
I20260709 13:01:53.897523  4736 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:53.887081  4780 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:53.937708  1455 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.941372  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.910874  4731 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:53.910086  4777 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:53.965020  4686 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.001672  4772 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.972180  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 81879.2us
I20260709 13:01:53.972333  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:53.886457  4700 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.988577  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:53.989641  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:53.962417  4697 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.970877  4756 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.042757  1556 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.942768  4752 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.824715  4609 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.909709  4763 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:01:53.922540  4783 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.936393  4655 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.004958  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:53.997112  4656 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.016400  4751 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.006469  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.007026  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:53.958304  4767 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.000s
I20260709 13:01:53.944046  4781 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:54.010797  1707 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:53.988386  4456 raft_consensus.cc:2764] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:53.948107  4726 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:53.961252  4642 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:53.997354  4368 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:54.132982  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.022060  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.024662  2297 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:54.052762  4739 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:53.851590  4796 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:54.021831  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:54.009195  4117 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.648s	user 0.005s	sys 0.009s
I20260709 13:01:53.996937  4774 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:54.149993  4774 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.150612  4655 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.152308  4774 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.001375  4769 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.160790  4730 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.161918  4653 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:54.148051  4739 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:54.135986  4781 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:54.172747  4739 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.173382  4781 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.948663  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:54.176657  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 228057us
I20260709 13:01:54.177589  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.034165  4731 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:54.179061  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.011166  4623 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.179980  4623 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.181200  4731 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.184192  4456 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:54.184746  4456 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.186837  4456 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.186995  4764 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.040009  2296 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:54.041060  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.007843  4754 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:53.950410  4610 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:54.048367  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58872.9us
I20260709 13:01:54.018162  4788 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.051981  1556 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:54.053262  1709 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:54.004356  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.218089  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:54.218918  1706 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:54.005666  1703 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:54.061408  4649 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:54.064791  4768 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:54.066371  4732 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.906s	user 0.006s	sys 0.002s
I20260709 13:01:53.955404  4766 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.097244  4767 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:53.913303  4740 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.068398  4759 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.015149  4794 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:54.002744  4735 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.072546  4783 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.073153  4752 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:54.074558  4609 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.074695  4700 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.075114  1551 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:54.076329  4697 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.077068  1627 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:54.061450  4494 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.077508  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:54.079288  4758 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.025295  4777 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.091796  1708 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:54.260736  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.042855  4772 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.098215  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:54.098812  1634 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:01:54.126084  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 429511us
I20260709 13:01:54.044435  4756 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:54.016120  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.016131  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:54.135345  1705 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:54.133787  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:54.146076  1700 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:54.140455  2295 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.148488  4117 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.142432  4796 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:54.038919  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 231978us
I20260709 13:01:54.280767  1633 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:54.198813  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 157852us
I20260709 13:01:54.204685  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:54.220292  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.282102  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.282534  1703 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.282653  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:54.283270  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.248251  4746 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.222330  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.151353  4728 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:54.203339  4775 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.006s
I20260709 13:01:54.228713  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:54.207403  4780 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.297441  1556 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:54.233736  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.235427  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:54.236265  2292 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.202576  4736 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:54.223163  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:54.235216  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:54.067334  4762 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.682s	user 0.002s	sys 0.011s
I20260709 13:01:54.204288  4456 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:54.217753  4481 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.001s
W20260709 13:01:54.223905  4778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.008s	sys 0.033s
I20260709 13:01:54.207705  4726 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:54.422240  4778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.009s	sys 0.033s
I20260709 13:01:54.422559  4726 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:54.423539  4766 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.423717  4726 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.424906  4767 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.355914  4762 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.369674  4763 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:54.380116  4776 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.000s
W20260709 13:01:54.251704  4745 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.000s
I20260709 13:01:54.261432  1708 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:01:54.272553  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 256469us
W20260709 13:01:54.273370  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.239409  4555 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:01:54.280664  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2307) took 1393 ms. Trace:
I20260709 13:01:54.379096  4785 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.206540  4809 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:54.286602  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:54.462801  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 176281us
I20260709 13:01:54.463596  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.465194  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:54.097250  4789 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.288573  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2148) took 1055 ms. Trace:
I20260709 13:01:54.467981  1633 rpcz_store.cc:276] 0709 13:01:53.233649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.233956 (+   307us) service_pool.cc:224] Handling call
0709 13:01:54.280557 (+1046601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.281297 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.288301 (+  7004us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540019c650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:54.288499 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2405994368}
I20260709 13:01:54.250162  4752 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.282357  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:54.287789  4774 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:54.292837  1558 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:54.469909  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 187622us
I20260709 13:01:54.470482  4640 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:54.470971  4640 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:54.471077  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:54.471469  4640 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.298166  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:01:54.298595  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63305.4us
I20260709 13:01:54.301419  2292 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:54.316999  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:54.316743  2288 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:54.429400  4748 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.009s
I20260709 13:01:54.243744  4732 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.478616  4732 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.478752  4778 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:54.266943  4777 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.318341  1706 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:54.480525  1553 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:54.480729  4767 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:54.481484  4739 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:54.481886  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.482213  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.482620  4610 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.471000  4782 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:01:54.448233  4776 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.483793  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:54.484609  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.485291  4776 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.353277  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.479449  4655 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:54.485893  1552 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.486493  1552 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:54.486434  4758 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.440811  4763 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.487311  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.488987  1557 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:01:54.489985  1706 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:54.490581  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.491966  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.350521  1709 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:54.298502  4735 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.483963  4782 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:54.493103  4782 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:54.493978  4782 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.495208  4782 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:54.352890  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.489917  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:54.207995  4754 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:54.319281  4653 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.291096  4494 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:54.508534  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:54.349462  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.515080  1705 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:54.297778  4700 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
W20260709 13:01:54.421811  4806 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:01:54.417186  4609 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.430835  4762 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.246078  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:54.385622  4749 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.534297  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:54.433892  4810 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:54.436013  4728 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.437353  4117 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.246732  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.438684  2291 raft_consensus.cc:2410] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:54.441844  1701 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:54.442098  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.191393  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.453574  1634 rpcz_store.cc:276] 0709 13:01:52.887466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.012117 (+124651us) service_pool.cc:224] Handling call
0709 13:01:54.280401 (+1268284us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b540019c654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:54.280595 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218211584}
I20260709 13:01:54.454876  4555 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:54.315429  4775 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.457787  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:54.458310  2295 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:54.472751  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.473819  2294 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:54.475955  4649 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.480293  4778 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:54.484997  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:54.481907  1626 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:54.487506  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:54.321069  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98031.4us
I20260709 13:01:54.487932  4744 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.487344  4763 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.492077  1555 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:54.301792  4736 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.297111  4656 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.353521  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:54.499938  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.500289  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.263866  4769 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:01:54.294680  4797 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:54.432828  4808 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:54.535234  1557 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:54.517216  1630 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:01:54.548215  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 302197us
I20260709 13:01:54.461328  4785 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.683696  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.732903  1632 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.674368  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.579110  1702 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:54.737005  1635 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:54.742177  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:54.743501  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.746888  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:54.514122  4774 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.736030  2290 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.734825  1709 raft_consensus.cc:2376] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8. Re-sending same reply.
I20260709 13:01:54.749078  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.737651  1559 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:54.749112  1635 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:54.737056  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 545814us
I20260709 13:01:54.749800  1704 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.751792  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.752359  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:54.749722  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.753602  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:54.756233  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.757121  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:54.682078  4817 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:01:54.707465  4368 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.008s
I20260709 13:01:54.682423  4610 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.766995  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:54.762450  4728 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:54.768282  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.548120  4494 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:54.732903  4555 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:54.726052  4655 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:54.760646  4656 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:01:54.552121  4686 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:54.682200  4762 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.739100  4736 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:54.732999  4778 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.775104  1628 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.775657  2293 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:54.652890  4764 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:54.778452  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:54.706883  4740 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:54.768468  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:54.798626  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.747678  4732 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:54.806463  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.024s
W20260709 13:01:54.821513  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.012s	sys 0.024s
I20260709 13:01:54.821626  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.803846  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:54.774037  4745 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:54.811167  1704 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:54.605300  4623 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.777997  4780 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:54.821516  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53322.3us
I20260709 13:01:54.835011  1709 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:54.835525  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:54.809969  4770 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.015s	user 0.011s	sys 0.000s
I20260709 13:01:54.890218  4770 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.891597  1700 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:01:54.897804  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 144327us
I20260709 13:01:54.900486  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:54.902560  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:54.904917  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69447.1us
I20260709 13:01:54.906067  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:54.815232  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:54.811369  4754 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:54.798959  4610 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.836191  4728 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:54.843225  4653 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:54.842882  4740 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.844133  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:54.916325  1709 raft_consensus.cc:606] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:54.916746  1709 raft_consensus.cc:614] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:54.803077  4697 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:54.846745  4623 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:54.549970  4609 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.816334  4762 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.772742  4774 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.834049  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.882506  4555 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:54.886350  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.933090  1704 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.888351  4494 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:54.809303  4700 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:54.908633  1708 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:54.940802  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.941377  1708 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:54.942202  1628 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:01:54.918773  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.739588  4744 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:54.929008  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 444042us
I20260709 13:01:54.830327  4778 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.991335  1708 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:54.779013  4368 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:54.939352  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:54.943051  1705 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:54.817744  4686 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:54.928946  4653 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:54.814970  1632 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:01:54.947436  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:55.011085  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63728.9us
I20260709 13:01:55.011312  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:54.920135  4739 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:54.919854  4732 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:54.931334  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:55.012776  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 81562.1us
W20260709 13:01:55.013410  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.999248  1704 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
W20260709 13:01:54.946884  4751 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.842s	user 0.006s	sys 0.005s
W20260709 13:01:55.018157  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 78879.6us
I20260709 13:01:55.018893  1633 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.942909  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:01:54.938172  4754 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:55.014310  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:55.004354  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:54.933154  4697 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:54.933388  4728 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:54.972031  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 239547us
I20260709 13:01:55.032845  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:55.013458  1630 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:55.033623  1705 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:55.015457  1632 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:01:55.012573  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:55.034763  1632 pending_rounds.cc:85] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:01:55.035027  1632 pending_rounds.cc:107] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:55.035243  1632 raft_consensus.cc:2889] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.001166  4780 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:55.025789  1708 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.038637  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:55.017292  4555 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:55.008826  4749 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.010116  4653 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:55.018620  4739 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.026248  4735 raft_consensus.cc:2764] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:01:55.041836  4700 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.022722  4797 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:55.018426  4732 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:55.090819  1633 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:55.154409  1708 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
W20260709 13:01:55.046797  4777 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.000s
I20260709 13:01:55.166527  4736 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.030808  4728 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:01:55.167212  4756 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.000s
I20260709 13:01:55.167505  4728 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:55.158296  4821 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:01:55.162617  4368 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:55.163043  4772 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.874s	user 0.008s	sys 0.000s
I20260709 13:01:55.165398  4555 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:55.166419  4777 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.043715  4769 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:55.172515  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 429) took 1321 ms. Trace:
I20260709 13:01:55.137791  4732 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:55.087963  4739 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:55.194016  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:55.284423  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 106744us
I20260709 13:01:55.084581  4789 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:55.283442  4653 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.291978  4653 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.292524  4736 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.276639  4456 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:55.294130  4456 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:55.294916  4456 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.295997  4456 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:55.297791  4777 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.200944  4481 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:55.299292  4481 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.302655  4481 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.173460  4610 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.200702  4735 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:55.172443  4762 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.167687  4756 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.305435  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 771) took 1334 ms. Trace:
I20260709 13:01:55.305781  4756 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.041172  4754 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.305935  2292 rpcz_store.cc:276] 0709 13:01:53.971113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.972079 (+   966us) service_pool.cc:224] Handling call
0709 13:01:55.305182 (+1333103us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400729054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.305361 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449289088}
I20260709 13:01:55.307992  2289 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:55.308310  4754 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.310207  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.248265  4772 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.311359  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:55.270542  1557 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:55.313215  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.313575  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 760) took 2011 ms. Trace:
I20260709 13:01:55.313901  2297 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:55.314117  2294 rpcz_store.cc:276] 0709 13:01:53.302483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.302750 (+   267us) service_pool.cc:224] Handling call
0709 13:01:55.313332 (+2010582us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.313503 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2564459776}
I20260709 13:01:55.314867  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.315894  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.317963  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1453) took 1266 ms. Trace:
I20260709 13:01:55.318533  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 428) took 1471 ms. Trace:
I20260709 13:01:55.319136  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 766) took 1390 ms. Trace:
I20260709 13:01:55.319603  2291 rpcz_store.cc:276] 0709 13:01:53.928890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.931482 (+  2592us) service_pool.cc:224] Handling call
0709 13:01:55.318928 (+1387446us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b5400644bd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.319069 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163670272}
I20260709 13:01:55.320711  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.321691  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.323262  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.323954  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1451) took 1285 ms. Trace:
I20260709 13:01:55.324340  1707 rpcz_store.cc:276] 0709 13:01:54.038675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.320545 (+281870us) service_pool.cc:224] Handling call
0709 13:01:54.373113 (+ 52568us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.373509 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.373536 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.386405 (+ 12869us) log.cc:844] Serialized 52 byte log entry
0709 13:01:54.632161 (+245756us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:54.732442 (+100281us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.896942 (+164500us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b180002f140. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:01:54.897196 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.897266 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.323659 (+426393us) raft_consensus.cc:1685] finished
0709 13:01:55.323760 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.323899 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21229,"prepare.queue_time_us":99980,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3660,"raft.queue_time_us":208308,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15943,"spinlock_wait_cycles":2983680,"thread_start_us":194032,"threads_started":1,"wal-append.queue_time_us":392054}
I20260709 13:01:55.319049  1553 rpcz_store.cc:276] 0709 13:01:53.846714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.846923 (+   209us) service_pool.cc:224] Handling call
0709 13:01:53.847193 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.480961 (+633768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.318242 (+837281us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54002efed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.318455 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456069376}
W20260709 13:01:55.326638  4815 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.621s	user 0.003s	sys 0.000s
I20260709 13:01:55.324508  2291 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:55.329872  4752 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.860s	user 0.012s	sys 0.000s
I20260709 13:01:55.330142  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.330451  4752 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.317044  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.173784  4609 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.163431  4686 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.174086  4794 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.000s
W20260709 13:01:55.176503  4788 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.939s	user 0.009s	sys 0.000s
I20260709 13:01:55.276613  1558 rpcz_store.cc:276] 0709 13:01:53.851059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.851203 (+   144us) service_pool.cc:224] Handling call
0709 13:01:53.851488 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.470140 (+618652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.172296 (+702156us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54001379d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.172458 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014549248}
I20260709 13:01:55.156155  1705 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:01:55.168592  4823 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:01:55.176930  4805 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.911s	user 0.011s	sys 0.000s
I20260709 13:01:55.303421  4735 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.194834  4745 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.334295  4728 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.331800  1551 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:55.318379  1701 rpcz_store.cc:276] 0709 13:01:54.051313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.396357 (+345044us) service_pool.cc:224] Handling call
0709 13:01:54.396803 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.738160 (+341357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.317710 (+579550us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b5400166a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.317901 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99438976}
I20260709 13:01:55.336558  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:55.367322  1627 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:55.336609  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:55.337051  1628 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:55.343923  4640 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.001s
I20260709 13:01:55.336532  2297 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:55.337913  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:55.371498  4740 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.372535  4740 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.081095  4780 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:55.373131  4740 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.374482  4780 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.315543  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.339584  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.342748  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:55.375706  4740 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.376787  2292 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:55.343565  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 329294us
I20260709 13:01:55.377173  4740 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:55.377647  4740 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.378639  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:55.375991  4768 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:55.273175  2290 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.001s
I20260709 13:01:55.380462  4768 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:55.380587  2290 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:55.172907  4785 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.074554  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:55.382165  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 307719us
W20260709 13:01:55.353308  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 340772us
I20260709 13:01:55.382977  1598 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:55.082002  4828 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:55.331064  2291 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:55.303480  4770 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.379937  4754 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:55.381064  4768 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:55.376487  2294 raft_consensus.cc:2368] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5.
I20260709 13:01:55.194329  4726 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.356637  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.393294  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.351136  4775 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:55.422747  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.423352  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.425760  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1442) took 1420 ms. Trace:
I20260709 13:01:55.426251  1702 rpcz_store.cc:276] 0709 13:01:54.005730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.006135 (+   405us) service_pool.cc:224] Handling call
0709 13:01:55.425502 (+1419367us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54001b6f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.425684 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120994688}
I20260709 13:01:55.200121  4700 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.428660  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 763) took 1768 ms. Trace:
I20260709 13:01:55.429270  2288 rpcz_store.cc:276] 0709 13:01:53.660767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.660935 (+   168us) service_pool.cc:224] Handling call
0709 13:01:53.661231 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.473432 (+812201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.428352 (+954920us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.428588 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504807680}
W20260709 13:01:55.423717  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.417699  2292 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:55.345938  4797 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:55.418149  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:55.433976  2291 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:01:55.434948  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.363232  4555 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:55.172829  4697 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.362957  4788 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.384936  4762 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:01:55.405479  4745 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.352473  4794 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.406412  4642 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:55.388445  4117 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:55.526475  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 775) took 1303 ms. Trace:
I20260709 13:01:55.406335  4610 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.379606  4740 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:55.519567  4763 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.751s	user 0.008s	sys 0.003s
I20260709 13:01:55.581077  2296 rpcz_store.cc:276] 0709 13:01:54.222731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.223700 (+   969us) service_pool.cc:224] Handling call
0709 13:01:54.224148 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.224654 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.224697 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.225285 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.821990 (+596705us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:54.822028 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.822093 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.493698 (+671605us) raft_consensus.cc:1685] finished
0709 13:01:55.499406 (+  5708us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.526386 (+ 26980us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3213,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8017,"thread_start_us":596335,"threads_started":1,"wal-append.queue_time_us":796912}
I20260709 13:01:55.420945  4785 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:55.577749  4642 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:55.588933  4642 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.502116  4828 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.512892  4783 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.244s	user 0.011s	sys 0.000s
W20260709 13:01:55.509717  4818 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.561s	user 0.005s	sys 0.005s
I20260709 13:01:55.584117  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.519913  4609 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:55.421717  4754 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:55.442217  4797 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:55.413276  4640 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.512234  4789 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.479820  4777 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.542522  4774 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.466284  4555 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:55.584677  4751 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.504026  4730 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:01:55.509163  4819 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:01:55.583489  4775 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:55.427570  4726 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.598680  4610 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.602773  4797 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:55.607529  4555 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:55.601223  4783 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.630373  4824 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
I20260709 13:01:55.641023  4754 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.641871  4794 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.610442  4777 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.643114  4833 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:01:55.644392  4825 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:01:55.645093  4777 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:55.645922  4830 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:01:55.647325  4785 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:55.630111  4752 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:55.648288  4827 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
W20260709 13:01:55.650681  4796 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.346s	user 0.001s	sys 0.009s
I20260709 13:01:55.652452  4744 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.652660  4728 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.652630  4700 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.654493  4735 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.639349  4775 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.659529  4726 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.660946  4775 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.661247  4783 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.663467  4732 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.665552  4732 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.666507  4788 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.666996  4772 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.666505  4745 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:55.667459  4814 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.120s	user 0.004s	sys 0.001s
I20260709 13:01:55.651235  4796 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.670284  4796 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:55.670961  4846 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:01:55.598559  4763 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.672487  4778 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.670s	user 0.007s	sys 0.004s
I20260709 13:01:55.673188  4763 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.609679  4789 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.673280  4778 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.674841  4789 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.630760  4751 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.676120  4782 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.180s	user 0.011s	sys 0.004s
I20260709 13:01:55.676008  4778 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.668042  4774 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:55.677368  4774 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:55.678174  4751 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.597733  4828 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.683379  4828 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.685341  4739 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.686121  4797 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.690806  4768 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.692083  4797 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.692659  4768 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:55.694569  4777 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:55.695322  4777 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:55.732654  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2141) took 1528 ms. Trace:
I20260709 13:01:55.746045  1555 rpcz_store.cc:276] 0709 13:01:54.204187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.204324 (+   137us) service_pool.cc:224] Handling call
0709 13:01:54.228069 (+ 23745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.736226 (+508157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.731610 (+995384us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000e9a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.732235 (+   625us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606685056}
I20260709 13:01:55.736603  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1500) took 1382 ms. Trace:
I20260709 13:01:55.749143  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2313) took 1236 ms. Trace:
I20260709 13:01:55.749716  1630 rpcz_store.cc:276] 0709 13:01:54.512930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.766391 (+253461us) service_pool.cc:224] Handling call
0709 13:01:55.013225 (+246834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.033894 (+ 20669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.748633 (+714739us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001414d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.748924 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553760000}
I20260709 13:01:55.752444  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1452) took 1707 ms. Trace:
I20260709 13:01:55.753054  1700 rpcz_store.cc:276] 0709 13:01:54.044599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.322844 (+278245us) service_pool.cc:224] Handling call
0709 13:01:54.396997 (+ 74153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.892011 (+495014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.751581 (+859570us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.752190 (+   609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136887680}
I20260709 13:01:55.585011  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.756817  2288 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:55.757052  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1460) took 1695 ms. Trace:
W20260709 13:01:55.737512  1629 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.011s	sys 0.039s
W20260709 13:01:55.758463  1629 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.011s	sys 0.039s
W20260709 13:01:55.759155  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 773) took 1681 ms (client timeout 1975 ms). Trace:
I20260709 13:01:55.759984  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2309) took 1251 ms. Trace:
I20260709 13:01:55.595810  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:55.534157  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72170.8us
I20260709 13:01:55.761076  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 449) took 1086 ms. Trace:
I20260709 13:01:55.761564  1704 rpcz_store.cc:276] 0709 13:01:54.674704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.014835 (+340131us) service_pool.cc:224] Handling call
0709 13:01:55.015135 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.021290 (+  6155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.021523 (+   233us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.153794 (+132271us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.760387 (+606593us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:55.760496 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.760615 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.760677 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:55.760689 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.760892 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4642,"raft.queue_time_us":5217,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7057,"thread_start_us":402865,"threads_started":1,"wal-append.queue_time_us":353458}
I20260709 13:01:55.734625  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2154) took 1452 ms. Trace:
I20260709 13:01:55.763337  1631 rpcz_store.cc:276] 0709 13:01:54.281883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.283903 (+  2020us) service_pool.cc:224] Handling call
0709 13:01:54.284254 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.288515 (+  4261us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:54.296693 (+  8178us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.548819 (+252126us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.724633 (+175814us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.724821 (+   188us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:54.724869 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.724966 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.734022 (+1009056us) raft_consensus.cc:1685] finished
0709 13:01:55.734076 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.734375 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13565,"prepare.queue_time_us":4671,"prepare.run_cpu_time_us":239,"prepare.run_wall_time_us":994,"raft.queue_time_us":16022,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":21167,"thread_start_us":164780,"threads_started":1,"wal-append.queue_time_us":463951}
I20260709 13:01:55.765492  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 891) took 1262 ms. Trace:
I20260709 13:01:55.766000  2291 rpcz_store.cc:276] 0709 13:01:54.502826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.405910 (+903084us) service_pool.cc:224] Handling call
0709 13:01:55.432292 (+ 26382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.434520 (+  2228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.765068 (+330548us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54004c1650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.765278 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59791232}
I20260709 13:01:55.767230  1556 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:55.745438  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1464) took 1522 ms. Trace:
I20260709 13:01:55.737831  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2145) took 1496 ms. Trace:
W20260709 13:01:55.768774  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1496) took 1416 ms (client timeout 1556 ms). Trace:
I20260709 13:01:55.768919  1551 rpcz_store.cc:276] 0709 13:01:54.241261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.257014 (+ 15753us) service_pool.cc:224] Handling call
0709 13:01:54.257391 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.366429 (+1109038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.737246 (+370817us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540036e7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.737645 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470415744}
W20260709 13:01:55.769282  1556 rpcz_store.cc:269] 0709 13:01:54.352486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.352622 (+   136us) service_pool.cc:224] Handling call
0709 13:01:55.768491 (+1415869us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540036ecd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.768692 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2846994304}
I20260709 13:01:55.598063  2294 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:55.768075  1706 rpcz_store.cc:276] 0709 13:01:54.223192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.491578 (+268386us) service_pool.cc:224] Handling call
0709 13:01:54.491876 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.492250 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.492278 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.504188 (+ 11910us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.938329 (+434141us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:54.938397 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.938493 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.744797 (+806304us) raft_consensus.cc:1685] finished
0709 13:01:55.744835 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.745140 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":305,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":433768,"threads_started":1,"wal-append.queue_time_us":544272}
W20260709 13:01:55.759677  2293 rpcz_store.cc:269] 0709 13:01:54.077087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.077230 (+   143us) service_pool.cc:224] Handling call
0709 13:01:55.758597 (+1681367us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400350054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.758872 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1196447104}
I20260709 13:01:55.762387  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:55.760603  1629 rpcz_store.cc:276] 0709 13:01:54.509016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.509146 (+   130us) service_pool.cc:224] Handling call
0709 13:01:54.509473 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.547330 (+ 37857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.547366 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.553854 (+  6488us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.934449 (+380595us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:55.126729 (+192280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.252390 (+125661us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b48001bc258. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:01:55.759345 (+506955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.759501 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.759581 (+    80us) raft_consensus.cc:1685] finished
0709 13:01:55.759681 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.759919 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66267,"prepare.queue_time_us":505329,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":279851,"raft.run_cpu_time_us":8790,"raft.run_wall_time_us":736390,"spinlock_wait_cycles":14721792,"thread_start_us":1036160,"threads_started":3,"wal-append.queue_time_us":436557}
I20260709 13:01:55.748205  1559 rpcz_store.cc:276] 0709 13:01:54.353640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.767641 (+414001us) service_pool.cc:224] Handling call
0709 13:01:54.768049 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.845079 (+ 77030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.845114 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.845604 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.086255 (+240651us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:55.086301 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.735532 (+649231us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b5400128ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.735637 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.735736 (+    99us) raft_consensus.cc:1685] finished
0709 13:01:55.735758 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.736319 (+   561us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":908,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":176971904,"thread_start_us":239975,"threads_started":1,"wal-append.queue_time_us":552883}
I20260709 13:01:55.780396  1634 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:55.780836  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.782368  2292 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:55.782791  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2317) took 1031 ms. Trace:
I20260709 13:01:55.783429  1673 leader_election.cc:400] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5.
I20260709 13:01:55.739024  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1499) took 1385 ms. Trace:
I20260709 13:01:55.783656  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:55.785640  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:55.784047  1554 rpcz_store.cc:276] 0709 13:01:54.353383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.705879 (+352496us) service_pool.cc:224] Handling call
0709 13:01:54.756539 (+ 50660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.761051 (+  4512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.761102 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.761784 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.026567 (+264783us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:55.026608 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.026673 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.738435 (+711762us) raft_consensus.cc:1685] finished
0709 13:01:55.738482 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.738805 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4059,"raft.queue_time_us":5963,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":264446,"threads_started":1,"wal-append.queue_time_us":473979}
W20260709 13:01:55.744501  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 326273us
I20260709 13:01:55.786695  2291 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:55.783918  1673 leader_election.cc:403] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.657455  4655 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:55.757592  1703 rpcz_store.cc:276] 0709 13:01:54.061976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.474063 (+412087us) service_pool.cc:224] Handling call
0709 13:01:54.474412 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.501320 (+ 26908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.501354 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.506399 (+  5045us) log.cc:844] Serialized 52 byte log entry
0709 13:01:54.917577 (+411178us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.923871 (+  6294us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.985273 (+ 61402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.985359 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.289875 (+304516us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b1800416d40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:55.756367 (+466492us) raft_consensus.cc:1685] finished
0709 13:01:55.756545 (+   178us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.756861 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86018,"prepare.queue_time_us":10758,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":96410,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":32597,"spinlock_wait_cycles":13266304,"thread_start_us":342103,"threads_started":1,"wal-append.queue_time_us":731196}
I20260709 13:01:55.783354  1634 rpcz_store.cc:276] 0709 13:01:54.751356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.840894 (+ 89538us) service_pool.cc:224] Handling call
0709 13:01:54.841305 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.781070 (+939765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.782350 (+  1280us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54002756d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:55.782697 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092423040}
I20260709 13:01:55.801460  4655 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:55.845829  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:55.846086  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110332us
I20260709 13:01:55.846906  1629 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:55.857641  4768 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.858525  4778 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:55.687568  4739 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.864552  4736 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:55.865650  4736 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.868011  4785 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:55.868577  4785 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.868988  4785 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.872187  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:55.872817  4769 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.873750  4769 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.874480  4117 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:55.877139  4730 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.878412  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:55.878577  4774 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.879920  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:55.881971  4797 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.882858  4649 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:55.883695  4649 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:55.883183  4797 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:55.884387  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.003s	sys 0.013s
W20260709 13:01:55.884864  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.003s	sys 0.013s
I20260709 13:01:55.885401  4649 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.887226  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:55.888394  1554 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:55.888286  4834 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:55.888996  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:55.889933  4819 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 4. Current term is 5.
I20260709 13:01:55.889130  4834 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:55.891778  4834 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.892218  4700 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:55.893116  4640 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.895948  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:55.878082  4730 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.858728  4768 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:55.882678  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:55.917562  4847 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.937790  4847 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:55.938165  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:55.939352  1700 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:55.940681  4847 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:55.941870  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:55.943989  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.945122  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.957108  1704 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:55.943877  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:55.947190  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 201020us
W20260709 13:01:55.965655  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83208.8us
I20260709 13:01:55.944399  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:55.997272  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:56.001214  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76006us
W20260709 13:01:55.970448  1626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.005s	sys 0.016s
I20260709 13:01:55.994462  4739 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:01:55.945053  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.008199  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:56.012773  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.013638  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.014730  1707 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:56.014935  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.015722  1631 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:56.016103  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.016774  4770 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:56.017195  4770 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:56.018476  4770 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.019073  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.020247  1708 raft_consensus.cc:2410] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:01:56.021155  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.022281  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.022922  1708 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:55.996932  4777 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.007189  4730 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:56.009207  1703 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:56.009804  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.010149  4769 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:55.940732  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:56.011355  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.011355  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:55.944490  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 640) took 1469 ms (client timeout 1582 ms). Trace:
I20260709 13:01:56.023732  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.018501  1709 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [6.3-8.4]
I20260709 13:01:56.016774  1707 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:55.955731  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1511) took 1594 ms. Trace:
I20260709 13:01:55.966332  4656 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:56.044927  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104559us
I20260709 13:01:56.045112  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:55.944096  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.042310  1630 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.050273  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.051219  1704 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:56.052173  4642 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.952632  4778 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:56.056321  2262 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.596s (SENT) [suppressed 46 similar messages]
I20260709 13:01:56.056454  4778 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:56.042608  1626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.009s	sys 0.016s
I20260709 13:01:56.042196  1552 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:01:56.059818  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 284312us
I20260709 13:01:55.961131  4736 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:56.039155  2295 rpcz_store.cc:269] 0709 13:01:54.474511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.772009 (+297498us) service_pool.cc:224] Handling call
0709 13:01:55.937446 (+1165437us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:55.944328 (+  6882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451311872}
I20260709 13:01:56.026350  4777 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.062860  1701 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:55.995944  4774 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.066413  1703 raft_consensus.cc:2368] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
W20260709 13:01:56.066565  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.024498  4610 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:56.062855  4847 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:56.042465  4770 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.951407  4819 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.084128  4819 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.086386  4819 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.046250  4656 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.073076  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.073523  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.074128  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.040273  1557 rpcz_store.cc:276] 0709 13:01:54.361170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.321274 (+960104us) service_pool.cc:224] Handling call
0709 13:01:55.321574 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.321936 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:55.731320 (+409384us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.913490 (+182170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.944458 (+ 30968us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.944647 (+   189us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:55.944687 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.944806 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.955293 (+ 10487us) raft_consensus.cc:1685] finished
0709 13:01:55.955319 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.955522 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59264,"prepare.queue_time_us":335842,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":8522,"raft.queue_time_us":66054,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":1102,"thread_start_us":512873,"threads_started":2,"wal-append.queue_time_us":99707}
I20260709 13:01:56.074972  1702 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:56.090843  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:01:56.076180  4456 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.780s	user 0.012s	sys 0.003s
I20260709 13:01:56.078051  4778 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.079454  4789 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:56.080001  4768 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.080475  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.100876  1559 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:55.969290  4828 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:01:56.087549  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.094719  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.102308  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.107290  1703 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:56.107400  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:01:56.091570  4656 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.117341  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:56.117218  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.118914  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.120456  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.121989  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:56.116215  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.123412  1555 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.123517  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.124262  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:56.125895  1709 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:56.126991  4655 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.112031  4797 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:56.128412  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58465.1us
I20260709 13:01:56.166108  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:56.167579  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.128393  4785 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.130100  4653 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:56.130748  1708 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:56.136096  1707 raft_consensus.cc:2368] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
W20260709 13:01:56.138633  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.138109  4840 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:01:56.140153  4740 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:56.140908  4780 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:56.142676  4847 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:56.143311  1554 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:01:56.338516  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.168040  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:56.305631  1703 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:56.305682  1634 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:56.280297  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:56.177048  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55565.9us
I20260709 13:01:56.310895  1629 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:56.174630  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58023.4us
W20260709 13:01:56.330715  4854 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.323s	user 0.001s	sys 0.003s
W20260709 13:01:56.364606  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 196744us
I20260709 13:01:56.328044  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.339291  4740 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:56.366705  4796 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.367072  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:56.295364  4456 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.367599  4740 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.368095  4456 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.369153  4740 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:56.369251  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 281862us
I20260709 13:01:56.370826  4610 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:56.287706  4797 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:56.374058  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1509) took 2013 ms (client timeout 1672 ms). Trace:
W20260709 13:01:56.374603  1552 rpcz_store.cc:269] 0709 13:01:54.360622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.487247 (+126625us) service_pool.cc:224] Handling call
0709 13:01:56.373811 (+1886564us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:56.373957 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297818880}
I20260709 13:01:56.375559  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:56.376314  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1505) took 2021 ms. Trace:
W20260709 13:01:56.376909  4858 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:01:56.376772  1560 rpcz_store.cc:276] 0709 13:01:54.359391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.280090 (+920699us) service_pool.cc:224] Handling call
0709 13:01:55.280514 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.736146 (+455632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.736234 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.737348 (+  1114us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.829764 (+ 92416us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:55.829818 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.843435 (+ 13617us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.329743 (+486308us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:56.375835 (+ 46092us) raft_consensus.cc:1685] finished
0709 13:01:56.375858 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.376086 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12901,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":83292544,"thread_start_us":278507,"threads_started":2,"wal-append.queue_time_us":169823}
I20260709 13:01:56.378499  4782 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:56.379825  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.394235  4610 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:56.395077  4780 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:56.400694  4780 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.399266  4751 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:56.338438  4653 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:01:56.402544  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.460462  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.379415  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:56.412047  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:56.436350  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43872.4us
I20260709 13:01:56.195283  4852 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:01:56.472918  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101719us
I20260709 13:01:56.458197  1633 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:56.457964  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.433794  1557 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.459154  1634 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:56.464541  1702 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:56.458603  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.466468  1704 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:56.461352  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:56.477255  1555 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:56.188318  4686 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.495677  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.467702  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:56.458969  4797 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:56.500484  1630 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:01:56.500659  1631 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.498090  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:56.504752  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 371663us
I20260709 13:01:56.367506  4796 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.503562  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.511132  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:56.512382  1703 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:56.516594  1559 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:56.513598  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:56.528416  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.529389  1628 raft_consensus.cc:2368] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
W20260709 13:01:56.529685  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 150384us
I20260709 13:01:56.517820  4769 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:56.532395  1635 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:56.533636  4610 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.535074  1557 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:01:56.517782  4730 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.536998  4730 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.538023  1557 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:56.539076  4730 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.539259  1557 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:56.512552  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.540450  1558 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.549232  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.550011  1560 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.550930  1701 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:56.482926  4819 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:56.556176  4847 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.559544  1553 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:56.563181  4868 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.232s	user 0.004s	sys 0.004s
I20260709 13:01:56.563930  1704 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:56.563211  1560 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:56.566177  4785 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.522549  1702 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:56.531524  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:01:56.514344  4782 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:56.591861  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:56.592579  1629 raft_consensus.cc:2368] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:56.593205  1558 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:56.585144  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:56.518119  4852 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.586872  4797 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.588207  1556 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:56.588709  4739 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:56.589686  4700 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:56.578298  4368 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.589720  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2316) took 2076 ms. Trace:
W20260709 13:01:56.591056  4555 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.768s	user 0.014s	sys 0.001s
I20260709 13:01:56.594444  1631 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:01:56.599398  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67956us
I20260709 13:01:56.607089  1555 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:56.611213  1559 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:56.613541  1552 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.513940  4686 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:56.604612  4642 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.004s
W20260709 13:01:56.607445  4754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.007s	sys 0.019s
I20260709 13:01:56.619637  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.620271  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.625801  1558 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:56.626278  1634 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
W20260709 13:01:56.620509  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 227901us
I20260709 13:01:56.602039  1626 rpcz_store.cc:276] 0709 13:01:54.513736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.825155 (+311419us) service_pool.cc:224] Handling call
0709 13:01:54.853795 (+ 28640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.022718 (+168923us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:56.059588 (+1036870us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.363590 (+304002us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.380376 (+ 16786us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.380530 (+   154us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:56.380562 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.380665 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.587704 (+207039us) raft_consensus.cc:1685] finished
0709 13:01:56.587723 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.589656 (+  1933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19199,"prepare.queue_time_us":546022,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":211322,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":42751,"thread_start_us":1408266,"threads_started":3,"wal-append.queue_time_us":453255}
I20260709 13:01:56.543658  4653 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:56.632143  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.640826  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 140290us
I20260709 13:01:56.629181  4653 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.626287  4789 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:56.627736  4754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.007s	sys 0.020s
I20260709 13:01:56.626510  4655 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:56.608022  4555 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.659124  1552 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:56.659616  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:56.660602  1629 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:56.662899  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:56.670631  4456 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:56.672585  1626 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:56.676213  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:56.648182  4789 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.679531  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:01:56.681120  4866 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:01:56.683792  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 175929us
I20260709 13:01:56.691531  1709 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:56.693082  1631 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:01:56.701375  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51241.4us
W20260709 13:01:56.706050  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.707309  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55551.9us
W20260709 13:01:56.684515  4834 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.019s
I20260709 13:01:56.645279  4368 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.688071  4789 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:56.691743  4456 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:56.711124  4656 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:01:56.718698  1558 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:56.718482  4834 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.009s	sys 0.019s
I20260709 13:01:56.712553  4782 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:56.728853  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:56.724180  1634 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:56.732605  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.724433  4456 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.740458  4456 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.743716  4819 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.744386  4819 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.746666  4819 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.749173  4368 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:56.751890  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:56.753288  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:56.755396  1523 leader_election.cc:400] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:56.772027  1523 leader_election.cc:403] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:01:56.750530  4874 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:01:56.776785  1555 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:56.778724  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.781277  1558 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:01:56.731630  4782 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.782171  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50588.3us
I20260709 13:01:56.731868  4796 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:56.787915  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62153.7us
I20260709 13:01:56.773509  4834 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:56.726397  4656 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.793020  1631 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:56.796952  1634 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:56.759246  4700 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:56.794694  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:56.798554  1556 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:56.787305  4697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.025s	sys 0.044s
I20260709 13:01:56.784375  4782 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.786911  4778 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.806351  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.808118  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88330.2us
I20260709 13:01:56.794060  4736 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.806010  4653 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:56.817711  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:56.803388  4697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.025s	sys 0.045s
I20260709 13:01:56.807814  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:56.815671  4778 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.804728  4852 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:56.821228  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:56.832082  4875 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:56.837634  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:56.838765  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.840730  1525 leader_election.cc:400] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:56.841081  1525 leader_election.cc:403] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:56.823714  4778 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:56.828110  4653 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:56.842952  4117 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.843510  4117 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.843729  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:56.843631  4653 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.830084  4740 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:56.846590  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:56.848340  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:56.848526  4754 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:56.849320  4873 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:01:56.851081  4841 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:56.851933  4751 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:56.862663  2290 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:56.863340  4751 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.863886  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.865145  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.866334  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:56.852548  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 391249us
I20260709 13:01:56.867091  2290 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:56.869470  4640 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:56.871273  4785 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:56.871706  4785 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:56.872701  4785 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.874104  4785 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.874644  4778 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:56.875366  4778 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:56.853322  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.876422  4778 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.878264  4117 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:56.879402  4861 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.687s	user 0.000s	sys 0.019s
I20260709 13:01:56.848387  4656 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:56.858846  2293 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.829349  4686 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:56.861049  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.861141  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:56.861961  2289 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:56.861171  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.854235  4766 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:56.886720  4766 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.887112  4766 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:56.880648  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49712.9us
I20260709 13:01:56.892149  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.892549  4796 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:56.893047  1525 leader_election.cc:400] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:56.893181  2290 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:56.894029  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:01:56.894878  4796 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:56.895685  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:56.899941  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.900571  4766 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.901800  2294 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:56.903513  4766 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:56.906627  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.841938  4828 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:56.907248  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.819382  4847 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:01:56.841535  4555 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:56.907727  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 106874us
I20260709 13:01:56.920318  1701 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:56.871163  4782 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:56.908449  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.861972  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:56.917021  4852 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:56.934952  4456 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:56.919855  4117 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 13:01:56.935990  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55613.4us
I20260709 13:01:56.936255  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.956573  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.936614  4686 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:56.936987  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:56.938380  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.858011  4740 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:56.940994  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:56.907680  4754 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:56.982336  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.982999  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:56.983222  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:56.881212  4841 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:56.966126  4890 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:56.966842  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71270.4us
W20260709 13:01:56.967550  4777 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.880s	user 0.009s	sys 0.009s
I20260709 13:01:56.938894  4736 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:56.970953  4117 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:56.972576  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:56.974990  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.975024  1707 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 5: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:56.988804  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 152617us
I20260709 13:01:56.974553  4740 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.981779  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.991588  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.991950  1631 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:56.984808  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.992133  1706 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:56.988179  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.984938  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.957211  1708 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:56.985848  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:56.993238  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56343.3us
I20260709 13:01:56.985524  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:56.995055  1700 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.995468  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.000000  1630 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:56.974566  4828 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.002305  1635 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:01:57.005475  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:56.975500  4686 consensus_queue.cc:1243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:01:57.019785  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49997.7us
I20260709 13:01:57.042377  1701 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:57.019932  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.020577  1634 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:57.021253  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:01:57.048871  1627 raft_consensus.cc:606] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:57.049391  1627 raft_consensus.cc:614] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:56.936609  4772 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:01:57.021816  1629 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:57.022279  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.023079  4754 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:56.985340  4640 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:56.991370  4117 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:57.026360  1708 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:56.999218  4768 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:56.986321  4890 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.029477  4653 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.984229  4796 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:57.032199  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:57.061476  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57635.6us
W20260709 13:01:57.061874  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 73078us
I20260709 13:01:57.062911  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.063195  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:56.956063  4697 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:01:56.986029  4766 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:01:57.079259  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.068950  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.095218  1708 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.066836  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.067090  2293 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:57.096735  1705 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:57.048672  4840 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:57.050984  4610 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:57.066911  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.063423  4782 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:57.081944  4778 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.088138  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.082122  4890 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:57.066975  4893 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:01:56.999557  4852 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:01:57.102586  1701 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:57.066954  4456 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:57.069411  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 180512us
I20260709 13:01:57.107808  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.108431  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.008809  4700 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.063858  4828 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.102880  4610 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:57.124475  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123965us
I20260709 13:01:57.126065  1634 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:57.099057  4768 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.091017  4736 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:01:57.111366  4841 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.106982  4653 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.170378  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:57.109113  4840 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:57.172111  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.265394  1635 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:57.153756  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.173107  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.258155  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.110980  4656 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:57.134538  4888 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:01:57.119454  4884 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:01:57.148490  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 890) took 2646 ms. Trace:
W20260709 13:01:57.252347  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72273.6us
I20260709 13:01:57.101907  4640 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.257929  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.259382  1700 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:57.254578  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:57.163019  4751 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:57.258854  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:57.312892  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58401.7us
I20260709 13:01:57.154966  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:57.257715  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.110834  4782 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:57.265779  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 463) took 1315 ms. Trace:
I20260709 13:01:57.317369  1706 rpcz_store.cc:276] 0709 13:01:55.937836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.944133 (+  6297us) service_pool.cc:224] Handling call
0709 13:01:57.191702 (+1247569us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400096d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:57.253577 (+ 61875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270726016}
W20260709 13:01:57.269161  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83723.7us
I20260709 13:01:57.113947  4739 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.164160  4896 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:57.279055  1631 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:57.144026  4730 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:57.297917  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.291832  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:57.301206  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 203609us
W20260709 13:01:57.304020  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 303983us
I20260709 13:01:57.276171  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:57.278192  1626 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:57.312561  1630 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:01:57.314374  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:57.152969  4686 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.315343  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:57.314051  1627 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:57.189082  4610 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.309849  2289 rpcz_store.cc:276] 0709 13:01:54.502538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.405263 (+902725us) service_pool.cc:224] Handling call
0709 13:01:55.417211 (+ 11948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.911915 (+1494704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.148161 (+236246us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54005186d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:57.148396 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3312011648}
I20260709 13:01:57.321830  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:57.321417  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 166522us
I20260709 13:01:57.321462  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:57.266630  4766 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.131650  4852 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.222s
I20260709 13:01:57.133424  4778 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.322566  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:57.158253  1454 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: config changed from index 4 to 5, term changed from 4 to 6, VOTER 8b2c3934407d422495728f84374c6b53 (127.0.107.194) evicted, no longer has a pending config: VOTER 444b494bb53842b59408a7230884398a (127.0.107.193), VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195), VOTER 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:57.354092  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.266876  4847 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:57.342626  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:57.450156  4847 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:57.356235  1629 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:57.311708  4656 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:57.354926  4368 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:57.344267  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:57.407753  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 93475.2us
I20260709 13:01:57.322857  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:57.132412  4897 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.275s
I20260709 13:01:57.171141  4789 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.164166  4117 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.416832  1633 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:57.416836  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:57.428020  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 112767us
I20260709 13:01:57.407182  4896 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:57.343410  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:57.418439  2288 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:57.406180  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.486274  1632 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:57.442477  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:57.416998  2291 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:57.152314  4796 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.315182  4754 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:57.450997  2290 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:57.407987  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 237703us
I20260709 13:01:57.376972  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.311399  4456 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.321236  4774 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:57.319172  4740 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.408033  4751 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:01:57.464320  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.496181  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.478273  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:57.497179  1558 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:57.453639  4852 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.409157  4730 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:57.441772  4797 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:57.501761  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:57.512972  1704 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:57.505417  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:57.519695  1635 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:57.504033  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:57.408565  1701 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:57.479548  1524 connection.cc:256] RPC call timeout handler was delayed by -1.1168005960000755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 13:01:57.534355  1524 connection.cc:256] RPC call timeout handler was delayed by -1.0721316060000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:01:57.508086  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:57.498076  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55714.2us
I20260709 13:01:57.498281  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:57.504935  2291 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:57.478963  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.536047  1626 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:57.483897  4368 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:57.540489  1629 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:57.441632  4686 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.431442  4766 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.457202  2296 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:57.534078  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:57.537031  1552 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:57.521170  4900 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:57.548408  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.486467  4736 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:57.471035  4895 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.535555  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.542326  2295 raft_consensus.cc:2368] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:57.491870  4774 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.557833  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.494449  4898 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:57.513430  4769 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:57.520406  4901 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:57.584022  4740 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:57.355042  4739 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.585391  4686 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:57.585940  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 90035.8us
I20260709 13:01:57.586227  4739 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:57.586622  4828 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:57.548717  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.587445  1559 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:57.551299  4819 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:57.588089  4828 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.588359  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.588481  4739 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:57.588613  4828 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.588945  4819 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:57.589056  4739 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.524386  4730 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.589476  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.555258  4891 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:57.590098  1559 raft_consensus.cc:2376] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8. Re-sending same reply.
I20260709 13:01:57.590281  4891 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:57.590703  4828 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:57.591030  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.553580  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:57.541392  4785 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.667s	user 0.014s	sys 0.000s
I20260709 13:01:57.590107  4730 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.592451  4653 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:57.589517  4819 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.595028  4819 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:57.555851  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.556524  1554 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:57.557258  1555 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:57.529615  4797 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:57.549543  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.546888  4655 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:57.567251  1628 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:01:57.524597  4840 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.576807  4768 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:57.519600  4894 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:57.577780  4769 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.578056  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.580132  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.581074  4777 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.581283  4368 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.498750  4456 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.503513  4656 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.591245  4891 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.512799  4782 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:57.555711  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.503687  4847 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.591594  1559 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9. Re-sending same reply.
I20260709 13:01:57.561214  4890 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.617871  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:57.618208  1556 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:57.637450  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:57.494603  4907 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.067s
W20260709 13:01:57.658731  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 346531us
W20260709 13:01:57.659660  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.565336  4774 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:57.664547  4819 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:57.668497  1628 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:57.678006  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132181us
I20260709 13:01:57.557201  4736 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:57.616678  4797 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.548296  4852 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.504330  4896 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.658530  4895 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.657907  4898 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:57.912626  1633 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:57.744529  1553 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:57.896082  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.793704  1551 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:57.928859  1552 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:01:57.922623  1708 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.816s	user 0.005s	sys 0.005s
I20260709 13:01:57.720798  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.929659  1708 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:57.886797  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:01:57.900954  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 507333us
I20260709 13:01:57.930891  1630 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:57.798899  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.837874  1626 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:57.693399  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.685712  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:57.815933  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.933923  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:57.937579  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.940855  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.670562  4655 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:57.668565  4909 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:57.677034  4897 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:57.956053  1552 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:57.956713  1557 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:57.944316  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.946272  1633 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:57.955647  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.865316  4917 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.960884  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.961563  1709 pending_rounds.cc:85] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 5
I20260709 13:01:57.958706  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.961859  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:57.715191  4847 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:57.836136  4796 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.967334  4905 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:01:57.968828  1701 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:57.969290  1559 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:57.714593  4744 raft_consensus.cc:2764] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:57.873169  4778 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:57.937237  4649 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.866451  4456 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:57.977314  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:57.873693  4789 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:57.972095  4768 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:01:57.979666  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.988903  1633 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:57.993656  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56616us
I20260709 13:01:57.782310  4766 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:57.972373  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:57.968330  1709 pending_rounds.cc:107] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:01:58.000618  1709 raft_consensus.cc:2889] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:57.991490  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:01:58.004434  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:58.006553  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.046330  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:58.085000  1707 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:57.979244  1556 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:01:58.087299  1628 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:58.085662  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:58.006054  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:57.769490  4769 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:57.682082  4777 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.088582  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:58.007874  4819 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:57.978127  4640 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.113029  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:58.122423  1633 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:58.001617  4909 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:58.124644  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:58.115948  1705 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:58.128693  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:58.133136  4740 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:58.134057  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.124624  4772 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:01:58.133651  4740 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:01:57.991685  4623 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:01:57.991864  4796 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:58.149904  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:58.116302  2296 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:57.988258  4895 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:58.129510  4610 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 13:01:57.956616  4780 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:01:58.120939  2293 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:58.114132  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 136466us
I20260709 13:01:57.980528  4649 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.130154  4789 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:58.148425  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.016s	sys 0.024s
I20260709 13:01:58.114814  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:58.130623  4754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.003s	sys 0.021s
I20260709 13:01:57.674257  4751 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.155947  1630 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:01:57.978791  4917 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:58.168267  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.016s	sys 0.024s
I20260709 13:01:58.131450  4897 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.141254  4778 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:58.229708  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.230073  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58539.3us
I20260709 13:01:58.171067  1628 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:58.179343  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:01:58.148449  4819 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:01:58.233485  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 474) took 1042 ms. Trace:
I20260709 13:01:58.234572  1704 rpcz_store.cc:276] 0709 13:01:57.191119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.258519 (+ 67400us) service_pool.cc:224] Handling call
0709 13:01:58.233270 (+974751us) spinlock_profiling.cc:243] Waited 89.6 ms on lock 0x7b540010e0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:58.233413 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206025600}
W20260709 13:01:58.236428  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.172408  4754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.003s	sys 0.021s
I20260709 13:01:58.167357  4610 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:58.272116  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 157388us
I20260709 13:01:58.170536  4789 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.276561  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:58.287002  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:58.232820  4456 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.233456  4778 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:58.169553  4736 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:58.170289  4895 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:01:58.167906  4907 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
W20260709 13:01:58.337826  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 209502us
W20260709 13:01:58.339790  1632 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.001s
I20260709 13:01:58.282934  4789 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:58.360320  1632 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:01:58.361907  1632 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:58.277910  4919 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.342235  1635 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:58.343753  2294 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:58.170917  4913 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.155s
I20260709 13:01:58.167316  4655 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:58.233265  4897 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:58.505445  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.037s	sys 0.027s
W20260709 13:01:58.496291  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.466713  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 179796us
W20260709 13:01:58.503991  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 235444us
I20260709 13:01:58.507432  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:01:58.505904  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.037s	sys 0.027s
W20260709 13:01:58.463570  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.506392  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1473) took 1535 ms. Trace:
I20260709 13:01:58.321734  4778 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:58.525621  1632 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:58.538609  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1485) took 1012 ms. Trace:
I20260709 13:01:58.539666  1708 raft_consensus.cc:606] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:01:58.540171  1708 raft_consensus.cc:614] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.434239  4736 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:58.467813  4780 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:58.539188  1706 rpcz_store.cc:276] 0709 13:01:57.526192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.526351 (+   159us) service_pool.cc:224] Handling call
0709 13:01:57.526737 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.526965 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:57.933845 (+406880us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.538211 (+604366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.538269 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:58.538314 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.538399 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.538533 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29574,"prepare.queue_time_us":35127,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":917539,"threads_started":2}
I20260709 13:01:58.466753  4623 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:58.516794  4919 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:58.566115  4915 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:01:58.543269  4768 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.567783  4768 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.572844  1523 leader_election.cc:400] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:58.573354  1523 leader_election.cc:403] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:01:58.574357  4906 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7.
I20260709 13:01:58.379302  4456 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.584831  1707 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:58.585902  4456 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.526345  4819 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.599347  4925 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:58.486712  4777 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.509759  1702 rpcz_store.cc:276] 0709 13:01:56.971073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.977191 (+  6118us) service_pool.cc:224] Handling call
0709 13:01:56.982127 (+  4936us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:57.065352 (+ 83225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.065403 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.119084 (+ 53681us) log.cc:844] Serialized 52 byte log entry
0709 13:01:57.472020 (+352936us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:57.472078 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.472163 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.506177 (+1034014us) raft_consensus.cc:1685] finished
0709 13:01:58.506199 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.506333 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47540,"raft.queue_time_us":56676,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":48292,"thread_start_us":203668,"threads_started":1,"wal-append.queue_time_us":547215}
I20260709 13:01:58.546916  4368 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:58.547974  4911 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
I20260709 13:01:58.550280  4740 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:58.552510  4924 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:01:58.513458  4898 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:58.558589  4923 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
I20260709 13:01:58.510854  4890 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.560948  4796 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:58.561475  4914 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:01:58.378646  4909 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:58.708909  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.788075  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1532) took 1209 ms. Trace:
I20260709 13:01:58.472114  4766 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.270s
I20260709 13:01:58.648061  4780 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:58.847172  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.847285  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 339939us
I20260709 13:01:58.849627  1559 rpcz_store.cc:276] 0709 13:01:57.578563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.015087 (+436524us) service_pool.cc:224] Handling call
0709 13:01:58.122575 (+107488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.129157 (+  6582us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:58.320133 (+190976us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.787547 (+467414us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.787632 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:58.787702 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.787822 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.787991 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20189,"raft.queue_time_us":101630,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":26854,"thread_start_us":363052,"threads_started":2}
W20260709 13:01:58.801136  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 222618us
I20260709 13:01:58.846927  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:58.851269  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 516522us
W20260709 13:01:58.850840  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.861804  4623 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:58.840895  4917 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:58.867969  4774 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.919s	user 0.011s	sys 0.001s
I20260709 13:01:58.868691  4774 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.873328  4919 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:58.886101  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:58.849232  4780 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:58.891222  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.887372  4937 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:58.883476  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.878862  4796 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:58.842895  4898 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:01:58.891214  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:58.917610  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.895743  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2342) took 1117 ms. Trace:
I20260709 13:01:58.878724  4774 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.939628  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:58.890828  4768 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:58.940388  2297 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:58.930850  4610 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:58.896625  4919 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:58.945745  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:58.942528  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:01:58.940899  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.005s	sys 0.016s
W20260709 13:01:58.943918  1556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.010s	sys 0.016s
W20260709 13:01:58.946638  1553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.019s
W20260709 13:01:58.950488  1553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.000s	sys 0.020s
W20260709 13:01:58.947191  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.884506  4906 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 13:01:58.949697  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.005s	sys 0.016s
W20260709 13:01:58.949705  1556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.010s	sys 0.016s
W20260709 13:01:58.951567  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.09841e+06us
W20260709 13:01:58.949942  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65928.9us
W20260709 13:01:58.947417  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61452.4us
I20260709 13:01:58.952440  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1528) took 1375 ms. Trace:
W20260709 13:01:58.957202  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.000s	sys 0.022s
W20260709 13:01:58.957587  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.000s	sys 0.022s
I20260709 13:01:58.960628  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.961604  1559 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:01:58.956303  1553 rpcz_store.cc:276] 0709 13:01:57.577428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.967006 (+389578us) service_pool.cc:224] Handling call
0709 13:01:57.967381 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.967655 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:58.320397 (+352742us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.952055 (+631658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.952111 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:58.952176 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.952250 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.952376 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1033,"prepare.queue_time_us":579235,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":886068,"threads_started":2}
I20260709 13:01:58.968122  4739 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:58.967689  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:58.968973  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:58.968894  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.969717  2293 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:01:58.970801  1553 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:58.953895  4898 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:58.979278  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:58.979575  4898 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:58.980172  4898 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:58.980310  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:58.981148  1553 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 5: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:58.982290  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:01:58.982587  4797 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.026s	user 0.001s	sys 0.010s
I20260709 13:01:58.983361  1553 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:58.899041  4937 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.984526  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.985152  1553 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:58.985813  4937 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:58.986234  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:58.987030  1553 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:58.955399  4368 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:01:58.983084  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2181) took 1495 ms. Trace:
I20260709 13:01:58.983075  4797 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.988505  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:01:58.946511  4907 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.034s	sys 0.036s
I20260709 13:01:58.951352  4919 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:01:58.995930  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.990391  1633 rpcz_store.cc:276] 0709 13:01:57.487815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.125629 (+637814us) service_pool.cc:224] Handling call
0709 13:01:58.141513 (+ 15884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.171144 (+ 29631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.171172 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.205545 (+ 34373us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.743454 (+537909us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:58.743512 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.743635 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.982843 (+239208us) raft_consensus.cc:1685] finished
0709 13:01:58.982872 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.983020 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31674,"raft.queue_time_us":27338,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":64221,"thread_start_us":454849,"threads_started":1,"wal-append.queue_time_us":687653}
I20260709 13:01:59.020249  1626 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:59.032915  2296 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:59.027872  4919 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:58.989915  4897 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:59.040544  2262 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.596s (SENT)
I20260709 13:01:59.042411  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:01:59.048445  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 96785.1us
W20260709 13:01:59.049933  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.056794  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2340) took 2064 ms (client timeout 1522 ms). Trace:
W20260709 13:01:59.057322  1631 rpcz_store.cc:269] 0709 13:01:56.977964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.022023 (+ 44059us) service_pool.cc:224] Handling call
0709 13:01:59.042399 (+2020376us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540019cb54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.042570 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506424192}
I20260709 13:01:59.059629  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.060491  1553 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:01:59.061743  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.062374  1553 raft_consensus.cc:2376] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7. Re-sending same reply.
I20260709 13:01:59.063268  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.064788  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.060567  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1524) took 1510 ms. Trace:
W20260709 13:01:59.072500  1552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.000s	sys 0.012s
W20260709 13:01:59.080648  1552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.001s	sys 0.012s
I20260709 13:01:59.080217  1560 rpcz_store.cc:276] 0709 13:01:57.550214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.550342 (+   128us) service_pool.cc:224] Handling call
0709 13:01:57.556737 (+  6395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.972784 (+416047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.972815 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.972869 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:57.972904 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.060331 (+1087427us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:59.060377 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.060506 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928672640}
I20260709 13:01:59.082051  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1533) took 1503 ms. Trace:
I20260709 13:01:59.081954  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.082965  1560 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:58.981119  4898 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.072069  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1535) took 1492 ms. Trace:
I20260709 13:01:59.056394  4555 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:59.089087  1553 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:59.089893  1703 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:59.098913  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.088667  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.100565  1702 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:59.082476  1552 rpcz_store.cc:276] 0709 13:01:57.578848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.082354 (+503506us) service_pool.cc:224] Handling call
0709 13:01:58.105131 (+ 22777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.118313 (+ 13182us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:58.319983 (+201670us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.081687 (+761704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.081749 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:59.081795 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.081859 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.081994 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":428819,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":145,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":900515,"threads_started":2}
I20260709 13:01:59.101639  1560 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:59.083689  4948 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:59.097044  4937 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.104861  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64350.6us
I20260709 13:01:59.105162  4937 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:01:59.105997  2262 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.630s (SENT)
I20260709 13:01:59.106397  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.087128  4947 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.107394  4947 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.109601  4947 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.072314  4768 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.110363  4768 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.110733  4919 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:59.107412  2262 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.661s (SENT)
I20260709 13:01:59.112536  4768 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.113499  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.114074  4797 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.042006  4700 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:01:59.115186  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.611s (SENT)
I20260709 13:01:59.115299  4700 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:59.115856  4700 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:59.117125  4700 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:01:59.087492  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 207707us
I20260709 13:01:59.118906  4700 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:59.087466  1556 rpcz_store.cc:276] 0709 13:01:57.579431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.171450 (+592019us) service_pool.cc:224] Handling call
0709 13:01:58.229237 (+ 57787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.229499 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:58.951793 (+722294us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.071349 (+119556us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.071425 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.071501 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.071601 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.071989 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":612315,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":254,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5670,"thread_start_us":792676,"threads_started":2}
I20260709 13:01:59.125552  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.126693  1559 raft_consensus.cc:2410] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:59.125571  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:01:59.128988  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161386us
I20260709 13:01:59.130378  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:01:59.136760  2263 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.823s (SENT)
W20260709 13:01:59.089306  4789 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.003s
I20260709 13:01:59.150866  1560 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:59.151116  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.151360  1560 pending_rounds.cc:85] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 6
W20260709 13:01:59.172089  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 231508us
I20260709 13:01:59.089174  4774 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.089282  4921 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.179085  2263 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.004s (SENT)
W20260709 13:01:59.177866  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60512.8us
I20260709 13:01:59.172415  1560 pending_rounds.cc:107] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:59.174723  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.180243  1560 raft_consensus.cc:2889] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.180583  1703 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:01:59.174971  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.179596  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.189759  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.189848  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.193495  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.190430  1709 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.195441  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.194367  1702 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:01:59.232841  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.172230  4555 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.179965  4655 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:59.229697  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44420.4us
I20260709 13:01:59.251210  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2168) took 1131 ms. Trace:
I20260709 13:01:59.251616  1560 rpcz_store.cc:276] 0709 13:01:58.120265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.122421 (+1002156us) service_pool.cc:224] Handling call
0709 13:01:59.150534 (+ 28113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.250951 (+100417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.251150 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.233785  1702 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:01:59.195036  4921 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.256637  4921 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.237380  4954 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:59.237818  4904 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.168694  4957 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.243131  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.252154  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:01:59.261950  4954 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:59.262533  4954 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:01:59.263478  4954 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.266726  4953 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.190721  1703 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
W20260709 13:01:59.244346  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63116.6us
W20260709 13:01:59.272405  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 100387us
W20260709 13:01:59.249982  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78398.8us
W20260709 13:01:59.273370  2263 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.111s (SENT)
I20260709 13:01:59.273725  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.234720  4919 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.181969  4930 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:59.260505  4117 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:59.269795  4959 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:01:59.270884  4945 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:01:59.193328  4847 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.139s	user 0.011s	sys 0.002s
I20260709 13:01:59.308880  1554 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:59.309700  4958 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.310611  4895 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.311165  4895 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.312739  4780 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.318485  4943 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:59.319952  4930 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:59.320808  4772 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:59.321241  4772 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:59.322063  4772 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.323050  4772 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.323952  2293 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:01:59.324240  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2184) took 1544 ms. Trace:
I20260709 13:01:59.324623  4740 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:59.325234  4740 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:01:59.326155  4740 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.273782  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1536) took 1694 ms. Trace:
I20260709 13:01:59.282081  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:59.287151  4778 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.742s	user 0.006s	sys 0.004s
W20260709 13:01:59.290164  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.290695  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.182909  4774 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.168982  4789 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.329612  1553 rpcz_store.cc:276] 0709 13:01:57.579698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.115371 (+1535673us) service_pool.cc:224] Handling call
0709 13:01:59.118608 (+  3237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.119815 (+  1207us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:59.148428 (+ 28613us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.273470 (+125042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.273519 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:59.273565 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.273626 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.273727 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32294,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":932,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":4880,"thread_start_us":124661,"threads_started":1,"wal-append.queue_time_us":179920}
W20260709 13:01:59.341965  1675 connection.cc:256] RPC call timeout handler was delayed by -1.2117699789999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3044s for a response
W20260709 13:01:59.340847  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.345348  1709 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:59.302770  4653 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.351058  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1824932549999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2371s for a response
W20260709 13:01:59.351603  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1284855419999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:01:59.352141  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:59.349232  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.324544  4623 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.353192  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.379244  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2161) took 1710 ms (client timeout 1527 ms). Trace:
I20260709 13:01:59.381563  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.324749  1628 rpcz_store.cc:276] 0709 13:01:57.779991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.237661 (+457670us) service_pool.cc:224] Handling call
0709 13:01:59.042690 (+805029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.086471 (+ 43781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.086540 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.108227 (+ 21687us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.190186 (+ 81959us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.190226 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.190287 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.314721 (+124434us) raft_consensus.cc:1685] finished
0709 13:01:59.314739 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.323960 (+  9221us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b540019cb50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.324169 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16441,"raft.queue_time_us":36787,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":21093,"spinlock_wait_cycles":1850417920,"thread_start_us":81535,"threads_started":1,"wal-append.queue_time_us":204774}
I20260709 13:01:59.379870  1552 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:59.390138  1557 rpcz_store.cc:269] 0709 13:01:57.668931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.679058 (+ 10127us) service_pool.cc:224] Handling call
0709 13:01:59.378996 (+1699938us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b54002bacd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.379147 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61024640}
I20260709 13:01:59.360193  1709 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:01:59.327389  4740 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:59.390353  1706 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:01:59.392517  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.394059  4623 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.395206  4780 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:01:59.396358  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.398883  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:59.401034  4789 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.402171  4913 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:59.402621  4913 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:01:59.403534  4913 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.404752  4913 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.408257  4774 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:59.412220  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 789) took 2146 ms (client timeout 2413 ms). Trace:
W20260709 13:01:59.412678  2294 rpcz_store.cc:269] 0709 13:01:57.265920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.267041 (+  1121us) service_pool.cc:224] Handling call
0709 13:01:59.411963 (+2144922us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b5400644bd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.412131 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025089920}
I20260709 13:01:59.400206  4730 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.482333  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 62177.7us
I20260709 13:01:59.483072  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:01:59.482394  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 130334us
I20260709 13:01:59.485566  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:01:59.486608  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.354326  4768 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.475986  1675 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.527s (SENT)
I20260709 13:01:59.354391  1700 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:59.492110  2262 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.611s (SENT)
I20260709 13:01:59.492585  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.353541  4796 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.493952  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.494212  2288 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:59.494632  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.495095  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:59.513542  2289 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:59.515194  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 479) took 2385 ms. Trace:
I20260709 13:01:59.371546  4653 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.515401  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:01:59.517241  4653 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.518596  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 484) took 1522 ms. Trace:
I20260709 13:01:59.518963  1707 rpcz_store.cc:276] 0709 13:01:57.996392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.205367 (+208975us) service_pool.cc:224] Handling call
0709 13:01:58.262298 (+ 56931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.585189 (+322891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.518379 (+933190us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400399c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:59.518540 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740969856}
I20260709 13:01:59.520121  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.521029  1707 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.484217  4774 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:59.521760  4774 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.497495  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.521839  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.522218  4774 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.522667  2290 raft_consensus.cc:2410] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:59.487960  4730 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.498108  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:59.524204  4949 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:59.515347  4961 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.099s	user 0.004s	sys 0.000s
I20260709 13:01:59.525416  4730 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:59.525840  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.526031  4730 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.526464  4730 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.526965  4730 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.527201  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.527751  4955 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:59.528455  4955 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.528508  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.528883  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 485) took 1395 ms. Trace:
I20260709 13:01:59.529663  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.529996  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.530345  1626 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:01:59.500063  4969 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:01:59.482537  4789 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.415629  4891 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.504882  4942 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:01:59.487738  4953 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.506811  4772 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.509483  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.355036  4921 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.391899  4655 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.520164  4973 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.347958  4852 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.497639  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.554811  1706 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.526396  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.531250  1633 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:59.527006  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.557953  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:01:59.382469  4847 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:59.495924  4754 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.521847  4653 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.560710  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.561218  4653 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.561997  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.562741  1631 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.566906  4649 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.568841  4754 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.569344  4754 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.550966  4942 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.571471  4751 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:59.573300  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1541) took 1634 ms. Trace:
I20260709 13:01:59.573936  4942 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.515676  1635 rpcz_store.cc:276] 0709 13:01:57.129746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.943190 (+813444us) service_pool.cc:224] Handling call
0709 13:01:57.943574 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.504479 (+560905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.514965 (+1010486us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54001428d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:59.515131 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":916214656}
I20260709 13:01:59.575100  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.575277  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.547778  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2345) took 1585 ms. Trace:
I20260709 13:01:59.548578  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.495487  4916 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:01:59.529237  1704 rpcz_store.cc:276] 0709 13:01:58.133672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.235516 (+101844us) service_pool.cc:224] Handling call
0709 13:01:58.278744 (+ 43228us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:58.503873 (+225129us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:58.938099 (+434226us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.150301 (+212202us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.150475 (+   174us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:59.156893 (+  6418us) log.cc:844] Serialized 52 byte log entry
0709 13:01:59.170410 (+ 13517us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.170680 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.171635 (+   955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.171724 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.414112 (+242388us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:01:59.507427 (+ 93315us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18003ea640. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:59.528682 (+ 21255us) raft_consensus.cc:1685] finished
0709 13:01:59.528704 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.528832 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4818,"prepare.queue_time_us":541275,"prepare.run_cpu_time_us":205,"prepare.run_wall_time_us":19471,"raft.queue_time_us":247046,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":65778,"spinlock_wait_cycles":80704128,"thread_start_us":566245,"threads_started":2,"wal-append.queue_time_us":416680}
I20260709 13:01:59.347144  4778 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.487533  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.582026  1560 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:59.551236  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.551928  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:01:59.578329  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2160) took 1963 ms (client timeout 2090 ms). Trace:
I20260709 13:01:59.571985  4653 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.499986  4768 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.573818  1552 rpcz_store.cc:276] 0709 13:01:57.939345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.121893 (+1182548us) service_pool.cc:224] Handling call
0709 13:01:59.379621 (+257728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.390635 (+ 11014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.573035 (+182400us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54002bacd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.573224 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590553984}
W20260709 13:01:59.583993  1554 rpcz_store.cc:269] 0709 13:01:57.614582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.673140 (+ 58558us) service_pool.cc:224] Handling call
0709 13:01:59.578065 (+1904925us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.578255 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3162114432}
W20260709 13:01:59.595952  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 120013us
I20260709 13:01:59.522648  1707 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.553920  4921 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.587028  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.582571  1632 rpcz_store.cc:276] 0709 13:01:57.962156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.526620 (+564464us) service_pool.cc:224] Handling call
0709 13:01:58.526987 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.527254 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.527291 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.241880 (+714589us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b2c00046b18. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:01:59.242358 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.389718 (+147360us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.389776 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.389861 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.536814 (+146953us) raft_consensus.cc:1685] finished
0709 13:01:59.536842 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.547693 (+ 10851us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741395840,"thread_start_us":146376,"threads_started":1,"wal-append.queue_time_us":241701}
I20260709 13:01:59.592218  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:01:59.601389  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.563511  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:01:59.597612  1675 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.561s (SENT)
I20260709 13:01:59.602010  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.602348  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:59.603911  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46037.7us
I20260709 13:01:59.604398  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:01:59.605057  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.605618  1633 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.550290  4789 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.607806  1709 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:59.610749  1559 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:59.611094  1559 pending_rounds.cc:85] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 6
I20260709 13:01:59.611318  1559 pending_rounds.cc:107] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:59.611548  1559 raft_consensus.cc:2889] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.581387  4895 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.587996  4796 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.619386  1704 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:59.577111  4623 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:01:59.601121  4949 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.624971  1707 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:59.637847  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.645175  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.598982  4847 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:59.617694  1675 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.592s (SENT)
I20260709 13:01:59.609843  4653 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:59.559372  4655 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.671689  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:01:59.672952  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55306.3us
I20260709 13:01:59.616691  1524 leader_election.cc:400] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8.
I20260709 13:01:59.652992  4921 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.654524  4768 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.606848  4766 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:01:59.664353  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1526) took 2111 ms. Trace:
I20260709 13:01:59.647342  4942 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:59.683568  1551 rpcz_store.cc:276] 0709 13:01:57.552981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.949794 (+396813us) service_pool.cc:224] Handling call
0709 13:01:57.950216 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.228447 (+278231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.228478 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.228533 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:58.228572 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.664103 (+1435531us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:59.664154 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.664297 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":480,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":29261,"spinlock_wait_cycles":588383104}
I20260709 13:01:59.664963  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:01:59.685048  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47280.7us
I20260709 13:01:59.664342  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.670404  1706 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:59.685866  1626 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:01:59.686451  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.687433  1706 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:01:59.676882  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.677772  1675 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.713s (SENT)
I20260709 13:01:59.688827  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.624043  4780 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:59.695648  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 82996.1us
I20260709 13:01:59.688113  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:01:59.609367  4898 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:01:59.689703  4766 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.692914  4974 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.665482  4958 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.697320  1553 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:59.695396  1626 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:59.699069  1631 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:59.682598  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.556658  4973 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.704659  1551 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:59.675467  4623 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:59.705524  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.680673  4916 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.675808  4949 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.708148  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.718207  1553 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:59.720782  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.711369  4768 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.722855  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.705703  1633 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:01:59.710638  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.675987  4653 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:01:59.559816  4953 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.711651  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.711396  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.705948  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.706658  4796 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.711584  4974 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.742319  1554 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:59.742631  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64901.2us
W20260709 13:01:59.746609  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.888s (SENT)
I20260709 13:01:59.750514  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.729101  4891 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.755133  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.752202  4852 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.714972  4958 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.757793  4852 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:58.922219  1630 rpcz_store.cc:276] 0709 13:01:57.778656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.227848 (+449192us) service_pool.cc:224] Handling call
0709 13:01:58.229752 (+  1904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.230029 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:58.742758 (+512729us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.895253 (+152495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.895329 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:58.895406 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.895501 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.895662 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2468,"raft.queue_time_us":2627,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4288,"thread_start_us":614843,"threads_started":2}
I20260709 13:01:59.759136  4898 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.711288  4610 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:01:59.761538  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.728775  4789 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.741863  4653 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:59.772917  1557 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:59.749153  4913 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:59.778066  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 474) took 2759 ms (client timeout 2433 ms). Trace:
W20260709 13:01:59.779979  1634 rpcz_store.cc:269] 0709 13:01:57.006602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.252005 (+245403us) service_pool.cc:224] Handling call
0709 13:01:59.765373 (+2513368us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54001419d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.765498 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481668864}
I20260709 13:01:59.749367  4974 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.786360  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:59.786438  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:59.720563  4916 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.783202  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.787542  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:59.788596  2288 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:59.765010  4904 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.789983  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.730422  4968 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:01:59.748073  4649 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:01:59.850191  2291 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:59.758605  4960 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:01:59.779004  4895 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:01:59.824510  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.761021  4955 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:59.759676  4898 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.860852  4916 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.862339  4898 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.862907  4953 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.864552  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.865262  1553 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:59.866443  4847 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.870167  4973 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.870477  4754 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.871668  1707 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.785131  4730 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.873472  4730 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.829977  4778 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:01:59.842217  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 311140us
I20260709 13:01:59.852836  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.882053  1555 raft_consensus.cc:2376] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10. Re-sending same reply.
I20260709 13:01:59.752261  4796 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.787919  4852 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:01:59.765316  4891 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.856524  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:01:59.857422  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.889700  2294 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:01:59.853940  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:01:59.890324  2288 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:01:59.890755  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70440.7us
I20260709 13:01:59.889839  2290 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:01:59.818380  4751 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.904350  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.905221  1555 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.892170  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.889086  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2170) took 1624 ms. Trace:
I20260709 13:01:59.867154  4847 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.907084  1556 rpcz_store.cc:276] 0709 13:01:58.264341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.148666 (+884325us) service_pool.cc:224] Handling call
0709 13:01:59.181863 (+ 33197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.233166 (+ 51303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.233210 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.254310 (+ 21100us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.603049 (+348739us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.618430 (+ 15381us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.636056 (+ 17626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.636138 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.664752 (+ 28614us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b18003fef80. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:01:59.880793 (+216041us) raft_consensus.cc:1685] finished
0709 13:01:59.888889 (+  8096us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.889029 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1721,"prepare.queue_time_us":20552,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":48,"raft.queue_time_us":29004,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":27747584,"thread_start_us":330967,"threads_started":1,"wal-append.queue_time_us":483342}
W20260709 13:01:59.890894  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 183744us
I20260709 13:01:59.912806  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 450) took 3126 ms. Trace:
W20260709 13:01:59.913295  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 182364us
I20260709 13:01:59.913278  1558 rpcz_store.cc:276] 0709 13:01:56.785049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.788526 (+  3477us) service_pool.cc:224] Handling call
0709 13:01:56.788857 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.497544 (+708687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.905939 (+2408395us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54002ba7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:01:59.906158 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628561792}
I20260709 13:01:59.856941  4740 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:01:59.920063  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.922220  1551 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:59.921029  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.932065  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.935308  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:01:59.937624  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.938390  1706 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:01:59.943199  1706 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:01:59.943879  1706 pending_rounds.cc:85] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 4
I20260709 13:01:59.944123  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.944168  1706 pending_rounds.cc:107] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:59.944772  1706 raft_consensus.cc:2889] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.946441  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:01:59.947472  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2175) took 1373 ms. Trace:
I20260709 13:01:59.947868  1560 rpcz_store.cc:276] 0709 13:01:58.574376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.252527 (+678151us) service_pool.cc:224] Handling call
0709 13:01:59.581589 (+329062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.582417 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.947274 (+364857us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000e8650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:01:59.947414 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710552448}
I20260709 13:01:59.948951  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.906497  4895 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:01:59.944793  1558 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:01:59.955996  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135830us
I20260709 13:01:59.957243  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:59.937129  4778 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.951086  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.923435  4774 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.966814  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.970731  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.972622  1558 raft_consensus.cc:2410] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:59.954066  4730 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:01:59.975142  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.975904  1706 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.980146  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.980846  1558 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:01:59.981029  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.981773  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:01:59.978029  4895 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.951987  4916 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:59.983349  1558 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:01:59.984680  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.986900  1706 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:01:59.931840  4975 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:01:59.939826  4740 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:01:59.988756  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.989512  1556 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:01:59.990175  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.990731  1558 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:01:59.991521  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:01:59.957365  4968 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:02:00.008704  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 57735.9us
I20260709 13:01:59.992746  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:01:59.992923  4847 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:01:59.998162  4951 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.749s	user 0.000s	sys 0.004s
I20260709 13:02:00.000573  4774 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.001192  4912 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:00.005060  4974 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.006455  4891 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:01:59.979648  4768 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.008091  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.011672  4841 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.012085  4744 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:00.013217  4754 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.014672  4796 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.016294  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2179) took 2899 ms. Trace:
I20260709 13:02:00.016469  4777 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:00.031065  1629 rpcz_store.cc:276] 0709 13:01:57.117303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.941802 (+824499us) service_pool.cc:224] Handling call
0709 13:01:57.963512 (+ 21710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.005454 (+ 41942us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:59.607199 (+1601745us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b1800410620. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:01:59.819304 (+212105us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.951443 (+132139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.986854 (+ 35411us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.987038 (+   184us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.987085 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.987182 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.010333 (+ 23151us) raft_consensus.cc:1685] finished
0709 13:02:00.016099 (+  5766us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.016230 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113132,"prepare.queue_time_us":847586,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2083,"raft.queue_time_us":143173,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":30400,"spinlock_wait_cycles":5469568,"thread_start_us":235230,"threads_started":2,"wal-append.queue_time_us":109463}
I20260709 13:02:00.009446  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:00.018428  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.019629  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:01:59.988879  4368 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:00.010237  4730 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.028463  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:01:59.998181  4960 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:00.033293  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:00.034027  1556 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:00.030586  4555 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:00.036978  4919 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:00.040153  4655 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.042869  4766 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:00.021518  4912 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:00.043982  4985 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.044661  4981 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.045259  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.228502  1555 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:00.046128  4984 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.046797  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.047258  4623 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:00.049459  4751 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:00.043588  4910 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:00.090459  4958 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:00.065409  4653 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.242586  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.073387  1627 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:00.100888  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 795) took 1129 ms. Trace:
I20260709 13:02:00.068723  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2171) took 1778 ms. Trace:
I20260709 13:02:00.243391  1559 rpcz_store.cc:276] 0709 13:01:58.290769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.127405 (+836636us) service_pool.cc:224] Handling call
0709 13:01:59.130158 (+  2753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.638512 (+508354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.068401 (+429889us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54003fb954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:00.068640 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1095195904}
I20260709 13:02:00.184705  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:00.245409  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.106618  2295 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:00.244321  2293 rpcz_store.cc:276] 0709 13:01:58.971679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.971820 (+   141us) service_pool.cc:224] Handling call
0709 13:02:00.100722 (+1128902us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:00.100829 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790945152}
W20260709 13:02:00.143674  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.064412  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:00.101260  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.247404  1560 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:00.188510  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.247933  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.248569  2296 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:00.057410  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.082237  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:00.099218  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47210 (request call id 26) took 2129 ms. Trace:
I20260709 13:02:00.062036  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:00.204493  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 152411us
I20260709 13:02:00.060209  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:00.108273  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 793) took 1155 ms. Trace:
W20260709 13:02:00.215725  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 182552us
I20260709 13:02:00.043567  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.115641  1454 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:00.223320  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:00.226851  1701 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:00.227617  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.231909  2290 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:00.109462  4686 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:00.054016  4974 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:00.057132  4740 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.238481  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.239470  1635 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:00.235046  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.290151  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.237799  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.093411  4769 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:00.290904  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.107290  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.291833  1635 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:00.292001  1553 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:00.235677  1706 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:00.236155  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.293015  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.293434  1628 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:00.245633  1558 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:00.078212  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.191102  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:00.105695  4940 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
W20260709 13:02:00.280747  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 482) took 2922 ms (client timeout 1630 ms). Trace:
W20260709 13:02:00.296960  1627 rpcz_store.cc:269] 0709 13:01:57.358329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.408551 (+ 50222us) service_pool.cc:224] Handling call
0709 13:02:00.280579 (+2872028us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b54001e8554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:00.280685 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5939468544}
I20260709 13:02:00.281213  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.298424  1629 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:00.297394  1705 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:00.280478  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.283694  1554 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:00.284124  1557 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.318764  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:00.319609  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 237448us
I20260709 13:02:00.320580  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:00.285650  1702 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.231596  4739 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:00.287815  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.287827  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.290426  1556 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:00.323060  2294 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:00.235352  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.324223  1555 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:00.325769  1555 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:00.326099  1555 pending_rounds.cc:85] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:02:00.326339  1555 pending_rounds.cc:107] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:00.326597  1555 raft_consensus.cc:2889] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.107758  4368 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:00.294731  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 251238us
I20260709 13:02:00.332056  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.279824  4954 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.249708  1708 rpcz_store.cc:276] 0709 13:01:57.969969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.970102 (+   133us) service_pool.cc:224] Handling call
0709 13:02:00.098954 (+2128852us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9de853 00007f70d39ac60a 00007f70ccf327d4 00007f70ccf32762 00007f70ccf32691 00007f70ccf3260c 00007f70ccf318c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:00.099088 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238197120}
I20260709 13:02:00.284317  2297 rpcz_store.cc:276] 0709 13:01:58.953293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.953441 (+   148us) service_pool.cc:224] Handling call
0709 13:01:58.965729 (+ 12288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.988180 (+ 22451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.988227 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.988290 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:58.988347 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.108033 (+1119686us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:00.108087 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.108220 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51000704}
W20260709 13:02:00.333159  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 142160us
I20260709 13:02:00.239004  4896 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.234227  4623 consensus_queue.cc:1243] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:02:00.281919  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 217596us
I20260709 13:02:00.339120  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:00.299093  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.300697  4730 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:00.296478  4819 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:00.240814  4910 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:00.300570  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.350075  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.350618  1559 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:02:00.301997  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 255262us
I20260709 13:02:00.220261  4555 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.234866  4937 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:00.317145  1626 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:02:00.317237  4740 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.326927  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:00.327962  2289 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:00.334684  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.355525  1557 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:00.280718  4891 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:02:00.363152  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.335831  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.281894  4686 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:00.297420  4975 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.234391  4985 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.321480  1454 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 5 to 6, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 6 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.350890  1554 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:00.356947  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:00.323484  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:00.385600  1556 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:00.339619  4754 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:00.362910  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.389503  2295 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:00.238695  4984 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.327371  4973 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.242918  4981 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.398489  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.399510  1626 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:00.367965  1557 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:00.368978  2289 pending_rounds.cc:85] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 6
I20260709 13:02:00.330571  4974 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:02:00.369601  1629 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:00.409183  1554 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:02:00.372735  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 327) took 19980 ms (client timeout 15000 ms). Trace:
I20260709 13:02:00.381316  4919 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:00.340456  4368 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:00.340631  4913 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.337003  4769 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:00.386961  4940 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:00.360122  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.494798  1554 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:00.342612  4766 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:00.347168  4958 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:00.497141  4766 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.497938  4766 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:00.498978  1554 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:00.394197  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.393689  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.400372  2289 pending_rounds.cc:107] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:00.280527  4912 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.413929  1551 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:00.416996  1556 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:00.345561  4778 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:00.425632  1633 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:00.347419  4772 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:02:00.364823  4986 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.387524  1705 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:00.425962  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:00.494047  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 159434us
I20260709 13:02:00.500620  2289 raft_consensus.cc:2889] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.505620  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.491583  1557 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:00.392884  4754 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.507452  4623 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:00.507911  4910 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.508976  4910 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.509160  4555 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:00.495055  4769 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:00.510121  4769 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.491973  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.386652  4975 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:00.357008  4954 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.497197  4958 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:00.279867  4960 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:00.500440  1634 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.514937  1629 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:00.493707  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.507385  1454 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 6 to 8, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 8 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:00.518251  2293 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:00.528775  2293 pending_rounds.cc:85] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 6
I20260709 13:02:00.529112  2293 pending_rounds.cc:107] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:02:00.529423  2293 raft_consensus.cc:2889] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.521009  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:00.522941  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 166102us
I20260709 13:02:00.530663  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:00.524561  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 130924us
I20260709 13:02:00.507392  4754 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.536552  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:00.539525  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.540418  2290 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:02:00.487464  4751 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:00.513974  4940 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:00.564749  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:00.564834  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 232157us
I20260709 13:02:00.558921  1454 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 5, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.565429  2291 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:00.566437  1454 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:00.563519  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.567691  2294 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:00.564172  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.568049  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.568883  2288 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:00.573452  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.574604  1626 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:00.588335  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:00.589500  2295 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:00.589673  2289 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:00.590436  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2245) took 1346 ms. Trace:
I20260709 13:02:00.549443  4948 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:00.415503  4973 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.561129  4768 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:00.593514  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 271930us
I20260709 13:02:00.553669  4852 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:00.607837  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.608505  1626 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:00.610008  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 79468.1us
I20260709 13:02:00.590952  1703 rpcz_store.cc:276] 0709 13:01:59.244316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.276194 (+ 31878us) service_pool.cc:224] Handling call
0709 13:02:00.013284 (+737090us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:00.013547 (+   263us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:00.014678 (+  1131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.014721 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:00.015481 (+   760us) log.cc:844] Serialized 52 byte log entry
0709 13:02:00.316578 (+301097us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:00.336083 (+ 19505us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.556893 (+220810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.556999 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.557060 (+    61us) raft_consensus.cc:1685] finished
0709 13:02:00.590108 (+ 33048us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.590348 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7860,"prepare.queue_time_us":20040,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3460,"raft.queue_time_us":23636,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":20583,"spinlock_wait_cycles":42787072,"thread_start_us":521165,"threads_started":2,"wal-append.queue_time_us":354409}
I20260709 13:02:00.610743  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:00.550659  4913 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.598443  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:00.611328  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.513293  4766 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.571385  4653 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:00.607498  2291 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:00.565004  4912 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:00.572782  4940 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.618580  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:00.624091  1704 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:00.630086  4610 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:00.630707  4610 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:00.631860  4610 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.633407  4610 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:00.642604  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.661455  1703 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:00.679625  4940 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.686368  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:00.689036  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:00.689689  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:00.689946  2288 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:00.697747  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 358723us
I20260709 13:02:00.698799  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:00.699995  4982 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:00.701462  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:00.702980  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:00.703152  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:00.704942  2294 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.706312  1634 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:00.607362  4975 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.708588  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 520124us
I20260709 13:02:00.714480  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.618315  4974 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:00.607357  4852 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:00.723101  1454 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 2 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.687318  4789 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:00.733927  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:00.734192  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.561281  4686 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.743714  1634 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:00.740099  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:00.766527  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:00.410008  1455 rpcz_store.cc:269] 0709 13:01:40.391859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.392097 (+   238us) service_pool.cc:224] Handling call
0709 13:01:41.573843 (+1181746us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b5400002210. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8c00b2 00007f70d4839a89 00007f70d47d1da6 00007f70d47a3088 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc
0709 13:02:00.365695 (+18791852us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:54.269322 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:54.779067 (+509745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:54.798558 (+ 19491us) write_op.cc:435] Acquired schema lock
0709 13:01:54.800666 (+  2108us) tablet.cc:662] Decoding operations
0709 13:01:54.849038 (+ 48372us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:54.849125 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:54.849168 (+    43us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:01:54.850476 (+  1308us) tablet.cc:701] Row locks acquired
0709 13:01:54.850498 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:01:54.850674 (+   176us) write_op.cc:270] Start()
0709 13:01:54.850854 (+   180us) write_op.cc:276] Timestamp: P: 1783602114850612 usec, L: 0
0709 13:01:54.850883 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:01:54.932352 (+ 81469us) log.cc:844] Serialized 4388 byte log entry
0709 13:01:55.653285 (+720933us) op_driver.cc:464] REPLICATION: finished
0709 13:01:55.945184 (+291899us) write_op.cc:301] APPLY: starting
0709 13:01:55.945352 (+   168us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:55.945715 (+   363us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:55.945867 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:55.947585 (+  1718us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:56.083827 (+136242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:01:56.083879 (+    52us) write_op.cc:312] APPLY: finished
0709 13:01:56.084426 (+   547us) log.cc:844] Serialized 83 byte log entry
0709 13:01:56.084632 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.084842 (+   210us) write_op.cc:454] Released schema lock
0709 13:01:56.084918 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:56.085477 (+   559us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:56.085515 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5134423,"reported_tablets":43,"spinlock_wait_cycles":156635520,"child_traces":[["op",{"apply.queue_time_us":291564,"apply.run_cpu_time_us":4765,"apply.run_wall_time_us":140274,"num_ops":11,"prepare.queue_time_us":77565,"prepare.run_cpu_time_us":36049,"prepare.run_wall_time_us":1107750,"raft.queue_time_us":129630,"raft.run_cpu_time_us":14986,"raft.run_wall_time_us":405191,"replication_time_us":802255,"thread_start_us":714924,"threads_started":4,"wal-append.queue_time_us":571400,"wal-append.run_cpu_time_us":29163,"wal-append.run_wall_time_us":1616499}]]}
W20260709 13:02:00.750483  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 213987us
I20260709 13:02:00.777475  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.771883  4852 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.836419  4896 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:00.770315  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:00.725881  4653 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:00.712898  4937 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:00.770135  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:00.820304  4994 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:02:00.825454  4973 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.824539  1454 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 5 to 7, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 7 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.824560  4955 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:00.827152  4940 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:00.827977  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.616740  4958 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.830180  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.830428  4954 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:00.831070  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.725368  4770 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:02:00.724932  4975 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:00.831933  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.832783  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.833654  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.834493  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:00.814896  4686 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.835410  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:00.836117  4907 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.838258  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:00.839555  1634 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:00.769973  4986 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:00.771632  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.725574  4974 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.857522  4925 raft_consensus.cc:2764] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:00.748689  4655 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
W20260709 13:02:01.208604  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.035s
W20260709 13:02:01.210387  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.003s	sys 0.035s
I20260709 13:02:01.210525  1635 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:01.211017  2297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.008s	sys 0.005s
W20260709 13:02:01.211439  2297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.008s	sys 0.005s
W20260709 13:02:00.885285  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115013us
I20260709 13:02:01.171870  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 459) took 1279 ms. Trace:
W20260709 13:02:00.866703  1702 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.000s
I20260709 13:02:01.052139  1705 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:02:01.202572  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 499442us
W20260709 13:02:00.903898  1700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.032s	sys 0.036s
I20260709 13:02:01.218461  1632 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:02:01.218962  1700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.035s	sys 0.044s
I20260709 13:02:01.057020  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1555) took 1017 ms. Trace:
I20260709 13:02:00.893025  2288 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:01.171660  2294 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:02:01.210925  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 440829us
I20260709 13:02:01.221530  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:01.213399  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.219487  1553 rpcz_store.cc:276] 0709 13:02:00.039862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.418546 (+378684us) service_pool.cc:224] Handling call
0709 13:02:00.418860 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.419077 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.419101 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.419653 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.750890 (+331237us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:00.750947 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.751049 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.056657 (+305608us) raft_consensus.cc:1685] finished
0709 13:02:01.056693 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.056932 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":214920,"threads_started":1,"wal-append.queue_time_us":526611}
I20260709 13:02:01.217938  1702 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:01.228245  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1509) took 1016 ms. Trace:
I20260709 13:02:01.228763  1702 rpcz_store.cc:276] 0709 13:02:00.211645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.223035 (+ 11390us) service_pool.cc:224] Handling call
0709 13:02:01.228164 (+1005129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.780542  5000 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:01.216647  1555 rpcz_store.cc:276] 0709 13:01:59.892631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.337680 (+445049us) service_pool.cc:224] Handling call
0709 13:02:00.338006 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:00.348812 (+ 10806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.348858 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:00.371005 (+ 22147us) log.cc:844] Serialized 52 byte log entry
0709 13:02:00.581472 (+210467us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:00.581514 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.581718 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.171569 (+589851us) raft_consensus.cc:1685] finished
0709 13:02:01.171592 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.171795 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35817,"raft.queue_time_us":11021,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":10407,"thread_start_us":210038,"threads_started":1,"wal-append.queue_time_us":535329}
W20260709 13:02:00.838418  4993 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.479s	user 0.001s	sys 0.000s
W20260709 13:02:01.232685  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.237978  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 399790us
W20260709 13:02:01.173079  4368 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.680s	user 0.007s	sys 0.002s
I20260709 13:02:01.232290  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:01.171408  4610 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.002s
I20260709 13:02:01.237141  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:00.855206  4777 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.794s	user 0.005s	sys 0.006s
W20260709 13:02:01.218041  4656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.003s	sys 0.009s
I20260709 13:02:00.908712  5003 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:01.259933  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.260255  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:01.102451  4947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.008s
I20260709 13:02:00.861639  4766 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:01.242413  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:01.237112  5000 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:01.297506  1630 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:00.860275  4968 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.405s
I20260709 13:02:00.859880  4998 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:00.837953  5014 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:01.164675  4999 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:00.852617  4896 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.849004  4913 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:01.279690  4947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.010s
I20260709 13:02:01.259388  4777 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:01.265931  4768 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:01.266189  4910 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:01.211678  4955 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:00.860008  4841 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.381s
W20260709 13:02:01.241137  4751 consensus_queue.cc:1175] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 9 } }
I20260709 13:02:01.264623  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.168939  4960 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:01.259186  4919 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:02:01.231312  4974 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:00.855755  4769 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:01.213064  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.867268  4948 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.398s
I20260709 13:02:01.234387  1629 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:02:01.043188  5008 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:02:01.251286  1628 raft_consensus.cc:2376] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:02:01.365087  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:01.352682  4910 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:01.349014  4740 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.016s	sys 0.024s
I20260709 13:02:01.365999  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:01.337745  4913 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:01.249640  5006 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.328431  1633 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:01.303426  1635 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:01.350063  4954 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.357508  5009 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.254655  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.250332  4904 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.387001  4769 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:01.332248  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 500) took 1797 ms. Trace:
I20260709 13:02:01.387717  4769 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:01.388244  4769 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:01.387943  1631 rpcz_store.cc:276] 0709 13:01:59.534590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.714314 (+179724us) service_pool.cc:224] Handling call
0709 13:01:59.723648 (+  9334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.924672 (+201024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.924718 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.930710 (+  5992us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.987997 (+ 57287us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:59.988040 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.988119 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.331993 (+1343874us) raft_consensus.cc:1685] finished
0709 13:02:01.332011 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.332165 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6223,"raft.queue_time_us":40592,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6198,"thread_start_us":209979,"threads_started":2,"wal-append.queue_time_us":278655}
I20260709 13:02:01.389371  4769 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.390805  1631 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:01.337831  4973 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:01.237197  4937 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:01.261660  4656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.003s	sys 0.009s
I20260709 13:02:01.249171  5002 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:01.271039  1454 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.231489  4770 consensus_queue.cc:1243] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:01.231061  4891 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:01.377934  4913 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.356384  4852 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:01.329126  4912 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.000s
I20260709 13:02:01.397286  5022 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:01.260551  5003 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:01.299918  4774 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.004s	sys 0.014s
I20260709 13:02:01.263948  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:01.480966  2288 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:02:01.357187  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.022s	sys 0.057s
W20260709 13:02:01.481760  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.022s	sys 0.057s
W20260709 13:02:01.482086  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 763176us
I20260709 13:02:01.265655  4982 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:02:01.328243  4909 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:01.490823  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:00.867543  4984 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:01.360538  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 96010us
I20260709 13:02:01.336403  4896 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:01.233212  4907 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.332703  5014 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:01.362437  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.277318  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:01.264950  4368 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.299892  5000 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:01.394955  4754 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.000s
W20260709 13:02:01.297811  4994 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:01.376811  4910 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:01.361378  4953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.005s	sys 0.036s
W20260709 13:02:01.406522  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.002s	sys 0.014s
W20260709 13:02:01.406919  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.000s	sys 0.015s
W20260709 13:02:01.407274  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.006s	sys 0.007s
W20260709 13:02:01.374028  4740 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.016s	sys 0.024s
W20260709 13:02:01.345767  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.019s
W20260709 13:02:01.336025  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.008s	sys 0.011s
I20260709 13:02:01.379926  4777 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.255286  5004 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:01.371814  4789 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:02:01.381474  4768 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:01.406777  4976 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.006s	sys 0.013s
I20260709 13:02:01.452657  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1507) took 1242 ms. Trace:
I20260709 13:02:01.509454  4847 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:01.405643  4916 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.000s	sys 0.016s
I20260709 13:02:01.432178  4891 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:01.405251  5009 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.457401  4937 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.458343  4904 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:01.397552  4955 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.393936  4954 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.384579  5006 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.459353  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1506) took 1250 ms. Trace:
I20260709 13:02:01.451519  5022 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:01.460073  5020 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.461715  1454 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:01.466229  4852 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:01.476831  4975 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:01.487939  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:01.597500  1703 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:01.493633  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.491524  1706 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:01.508955  2294 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:01.599260  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:01.512142  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.512774  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:01.602562  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108997us
I20260709 13:02:01.602097  1454 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 6, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:01.603873  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:01.591549  1707 rpcz_store.cc:276] 0709 13:02:00.209377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.209485 (+   108us) service_pool.cc:224] Handling call
0709 13:02:00.209859 (+   374us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:00.231844 (+ 21985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.231875 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:00.287939 (+ 56064us) log.cc:844] Serialized 78 byte log entry
0709 13:02:00.576766 (+288827us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:00.617742 (+ 40976us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.686345 (+ 68603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.686455 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.836370 (+149915us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180038a820. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:01.459011 (+622641us) raft_consensus.cc:1685] finished
0709 13:02:01.459114 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.459287 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173233,"prepare.queue_time_us":120031,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":124025,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":35176576,"thread_start_us":246271,"threads_started":1,"wal-append.queue_time_us":424559}
I20260709 13:02:01.605723  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:01.605741  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:01.607057  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:01.601882  1631 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:01.515189  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 149247us
I20260709 13:02:01.492617  4909 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:01.552251  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.000s	sys 0.015s
W20260709 13:02:01.552874  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.002s	sys 0.015s
W20260709 13:02:01.554029  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.007s	sys 0.008s
W20260709 13:02:01.555989  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.008s	sys 0.011s
W20260709 13:02:01.556334  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.000s	sys 0.019s
I20260709 13:02:01.561230  1630 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:01.510219  4649 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.552695  4953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.006s	sys 0.036s
I20260709 13:02:01.514609  4841 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.569231  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:01.565968  1709 rpcz_store.cc:276] 0709 13:02:00.210452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.210562 (+   110us) service_pool.cc:224] Handling call
0709 13:02:00.210871 (+   309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:00.222089 (+ 11218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.222137 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:00.242861 (+ 20724us) log.cc:844] Serialized 52 byte log entry
0709 13:02:00.564395 (+321534us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:00.564437 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.564507 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.452409 (+887902us) raft_consensus.cc:1685] finished
0709 13:02:01.452428 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.452586 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60638,"raft.queue_time_us":2094,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":15777,"thread_start_us":237008,"threads_started":1,"wal-append.queue_time_us":517315}
I20260709 13:02:01.510418  5014 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:01.574381  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.004s	sys 0.016s
W20260709 13:02:01.656680  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.004s	sys 0.016s
W20260709 13:02:01.565707  4976 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.007s	sys 0.014s
I20260709 13:02:01.574143  4891 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:01.572350  4916 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.001s	sys 0.017s
I20260709 13:02:01.570005  4768 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:01.551430  4754 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.597841  2289 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:01.480399  5003 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.600841  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:01.616271  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:01.712313  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111587us
I20260709 13:02:01.611966  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:01.612185  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 997477us
I20260709 13:02:01.715477  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:01.612368  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 907725us
W20260709 13:02:01.612551  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 844150us
W20260709 13:02:01.628540  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.572594  4847 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.588441  5019 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:01.658866  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.660781  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:01.663550  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 94415.8us
I20260709 13:02:01.610737  4955 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.638018  4909 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.635581  4904 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.637388  4895 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:01.609781  5036 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:01.641286  4852 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.634375  5002 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:01.712432  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 96377.4us
W20260709 13:02:01.767853  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.768018  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.717480  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:01.769446  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52040us
W20260709 13:02:01.613015  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.00226e+06us
I20260709 13:02:01.770756  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:01.725835  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:01.726864  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:01.729962  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 217518us
I20260709 13:02:01.664155  4942 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:01.745581  1701 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:01.706861  5021 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:01.750891  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:01.872951  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122206us
I20260709 13:02:01.873790  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.609575  5020 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.705183  4789 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.723045  4975 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:01.647231  4649 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.661124  4841 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.722527  5044 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:01.732365  4774 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.812219  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:01.803882  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.813097  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:01.813097  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:01.891832  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.728118  4686 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:01.786574  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59791.1us
I20260709 13:02:01.892724  4686 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.767409  5002 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.894807  4686 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:01.895021  5002 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.783274  5056 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.749977  4919 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:02:01.876277  4623 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.784765  4912 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:01.811928  4953 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.757682  4904 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.872443  5021 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.891603  1553 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.749130  4982 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:02:01.879644  4368 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:01.773480  4852 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.881676  4975 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.911422  4912 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:01.887593  4789 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.888145  4984 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:01.923421  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 155490us
I20260709 13:02:01.735893  5019 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:01.941897  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.873838  4942 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:01.948151  4942 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:01.956429  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:01.742626  5022 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.958567  5022 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.922340  5021 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:01.914690  4754 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:01.971275  5043 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:02:01.881019  4768 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.982754  5026 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:02:01.932911  4975 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:01.987305  1553 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:01.987947  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 174927us
I20260709 13:02:01.931159  4912 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:02.000371  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:01.937601  4984 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.911911  4623 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.907888  5056 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.943876  5019 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:01.971431  4954 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:01.969053  5064 raft_consensus.cc:2764] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:02:02.012017  1525 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.593s (SENT)
I20260709 13:02:01.918836  4953 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:01.991025  5009 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.015059  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.018474  5062 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:02.018739  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.604s (SENT) [suppressed 46 similar messages]
I20260709 13:02:02.006198  4942 raft_consensus.cc:2764] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:02.020581  5059 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.021180  5059 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.023037  5041 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.023586  5041 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.027051  5037 raft_consensus.cc:2764] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:02.027348  4981 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.027465  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.028110  4796 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.028198  2290 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:02.029358  5046 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:02:02.029129  4910 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.030011  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2259) took 1696 ms. Trace:
I20260709 13:02:02.030583  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 811) took 1337 ms. Trace:
I20260709 13:02:02.031075  2296 rpcz_store.cc:276] 0709 13:02:00.692784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.695362 (+  2578us) service_pool.cc:224] Handling call
0709 13:02:00.717914 (+ 22552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.739440 (+ 21526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.739473 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.768916 (+ 29443us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.657172 (+888256us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:01.657224 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.657302 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.030370 (+373068us) raft_consensus.cc:1685] finished
0709 13:02:02.030392 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.030526 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17391,"raft.queue_time_us":7052,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":26238,"thread_start_us":819727,"threads_started":1,"wal-append.queue_time_us":939546}
I20260709 13:02:02.030383  1701 rpcz_store.cc:276] 0709 13:02:00.333231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.560732 (+227501us) service_pool.cc:224] Handling call
0709 13:02:00.593366 (+ 32634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.772525 (+1179159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.029755 (+257230us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540016c954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:02.029952 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649195776}
I20260709 13:02:02.033691  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:01.991370  5006 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:02.036743  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:02.006865  4984 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:02.025884  5041 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.006723  4740 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:02.044564  5042 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 13:02:02.044953  4976 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:02.046116  4649 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:02.047911  5040 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:02:02.065294  5059 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.067086  5020 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.067710  5020 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:02.068111  5020 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:02.051695  4896 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.004s
I20260709 13:02:02.068442  5020 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:02.019338  5060 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:02.068907  4896 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.069783  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 914) took 1532 ms. Trace:
I20260709 13:02:02.070183  2297 rpcz_store.cc:276] 0709 13:02:00.537746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.554365 (+ 16619us) service_pool.cc:224] Handling call
0709 13:02:00.554826 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.211973 (+657147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.212002 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.232607 (+ 20605us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.502349 (+269742us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:01.502385 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.502453 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.069575 (+567122us) raft_consensus.cc:1685] finished
0709 13:02:02.069594 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.069729 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156580,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":890560,"threads_started":2,"wal-append.queue_time_us":348071}
I20260709 13:02:02.058246  4953 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.073347  4985 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.054824  5056 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.049067  4789 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.049536  4623 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.003953  5050 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:02.043701  5019 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.053551  4974 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.001s
W20260709 13:02:02.174336  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 94969.2us
I20260709 13:02:02.197160  4904 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.173923  4896 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:02.189656  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 248636us
I20260709 13:02:02.199966  4852 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.200753  4852 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.201282  4852 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.204066  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:02.205241  4913 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.753s	user 0.013s	sys 0.001s
I20260709 13:02:02.205637  4913 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:02.206178  5000 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.003s
I20260709 13:02:02.208289  2291 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:02.179170  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.203485  4852 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.212730  2296 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:02:02.206754  5000 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.213423  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.189647  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1556) took 2149 ms. Trace:
I20260709 13:02:02.215461  1708 pending_rounds.cc:85] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:02:02.264748  1708 pending_rounds.cc:107] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:02:02.265146  1708 raft_consensus.cc:2889] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.262825  1552 rpcz_store.cc:276] 0709 13:02:00.040143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.420456 (+380313us) service_pool.cc:224] Handling call
0709 13:02:00.420898 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:01.156101 (+735203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.156143 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:01.215030 (+ 58887us) log.cc:844] Serialized 52 byte log entry
0709 13:02:01.508462 (+293432us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:01.508500 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.509054 (+   554us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.077459 (+568405us) raft_consensus.cc:1685] finished
0709 13:02:02.077481 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.189570 (+112089us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":268058,"raft.queue_time_us":295874,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":13118,"thread_start_us":625229,"threads_started":2,"wal-append.queue_time_us":318264}
I20260709 13:02:02.262866  2293 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:02:02.270130  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.009s	sys 0.035s
I20260709 13:02:02.231666  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.271716  1707 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:02.271790  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.272675  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:02.058732  4847 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.270872  1701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:02:02.273681  1701 raft_consensus.cc:606] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:02.273600  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:02.274258  1701 raft_consensus.cc:614] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:02.275488  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:02.173875  4841 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:02.276407  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.010s	sys 0.035s
I20260709 13:02:02.276469  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.280663  1631 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:02.257203  4649 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.277753  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.288565  1552 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:02.257936  4913 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.285635  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.286937  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.214874  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.290333  1551 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:02.289736  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.290971  2297 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:02.290987  1702 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:02:02.052120  5014 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.298703  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.299319  1557 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:02.174903  4974 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.307188  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:02.283481  5050 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.308183  5034 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.299149  4841 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.310477  5016 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.846s	user 0.002s	sys 0.001s
W20260709 13:02:02.284731  4777 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.722s	user 0.003s	sys 0.011s
I20260709 13:02:02.281179  4852 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:02.293658  5003 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.001s
I20260709 13:02:02.315754  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:02.283502  4904 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:02.183375  5019 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.324640  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:02.324890  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.328989  1556 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 5.
I20260709 13:02:02.300359  5026 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.347719  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.296523  4649 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.301915  5044 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:02.348539  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.419325  1551 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:02.375141  1557 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:02.448043  1702 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:02.320330  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 422786us
W20260709 13:02:02.419854  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.008s	sys 0.005s
W20260709 13:02:02.456157  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.009s	sys 0.006s
W20260709 13:02:02.456825  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 583080us
I20260709 13:02:02.457615  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.459031  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.461496  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 461) took 2568 ms. Trace:
W20260709 13:02:02.420096  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 401349us
I20260709 13:02:02.462083  1559 rpcz_store.cc:276] 0709 13:01:59.893209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.367223 (+474014us) service_pool.cc:224] Handling call
0709 13:02:00.370519 (+  3296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.408942 (+ 38423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.408996 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.432059 (+ 23063us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.982912 (+550853us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.982972 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.983078 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.309777 (+1326699us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:02.461133 (+151356us) raft_consensus.cc:1685] finished
0709 13:02:02.461164 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.461418 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125640,"raft.queue_time_us":122593,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5359,"spinlock_wait_cycles":46103296,"thread_start_us":423383,"threads_started":1,"wal-append.queue_time_us":940592}
I20260709 13:02:02.468060  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:02.437906  4909 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 13:02:02.337606  4686 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.337705  4910 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:02.327862  5006 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.388792  4913 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:02.420713  4916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.005s	sys 0.007s
I20260709 13:02:02.335439  4777 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:02.448451  4955 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.704s	user 0.006s	sys 0.008s
W20260709 13:02:02.474308  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.001s	sys 0.014s
I20260709 13:02:02.466848   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:02.399641  5036 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.336542  4904 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.323266  5050 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.334923  4852 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.476707  5081 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.477442  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:02.466271  5002 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.005s
I20260709 13:02:02.482185  1552 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:02.482205  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.622018  1554 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:02.328244  4841 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.332788  5034 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:02.609736  4984 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.003s
W20260709 13:02:02.499995  5039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.025s	sys 0.078s
I20260709 13:02:02.500568  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.651074  1551 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:02.651698   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:02.652076  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.652490  5034 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.653144  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.653712  1551 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:02.654582  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.500590  4981 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:02.501869  4974 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.502485  5000 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.472945  4686 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.655757  4686 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:02.656265  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.618747  4913 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.661962  4913 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:02.557727  4768 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:02.556509  4751 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.612972  4904 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:02.557090  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.564436  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.556295  2290 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:02.558241  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 99308.6us
W20260709 13:02:02.609498  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.004s	sys 0.026s
W20260709 13:02:02.691931  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 921179us
I20260709 13:02:02.693288  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.611420  5083 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.472484  4937 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.399422  5026 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.498880  5006 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.634194  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.730403  1554 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:02:02.499802  4916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.005s	sys 0.009s
I20260709 13:02:02.643390  1703 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:02.470606  5021 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.338012  5003 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.649228  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.503268  5044 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.474992  4909 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.682399  1551 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:02.683465  5080 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:02:02.620908  4852 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.683815  5082 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.696643  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:02.704537  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 817) took 1135 ms. Trace:
I20260709 13:02:02.824663  5082 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:02.619802  5020 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.542s	user 0.014s	sys 0.007s
W20260709 13:02:02.825202  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 128648us
I20260709 13:02:02.824785  2292 rpcz_store.cc:276] 0709 13:02:01.568804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.568913 (+   109us) service_pool.cc:224] Handling call
0709 13:02:02.059587 (+490674us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:02.059817 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.060009 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:02.060288 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.703212 (+642924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.703262 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:02.703307 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.704348 (+  1041us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.704477 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106187,"prepare.queue_time_us":510,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":43,"raft.queue_time_us":315223,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":7622400,"thread_start_us":536265,"threads_started":2,"wal-append.queue_time_us":442573}
I20260709 13:02:02.826495  5066 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:02.634792  4976 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.827422  4976 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:02.709391  4891 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.981s	user 0.011s	sys 0.000s
I20260709 13:02:02.751363  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.840128  1554 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:02:02.841338  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.755931  1552 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:02.758948  1675 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:02.705523  4623 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:02.766513  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:02.846289  1559 raft_consensus.cc:2368] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:02.847100  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.728148  5079 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.640724  5002 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.772194  5074 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:02.731168  4937 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.755831  5014 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:02.785948  5084 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:02:02.787989  5026 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.705622  4768 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.678962  4984 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.789528  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2368) took 1570 ms. Trace:
I20260709 13:02:02.792385  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:02.799154  4841 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.800266  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2208) took 1942 ms. Trace:
I20260709 13:02:02.883162  1630 rpcz_store.cc:276] 0709 13:02:00.857562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.298778 (+441216us) service_pool.cc:224] Handling call
0709 13:02:01.337213 (+ 38435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.614036 (+276823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.799952 (+1185916us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002760d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:02.800186 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514522496}
I20260709 13:02:02.801982  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 513) took 1287 ms. Trace:
I20260709 13:02:02.884920  1631 rpcz_store.cc:276] 0709 13:02:01.514787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.725488 (+210701us) service_pool.cc:224] Handling call
0709 13:02:02.801767 (+1076279us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:02.801907 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1169683200}
I20260709 13:02:02.886518  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:02.804610  5053 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.771s	user 0.003s	sys 0.000s
I20260709 13:02:02.805256  4649 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.892982  1706 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:02:02.806041  5005 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.350s	user 0.002s	sys 0.001s
I20260709 13:02:02.807579  5021 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.895138  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1504) took 2687 ms. Trace:
I20260709 13:02:02.808959  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:02.896149  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87323.9us
I20260709 13:02:02.897671  1599 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:02.898984  4986 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.892758  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:02.899662  4986 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:02.899981  1709 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:02.810963  4913 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.811602  4772 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.902287  4913 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.902369  4772 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.904366  4913 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.904501  4772 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.678001  5081 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.904978  4981 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:02.905681  4981 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:02.906491  5081 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:02.906702  4912 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.001s
I20260709 13:02:02.908038  1709 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:02.908421  5085 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:02:02.850690  4770 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:02:02.819186  5072 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.819362  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.627395  5036 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.910620  1708 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
W20260709 13:02:02.820062  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 165329us
I20260709 13:02:02.907140  4912 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.749666  5083 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.825469  5020 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.731714  5077 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:02:02.737893  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 413294us
I20260709 13:02:02.828903  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:02.931082  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102281us
I20260709 13:02:02.931118  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:02.931950  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.827320  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:02.785948  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.932967  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:02.825178  5082 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:02.835991  1707 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.003s
W20260709 13:02:02.936503  1707 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.003s	sys 0.003s
I20260709 13:02:02.843477  1704 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:02.843021  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:02.882022  5071 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:02.847683  1551 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:02.866429  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107611us
I20260709 13:02:02.873413  1554 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:02.845808  4891 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.872465  4916 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:02.890053  4768 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.890228  4686 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.872133  5074 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:02.892037  4984 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.957579  4984 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:02.958789  4984 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:02.895751  4904 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.740120  4955 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.960265  4955 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:02.961633  4904 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.895553  1700 rpcz_store.cc:276] 0709 13:02:00.207644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.221203 (+ 13559us) service_pool.cc:224] Handling call
0709 13:02:00.226209 (+  5006us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:00.240840 (+ 14631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.240875 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:00.246825 (+  5950us) log.cc:844] Serialized 52 byte log entry
0709 13:02:01.219429 (+972604us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:01.219466 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.219536 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.805296 (+1585760us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:02.894794 (+ 89498us) raft_consensus.cc:1685] finished
0709 13:02:02.894819 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.895049 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8014,"raft.queue_time_us":14910,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":37684,"spinlock_wait_cycles":59940608,"thread_start_us":874429,"threads_started":1,"wal-append.queue_time_us":1088990}
I20260709 13:02:02.881153  1635 rpcz_store.cc:276] 0709 13:02:01.219214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.375703 (+156489us) service_pool.cc:224] Handling call
0709 13:02:01.376088 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:01.401005 (+ 24917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.401044 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:01.410538 (+  9494us) log.cc:844] Serialized 52 byte log entry
0709 13:02:02.282342 (+871804us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:02.282400 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.282490 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.789217 (+506727us) raft_consensus.cc:1685] finished
0709 13:02:02.789247 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.789456 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24495,"raft.queue_time_us":28815,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":9092,"thread_start_us":802170,"threads_started":1,"wal-append.queue_time_us":1322911}
W20260709 13:02:02.809583  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.920512  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.041s	sys 0.056s
W20260709 13:02:02.964092  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.041s	sys 0.057s
I20260709 13:02:02.921447  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:02.965701  1631 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:02.963820  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:02.967288  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.929140  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:02.933564  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:02.932305  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:02.967947  1708 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:02.968284  1630 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:02.939057  5081 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:02.937687  1552 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:02.842913  1553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.024s	sys 0.011s
W20260709 13:02:02.973065  1553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.024s	sys 0.012s
I20260709 13:02:02.880683  4368 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.981385  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.917090  5022 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:02.941788  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:02.931258  4974 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:02.987443  1630 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:02.926934  4954 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:02.932291  4649 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.965761  2297 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:02.936292  5066 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:02.934087  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106826us
I20260709 13:02:02.991353  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:02.968699  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:02.992231  2297 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:02.873553  5039 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.939795  4789 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.976119  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:02.978134  4891 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.979408  5020 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:02.980098  1557 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:02.980031  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.992142  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 919) took 1222 ms. Trace:
I20260709 13:02:03.014294  2294 rpcz_store.cc:276] 0709 13:02:01.769534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.769761 (+   227us) service_pool.cc:224] Handling call
0709 13:02:01.770104 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.783466 (+ 13362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.786734 (+  3268us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.806575 (+ 19841us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.295476 (+488901us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:02.295522 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.295598 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.991934 (+696336us) raft_consensus.cc:1685] finished
0709 13:02:02.991955 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.992086 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2770,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":20544,"thread_start_us":395271,"threads_started":1,"wal-append.queue_time_us":587683}
I20260709 13:02:02.993083  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:02.968935  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:02.989789  5026 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 13:02:02.994064  5086 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:02:02.994500  4780 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:02.994675  1631 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:03.022034  1708 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:03.022931  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:02.991178  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.017184  4649 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.024669  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:03.025200  1709 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:02.991217  4368 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:03.028097  5018 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.552s	user 0.002s	sys 0.001s
W20260709 13:02:03.028585  4975 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.016s	user 0.016s	sys 0.000s
I20260709 13:02:03.029001  4975 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.029732  4368 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:03.030534  4744 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:02.945641  5071 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.030941  4744 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:03.034061  4958 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:03.035300  1709 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:03.036996  5050 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:03.038532  4686 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:02.996112  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.013780  5039 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:03.039933  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.041036  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.041648  5039 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:02.996469  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54806.4us
W20260709 13:02:02.997314  5092 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:03.041657  1635 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:03.071401  1629 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:02.999603  5083 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:03.001281  4895 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:03.075601  4895 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.002501  4653 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:02.963732  5074 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:03.004946  5015 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.542s	user 0.004s	sys 0.000s
I20260709 13:02:02.923506  5072 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.007187  2296 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:03.009994  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:02.959456  4984 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:02.982641  4904 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:03.018537  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:03.042814  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:03.045593  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.049336  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:02.998077  5066 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.072180  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:02.987150  4974 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:03.005406  5036 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:03.005729  5081 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:03.118676  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 189618us
I20260709 13:02:03.006702  5022 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.016156  4916 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:03.318374  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.022s
I20260709 13:02:03.311022  5003 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.072543  5095 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:03.113574  4653 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:03.441860  5044 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.071905  4955 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.071811  4772 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.482182  2294 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:03.372851  4998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.981s	user 0.009s	sys 0.009s
I20260709 13:02:03.395061  5079 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:03.483023  4998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.117s	user 0.009s	sys 0.016s
W20260709 13:02:03.470340  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 260510us
I20260709 13:02:03.483331  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:03.483467  4998 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:03.484249  2294 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:03.484678  4998 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:03.426714  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 477) took 1143 ms. Trace:
I20260709 13:02:03.110004  5020 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.376s
W20260709 13:02:03.425814  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 491283us
I20260709 13:02:03.486178  1560 rpcz_store.cc:276] 0709 13:02:02.283071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.283326 (+   255us) service_pool.cc:224] Handling call
0709 13:02:02.298779 (+ 15453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.305043 (+  6264us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:02.305088 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.305668 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.696073 (+390405us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:02.696111 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.696195 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.426429 (+730234us) raft_consensus.cc:1685] finished
0709 13:02:03.426457 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.426639 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5870,"raft.queue_time_us":6752,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":2432,"thread_start_us":356784,"threads_started":1,"wal-append.queue_time_us":526822}
I20260709 13:02:03.073472  4766 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:03.488286  4766 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.481706  4904 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.488900  4891 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:03.489529  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:03.489678  4904 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.489995  4891 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:03.491529  4766 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:03.218436  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.488734  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:03.412904  4937 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:03.180428  4780 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:03.492419  4955 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:03.112322  5083 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.479151  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.001s	sys 0.036s
I20260709 13:02:03.494814  5083 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.492182  4891 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:03.478940  4916 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:03.449661  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.009s	sys 0.023s
W20260709 13:02:03.496626  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.011s	sys 0.025s
I20260709 13:02:03.496997  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 476) took 1215 ms. Trace:
W20260709 13:02:03.129626  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 137589us
I20260709 13:02:03.499270  1555 rpcz_store.cc:276] 0709 13:02:02.281427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.281570 (+   143us) service_pool.cc:224] Handling call
0709 13:02:02.281841 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.282033 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:02.686931 (+404898us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.496530 (+809599us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.496681 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.496724 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:03.496765 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.496831 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.496942 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":627451,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":84,"thread_start_us":1184658,"threads_started":2,"wal-append.queue_time_us":776528}
W20260709 13:02:03.500774  1552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.013s
I20260709 13:02:03.485188  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:03.501266  1552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.009s	sys 0.013s
I20260709 13:02:03.501662  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:03.482321  4772 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.216897  5006 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:03.494804  4780 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.215703  5026 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:03.540858  5026 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:03.541993  5026 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.481487  4653 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:03.543035  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1528) took 1092 ms. Trace:
I20260709 13:02:03.544516  4653 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:03.495734  5112 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.496076  1630 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:02:03.547446  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.548802  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.549014  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.549731  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2258) took 3216 ms. Trace:
I20260709 13:02:03.550194  1634 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:03.550108  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.550880  1630 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:03.551191  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.481444  5044 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.480180  5095 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:03.488617  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:03.215958  4913 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:03.498117  1700 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:03.260201  4960 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:03.485548  5066 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:03.551736  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 684) took 2186 ms. Trace:
I20260709 13:02:03.555670  5114 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:03.287817  4909 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.556190  5114 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:03.555928  2291 rpcz_store.cc:276] 0709 13:02:01.365446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.365576 (+   130us) service_pool.cc:224] Handling call
0709 13:02:01.365903 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.208820 (+842917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.551440 (+1342620us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b5400350cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:03.551642 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1936649088}
I20260709 13:02:03.557639  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:03.557471  5114 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:03.513101  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 470338us
I20260709 13:02:03.558511  4909 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.558987  5114 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:03.559111  4909 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.517297  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:03.560177  5107 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 13:02:03.561290  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:03.561477  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1530) took 1090 ms. Trace:
I20260709 13:02:03.561971  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:02:03.562036  4909 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.562942  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:03.562665  5081 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:03.563438  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62744.3us
I20260709 13:02:03.525902  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:03.564757  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:03.566180  4766 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.566609  5003 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:03.566766  4766 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.567440  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 816) took 1999 ms. Trace:
I20260709 13:02:03.543738  2289 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:03.502297  4955 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:03.569792  4955 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:03.570935  5112 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:03.571116  4955 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:03.375345  5082 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:03.465026  4976 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.498862  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:03.558274  2291 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:03.542482  4772 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:03.561854  1706 rpcz_store.cc:276] 0709 13:02:02.470656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.894359 (+423703us) service_pool.cc:224] Handling call
0709 13:02:02.894671 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.894986 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.895014 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.895475 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.356196 (+460721us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.356258 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.356363 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.559560 (+203197us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:03.561186 (+  1626us) raft_consensus.cc:1685] finished
0709 13:02:03.561219 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.561417 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3082,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"spinlock_wait_cycles":216372992,"thread_start_us":460233,"threads_started":1,"wal-append.queue_time_us":568514}
I20260709 13:02:03.551908  1634 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:03.553682  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.566165  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:03.550220  1705 rpcz_store.cc:276] 0709 13:02:00.332962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.503813 (+170851us) service_pool.cc:224] Handling call
0709 13:02:00.504183 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.218331 (+714148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.549469 (+2331138us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54003e2454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:03.549665 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258841984}
I20260709 13:02:03.582144  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 516) took 1513 ms. Trace:
I20260709 13:02:03.575672  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:03.583200  1630 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:03.582567  1631 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:03.554181  4960 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:03.584128  4960 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.584554  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:03.585378  5099 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:03.130380  4789 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.469s
I20260709 13:02:03.500837  5083 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.498886  4916 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:03.553992  4913 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.585346  1631 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:03.567930  2288 rpcz_store.cc:276] 0709 13:02:01.568504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.568654 (+   150us) service_pool.cc:224] Handling call
0709 13:02:03.566673 (+1998019us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:03.566890 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.567073 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.567100 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.567145 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.567194 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.567263 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.567382 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4973952}
I20260709 13:02:03.592397  1634 rpcz_store.cc:276] 0709 13:02:02.069094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.550946 (+1481852us) service_pool.cc:224] Handling call
0709 13:02:03.582074 (+ 31128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:03.544713  5096 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:02:03.544852  1702 rpcz_store.cc:276] 0709 13:02:02.450419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.465319 (+ 14900us) service_pool.cc:224] Handling call
0709 13:02:02.477500 (+ 12181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.477712 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:02.846061 (+368349us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.541126 (+695065us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.541779 (+   653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.541934 (+   155us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:03.542121 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.542379 (+   258us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.542784 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3284,"prepare.queue_time_us":464151,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":5091,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":166,"thread_start_us":661553,"threads_started":3,"wal-append.queue_time_us":596417}
I20260709 13:02:03.599902  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:03.563526  1555 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:03.570278  4909 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:03.543462  5026 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.264259  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:03.622805  2291 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:03.707729  1555 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:04.265318  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 927) took 1271 ms. Trace:
W20260709 13:02:04.244743  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.937975  5020 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:04.265904  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.266515  2291 rpcz_store.cc:276] 0709 13:02:02.993905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.557368 (+563463us) service_pool.cc:224] Handling call
0709 13:02:04.265261 (+707893us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:03.704614  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 205816us
I20260709 13:02:03.553843  5044 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:03.809767  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 687) took 1331 ms. Trace:
I20260709 13:02:03.801208  5014 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.251446  4769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.794s	user 0.007s	sys 0.052s
W20260709 13:02:04.255987  4916 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.668s	user 0.007s	sys 0.003s
I20260709 13:02:04.268492  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:03.707134  4744 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.684690  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:03.717167  5052 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.907s	user 0.000s	sys 0.003s
I20260709 13:02:03.713783  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1584) took 1060 ms. Trace:
I20260709 13:02:04.283723  1551 rpcz_store.cc:276] 0709 13:02:02.653120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.654377 (+  1257us) service_pool.cc:224] Handling call
0709 13:02:03.713563 (+1059186us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b54003098d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:03.713711 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60854144}
I20260709 13:02:04.277755  5083 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:03.713619  4913 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.286214  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:03.699808   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:03.706017  4974 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.285419  5083 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:03.791944  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.137562  2290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.009s	sys 0.009s
W20260709 13:02:04.291931  2290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.011s	sys 0.011s
I20260709 13:02:04.262861  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 828) took 1254 ms. Trace:
W20260709 13:02:04.245219  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 680486us
I20260709 13:02:04.263741  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1580) took 1759 ms. Trace:
I20260709 13:02:04.292356  2289 rpcz_store.cc:276] 0709 13:02:03.008342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.009285 (+   943us) service_pool.cc:224] Handling call
0709 13:02:04.262668 (+1253383us) spinlock_profiling.cc:243] Waited 63.9 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:04.262789 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146944896}
I20260709 13:02:04.293664  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.292866  1552 rpcz_store.cc:276] 0709 13:02:02.504099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.971935 (+467836us) service_pool.cc:224] Handling call
0709 13:02:02.972312 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.501669 (+529357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.501702 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.502189 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.541023 (+ 38834us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b4800002958. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:03.787626 (+246603us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.787693 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.787838 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.263440 (+475602us) raft_consensus.cc:1685] finished
0709 13:02:04.263459 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.263680 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85730,"raft.queue_time_us":98595,"raft.run_cpu_time_us":2246,"raft.run_wall_time_us":467082,"spinlock_wait_cycles":143529728,"thread_start_us":776149,"threads_started":2,"wal-append.queue_time_us":721818}
W20260709 13:02:03.555352  5103 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:02:04.294675  2289 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:03.717090  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1529) took 1251 ms. Trace:
I20260709 13:02:03.712939  1632 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:04.266907  1555 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:04.302490  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:03.836862  5082 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.270320  2296 rpcz_store.cc:276] 0709 13:02:02.478185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.478301 (+   116us) service_pool.cc:224] Handling call
0709 13:02:03.809507 (+1331206us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540050b754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:03.809689 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691593600}
I20260709 13:02:03.577772  4912 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:04.045917  5074 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.000s
I20260709 13:02:04.301792  1703 rpcz_store.cc:276] 0709 13:02:02.465312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.467829 (+  2517us) service_pool.cc:224] Handling call
0709 13:02:03.716892 (+1249063us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:03.717015 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401713920}
I20260709 13:02:04.304483  5074 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.305347  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:03.579035  4766 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:04.305693  5066 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.000s
W20260709 13:02:04.107858  5123 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:02:04.306265  2296 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
W20260709 13:02:04.202841  4852 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.378s	user 0.013s	sys 0.000s
I20260709 13:02:03.788450  4960 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:03.684412  4772 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:04.312215  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.313081  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.282202  2288 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:03.610906  4986 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.711s	user 0.006s	sys 0.004s
I20260709 13:02:03.808715  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1563) took 1329 ms. Trace:
I20260709 13:02:03.609020  4919 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:04.281491  4744 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:04.324398  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.317153  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.288738  4974 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.710919  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1518) took 2224 ms. Trace:
I20260709 13:02:04.264074  1634 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:04.340296  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2209) took 1468 ms. Trace:
I20260709 13:02:04.340714  1634 rpcz_store.cc:276] 0709 13:02:02.871580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.705632 (+834052us) service_pool.cc:224] Handling call
0709 13:02:04.263886 (+558254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.339989 (+ 76103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.340221 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.283448  5044 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:04.036038  4789 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:04.274998  4769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.818s	user 0.008s	sys 0.052s
I20260709 13:02:03.933828  5050 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:04.296815  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 925) took 1589 ms. Trace:
I20260709 13:02:04.303671  1555 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:04.353487  5088 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.745s	user 0.000s	sys 0.002s
I20260709 13:02:03.552776  5095 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:04.359270  5108 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.847s	user 0.000s	sys 0.000s
W20260709 13:02:04.305503  5024 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.845s	user 0.000s	sys 0.005s
I20260709 13:02:04.306414  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.274406  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.278414  5014 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.315007  4772 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.264859  5036 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:03.705878  5110 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:03.548882  4904 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:03.623039  5109 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.703s
I20260709 13:02:04.338114  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:04.309809  4909 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:02:04.290477  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.300463  4913 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:03.685627  4975 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.342041  1704 rpcz_store.cc:276] 0709 13:02:01.486931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.487063 (+   132us) service_pool.cc:224] Handling call
0709 13:02:01.487387 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.937194 (+1449807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.710556 (+773362us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400132ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:03.710830 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3118217088}
I20260709 13:02:04.316232  1557 rpcz_store.cc:276] 0709 13:02:02.480520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.515780 (+ 35260us) service_pool.cc:224] Handling call
0709 13:02:02.606067 (+ 90287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.014147 (+408080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.808346 (+794199us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002bbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:03.808621 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829962496}
W20260709 13:02:04.315176  4958 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.281s	user 0.015s	sys 0.000s
I20260709 13:02:04.277571  4916 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.347985  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2378) took 1358 ms. Trace:
W20260709 13:02:04.351033  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2200) took 3521 ms (client timeout 1593 ms). Trace:
I20260709 13:02:04.351274  1633 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:04.364367  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:04.406111  1628 rpcz_store.cc:269] 0709 13:02:00.829399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.834216 (+  4817us) service_pool.cc:224] Handling call
0709 13:02:04.350822 (+3516606us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b54000cd0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:04.350957 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71666560}
I20260709 13:02:04.368629  1552 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:04.369171  2291 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:04.408864  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:04.303007  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2211) took 1422 ms. Trace:
I20260709 13:02:04.409762  1632 rpcz_store.cc:276] 0709 13:02:02.880003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.712281 (+832278us) service_pool.cc:224] Handling call
0709 13:02:03.712721 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.301839 (+589118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.302758 (+   919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.277029  5020 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.371840  2288 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:04.353642  2290 rpcz_store.cc:276] 0709 13:02:02.707701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.708127 (+   426us) service_pool.cc:224] Handling call
0709 13:02:02.708520 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.708796 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:03.127003 (+418207us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.292421 (+1165418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.292475 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:04.292525 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.296528 (+  4003us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.296733 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85614,"prepare.queue_time_us":732489,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":1477440,"threads_started":2,"wal-append.queue_time_us":1224322}
I20260709 13:02:04.378432  4772 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.414299  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.337142  4653 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.388820  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.419605  1555 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.390353  5110 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.371212  4919 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:04.405468  4955 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.008s
W20260709 13:02:04.360996  5052 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:04.396481  2294 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:04.342198  4954 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:04.473158  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1559) took 3984 ms. Trace:
I20260709 13:02:04.367856  4789 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.473635  1558 rpcz_store.cc:276] 0709 13:02:00.489134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.490701 (+  1567us) service_pool.cc:224] Handling call
0709 13:02:00.491159 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.609265 (+118106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.609291 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.684096 (+ 74805us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.613310 (+929214us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:01.613347 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.613444 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.298726 (+2685282us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:04.420696 (+121970us) raft_consensus.cc:1685] finished
0709 13:02:04.420716 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.473043 (+ 52327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131566,"raft.queue_time_us":118204,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":72073,"spinlock_wait_cycles":72757120,"thread_start_us":844119,"threads_started":1,"wal-append.queue_time_us":738358}
I20260709 13:02:04.393522  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 690) took 1280 ms. Trace:
I20260709 13:02:04.475942  2289 rpcz_store.cc:276] 0709 13:02:03.112913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.316785 (+1203872us) service_pool.cc:224] Handling call
0709 13:02:04.393429 (+ 76644us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:04.377568  4895 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.298s	user 0.013s	sys 0.003s
I20260709 13:02:04.400198  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:04.377326  5123 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:04.367646  5002 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.314384  4852 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.337633  4986 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.413621  2296 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:04.312309  5066 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.415470  5069 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:04.402196  4913 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.422717  2290 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:04.429360  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.369292  5044 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.393075  5131 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:04.353628  5050 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.453291  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.534286  1557 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:04.475636  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.516896  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:04.525290  1551 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:04.536633  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.537400  1551 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:04.423714  4907 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.526569  4896 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.545986  1525 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.729s (SENT)
I20260709 13:02:04.546572  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:04.530227  1552 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:04.530303  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 689) took 1459 ms. Trace:
I20260709 13:02:04.548103  2290 rpcz_store.cc:276] 0709 13:02:03.071090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.422077 (+1350987us) service_pool.cc:224] Handling call
0709 13:02:04.422387 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.529992 (+107605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.530224 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.432176  4976 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:04.432704  5110 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:04.395121  5036 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.535656  1558 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:04.452003  4769 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:04.540473  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.542140  1560 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:04.452275  5082 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:04.576819  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 483) took 1555 ms. Trace:
I20260709 13:02:04.577401  1558 rpcz_store.cc:276] 0709 13:02:03.021162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.475365 (+1454203us) service_pool.cc:224] Handling call
0709 13:02:04.576710 (+101345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.578641  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.419023  4653 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.556027  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.576570  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.601063  1557 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:04.601332  1555 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:04.538573  5074 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.579471  1558 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:04.605787  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 482) took 1585 ms. Trace:
I20260709 13:02:04.606231  1558 rpcz_store.cc:276] 0709 13:02:03.020857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.578313 (+1557456us) service_pool.cc:224] Handling call
0709 13:02:04.605697 (+ 27384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.607399  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.610301  1558 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:04.616804  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.527484  4895 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.582986  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 511) took 3068 ms. Trace:
I20260709 13:02:04.540117  5069 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.554319  4976 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.601771  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2212) took 1026 ms. Trace:
I20260709 13:02:04.601895  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2205) took 1034 ms. Trace:
I20260709 13:02:04.629675  1557 rpcz_store.cc:276] 0709 13:02:03.574974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.535057 (+960083us) service_pool.cc:224] Handling call
0709 13:02:04.601679 (+ 66622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.629833  1555 rpcz_store.cc:276] 0709 13:02:03.567105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.576212 (+1009107us) service_pool.cc:224] Handling call
0709 13:02:04.601832 (+ 25620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.553553  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.631383  1552 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:04.631960  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:04.632678  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.633493  1552 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:04.622892  1630 rpcz_store.cc:276] 0709 13:02:01.514252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.608437 (+2094185us) service_pool.cc:224] Handling call
0709 13:02:03.684846 (+ 76409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.269414 (+584568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.269490 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.270503 (+  1013us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.582502 (+311999us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:04.582562 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.582655 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.582717 (+    62us) raft_consensus.cc:1685] finished
0709 13:02:04.582736 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.582910 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68868,"raft.queue_time_us":579336,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7269,"thread_start_us":271449,"threads_started":1,"wal-append.queue_time_us":280925}
I20260709 13:02:04.634532  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.635442  1552 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:04.637315  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 329765us
I20260709 13:02:04.638923  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2217) took 1005 ms. Trace:
I20260709 13:02:04.639443  1552 rpcz_store.cc:276] 0709 13:02:03.634078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.634199 (+1000121us) service_pool.cc:224] Handling call
0709 13:02:04.638843 (+  4644us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:04.581677  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 313604us
I20260709 13:02:04.618149  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2210) took 1043 ms. Trace:
I20260709 13:02:04.640619  1558 rpcz_store.cc:276] 0709 13:02:03.574419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.616474 (+1042055us) service_pool.cc:224] Handling call
0709 13:02:04.618075 (+  1601us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.640908  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:04.641989  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.612715  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 514) took 3097 ms. Trace:
I20260709 13:02:04.586957  1551 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:04.618722  5145 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:04.571363  4896 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.636240  4960 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.652165  4954 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:04.655174  4916 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.653012  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:04.691218  2290 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:04.709352  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.710131  2290 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:04.691238  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.692006  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.599753  5082 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.603600  4769 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.697346  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1581) took 2079 ms. Trace:
W20260709 13:02:04.698038  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.680330  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.680526  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:04.728193  1555 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:04.728740  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2207) took 1161 ms. Trace:
I20260709 13:02:04.729136  1555 rpcz_store.cc:276] 0709 13:02:03.567664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.630953 (+1063289us) service_pool.cc:224] Handling call
0709 13:02:04.728666 (+ 97713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.654126  4904 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.653940  5083 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.539178  5131 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.691152  5030 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:04.711206  1552 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:04.712822  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59914.6us
I20260709 13:02:04.764664  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2216) took 1157 ms. Trace:
I20260709 13:02:04.765321  1552 rpcz_store.cc:276] 0709 13:02:03.607294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.640272 (+1032978us) service_pool.cc:224] Handling call
0709 13:02:04.764570 (+124298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.702590  5077 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.770531  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:04.772287  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 336705us
I20260709 13:02:04.731424  1551 rpcz_store.cc:276] 0709 13:02:02.617867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.658209 (+2040342us) service_pool.cc:224] Handling call
0709 13:02:04.696451 (+ 38242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.696924 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.696959 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.697015 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.697067 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.697134 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.697269 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:04.739228  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 193259us
I20260709 13:02:04.773814  1601 leader_election.cc:400] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:04.774430  1601 leader_election.cc:403] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:02:04.775315  1525 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.813s (SENT)
I20260709 13:02:04.776099  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:04.704818  5002 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:04.797156  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:04.786933  1703 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:04.790313  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:04.719998  5014 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.798317  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 510) took 3265 ms. Trace:
W20260709 13:02:04.798940  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 313953us
I20260709 13:02:04.808241  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:04.816351  1633 rpcz_store.cc:276] 0709 13:02:01.513646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.818395 (+304749us) service_pool.cc:224] Handling call
0709 13:02:04.351000 (+2532605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.406355 (+ 55355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.778615 (+372260us) spinlock_profiling.cc:243] Waited 2.47 s on lock 0x7b54000cd0d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:04.778839 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5671475456}
I20260709 13:02:04.720820  4555 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:04.818787  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 180086us
I20260709 13:02:04.741359  5095 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.825189  1557 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:04.742674  5146 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.822888  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.823993  1702 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:04.755304  4909 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.826196  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2208) took 1252 ms. Trace:
I20260709 13:02:04.840385  1557 rpcz_store.cc:276] 0709 13:02:03.573812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.790475 (+1216663us) service_pool.cc:224] Handling call
0709 13:02:04.824930 (+ 34455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.825871 (+   941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.826122 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.827385  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:04.740970  4780 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:04.842403  1703 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:04.775007  5030 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:04.761883  4904 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.795706  4919 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.829145  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 503) took 2588 ms. Trace:
I20260709 13:02:04.828969  1552 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:04.844517  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:04.825312  4907 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:04.850210  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 51813.4us
I20260709 13:02:04.867025  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:04.862849  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2204) took 1296 ms. Trace:
I20260709 13:02:04.861049  1707 rpcz_store.cc:276] 0709 13:02:02.240444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.282400 (+ 41956us) service_pool.cc:224] Handling call
0709 13:02:02.282672 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.282880 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:04.350070 (+2067190us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b18003d1080. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf
0709 13:02:04.350244 (+   174us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.828598 (+478354us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.828796 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.828849 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:04.828901 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.828972 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.829086 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23909,"prepare.queue_time_us":598302,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":3644,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":40716,"spinlock_wait_cycles":7140096,"thread_start_us":1116430,"threads_started":3}
I20260709 13:02:04.869122  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 829) took 1856 ms. Trace:
I20260709 13:02:04.822710  4555 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:04.829581  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65280us
I20260709 13:02:04.871665  2288 rpcz_store.cc:276] 0709 13:02:03.013069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.430033 (+1416964us) service_pool.cc:224] Handling call
0709 13:02:04.430425 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.430688 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:04.772005 (+341317us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.868742 (+ 96737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.868811 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:04.868866 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.868937 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.869065 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13039,"raft.queue_time_us":13165,"thread_start_us":352422,"threads_started":2}
I20260709 13:02:04.797775  5077 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.838578  5014 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:04.879901  1557 pending_rounds.cc:85] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 3
I20260709 13:02:04.878461  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:04.875002  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:04.854619  5030 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:04.880903  1557 pending_rounds.cc:107] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:02:04.908967  1557 raft_consensus.cc:2889] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:04.870029  1598 connection.cc:256] RPC call timeout handler was delayed by -1.1741233580000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 13:02:04.882272  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:04.868733  1552 rpcz_store.cc:276] 0709 13:02:03.566829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.789859 (+1223030us) service_pool.cc:224] Handling call
0709 13:02:04.828685 (+ 38826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.862487 (+ 33802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.862748 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.911727  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:04.912330  1552 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:04.912940  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:04.847610  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:04.883549  5122 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:04.883904  4653 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.854202  4909 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.884555  5147 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:04.880030  4907 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:04.885418  5150 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:04.889063  5082 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.889534  4976 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.848436  5146 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.889971  4998 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:04.890475  5050 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.892449  4789 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.953474  4653 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:04.890506  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:04.899190  4744 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.000s
I20260709 13:02:04.855038  4780 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.855347  5148 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:04.860823  4904 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.803259  5002 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.855271  4960 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:02:04.874136  5136 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 13:02:04.902225  5036 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.839895  5095 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:04.917923  5083 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:04.995757  5083 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.996273  5083 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:04.984606  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:04.962591  1555 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:04.905112  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:04.910063  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:05.000265  5082 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.913173  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2220) took 1163 ms. Trace:
I20260709 13:02:05.001479  5082 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.870893  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:04.877799  5134 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 13:02:05.002313  5082 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:04.951599  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.003548  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:05.001533  1552 rpcz_store.cc:276] 0709 13:02:03.750308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.911550 (+1161242us) service_pool.cc:224] Handling call
0709 13:02:04.912015 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.912872 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.913100 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.003705  1707 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:05.005453  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1585) took 1078 ms. Trace:
I20260709 13:02:05.005982  1552 rpcz_store.cc:276] 0709 13:02:03.927269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.004388 (+1077119us) service_pool.cc:224] Handling call
0709 13:02:05.004790 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.005009 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.005048 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.005106 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:05.005163 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.005234 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.005376 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.880108  5014 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.008059  5014 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:04.903947  4895 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:04.891674  4769 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:04.960544  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.956753  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:04.917737  4916 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:05.012984  4916 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:05.013584  4916 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:04.877324  4555 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.891645  5131 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.982939  5158 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:05.020610  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:04.959331  5026 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.000s
I20260709 13:02:04.981186  4653 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:04.998931  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 93952.6us
I20260709 13:02:04.945356  5152 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:05.024537  5159 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.990592  4974 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.648s	user 0.005s	sys 0.005s
I20260709 13:02:05.025169  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.024722  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:04.916370  4975 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 13:02:04.995368  4772 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.001s
I20260709 13:02:05.031261  2291 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:05.093906  2288 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:05.093994  1705 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:05.124835  4895 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.125267  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:05.125694  4780 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:05.125476  4895 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.126076  2288 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:05.127427  4909 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:05.128610  5114 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.569s	user 0.015s	sys 0.001s
I20260709 13:02:05.130453  5002 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:05.131887  5136 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:02:05.134516  5095 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.992144  5069 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:04.960064  4998 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.143895  2288 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:04.951478  5147 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:05.059494  5148 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:05.027514  4954 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.026723  4653 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:05.026492  4789 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.171375  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:05.065627  4744 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:05.173861  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:05.174547  1707 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:05.175064  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 488) took 1671 ms. Trace:
W20260709 13:02:05.178809  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 116639us
I20260709 13:02:05.185420  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.263123  1705 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:05.267688  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.269474  1703 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:05.145879  5122 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:05.156275  5077 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.148371  5146 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.275281  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.136783  5159 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.276798  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.277434  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:05.284462  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.144783  4772 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.276458  5148 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:05.292327  2288 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:05.293007  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.294209  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1577) took 2791 ms. Trace:
I20260709 13:02:05.367681  5148 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:05.368973  4772 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.368986  1556 rpcz_store.cc:276] 0709 13:02:02.503160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.933635 (+430475us) service_pool.cc:224] Handling call
0709 13:02:02.949676 (+ 16041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.041901 (+ 92225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.041931 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.041975 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:03.042011 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.293956 (+2251945us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:05.294003 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.294147 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211611904}
I20260709 13:02:05.312448  5142 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:05.372150  5142 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:05.338403  1700 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:04.982583  4907 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.368072  2295 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:05.169651  4904 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.172942  4895 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.157377  4998 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:05.262660  4744 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.264547  1558 rpcz_store.cc:276] 0709 13:02:03.503214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.656022 (+1152808us) service_pool.cc:224] Handling call
0709 13:02:04.656425 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.656682 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:04.822971 (+166289us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.174594 (+351623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.174669 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:05.174721 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.174824 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.174990 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3364,"thread_start_us":477305,"threads_started":2}
W20260709 13:02:05.402333  5104 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.959s	user 0.004s	sys 0.004s
W20260709 13:02:05.402629  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 818) took 3833 ms (client timeout 2161 ms). Trace:
I20260709 13:02:05.268558  4954 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.411108  2295 rpcz_store.cc:269] 0709 13:02:01.569095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.569196 (+   101us) service_pool.cc:224] Handling call
0709 13:02:04.263720 (+2694524us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:05.402390 (+1138670us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005e1854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:05.402557 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543790592}
I20260709 13:02:05.405706  4913 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.186698  5147 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:05.416102  5110 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.815s	user 0.002s	sys 0.009s
I20260709 13:02:05.416850  5142 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.409343  5135 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.673s	user 0.002s	sys 0.000s
I20260709 13:02:05.022711  5158 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.420783  5141 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.520s	user 0.000s	sys 0.003s
I20260709 13:02:05.432862  5022 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.434257  4852 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:05.450266  2289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.024s	sys 0.022s
I20260709 13:02:05.412788  4744 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:05.469950  1551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.011s	sys 0.035s
W20260709 13:02:05.476948  1551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.011s	sys 0.035s
I20260709 13:02:05.189375  5082 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:05.453752  1706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.028s	sys 0.018s
W20260709 13:02:05.482968  1706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.028s	sys 0.019s
W20260709 13:02:05.469272  2289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.024s	sys 0.022s
I20260709 13:02:05.361686  4789 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.503037  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2209) took 1927 ms. Trace:
I20260709 13:02:05.411191  4555 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.505697  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.002s	sys 0.035s
I20260709 13:02:05.487393  5142 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:05.451449  5110 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:05.522684  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.002s	sys 0.035s
I20260709 13:02:05.493141  5159 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.492453  4852 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.515565  1551 rpcz_store.cc:276] 0709 13:02:03.574112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.790932 (+1216820us) service_pool.cc:224] Handling call
0709 13:02:04.791255 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.791560 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.791586 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.792104 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.477729 (+685625us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:05.500489 (+ 22760us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.500747 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.500828 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.500868 (+    40us) raft_consensus.cc:1685] finished
0709 13:02:05.500878 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.501013 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25859,"raft.queue_time_us":167285,"thread_start_us":639376,"threads_started":1,"wal-append.queue_time_us":652131}
I20260709 13:02:05.492599  4895 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:05.524818  1556 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:05.503463  5127 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:05.533252  1557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.010s	sys 0.026s
W20260709 13:02:05.534689  1557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.010s	sys 0.026s
I20260709 13:02:05.539966  4649 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.546531  4954 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.535260  4895 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.526086  4852 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:05.561486  5117 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 2.118s	user 0.003s	sys 0.000s
I20260709 13:02:05.561957  5147 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:05.563163  4555 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.564455  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 696) took 1195 ms. Trace:
I20260709 13:02:05.564879  2291 rpcz_store.cc:276] 0709 13:02:04.368686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.408536 (+ 39850us) service_pool.cc:224] Handling call
0709 13:02:05.564231 (+1155695us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54004d49d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:05.564387 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428791552}
I20260709 13:02:05.502768  5014 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.569890  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.007s	sys 0.015s
W20260709 13:02:05.570334  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.015s
I20260709 13:02:05.570943  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 821) took 2606 ms. Trace:
W20260709 13:02:05.568110  5030 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.004s
W20260709 13:02:05.570628  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 540753us
I20260709 13:02:05.573369  2292 rpcz_store.cc:276] 0709 13:02:02.964495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.966527 (+  2032us) service_pool.cc:224] Handling call
0709 13:02:02.985975 (+ 19448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.986391 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.986430 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.986958 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.443259 (+456301us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:03.443319 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.443413 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.560806 (+2117393us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:05.565886 (+  5080us) raft_consensus.cc:1685] finished
0709 13:02:05.565908 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.570845 (+  4937us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":23824,"spinlock_wait_cycles":299062528,"thread_start_us":407131,"threads_started":1,"wal-append.queue_time_us":428075}
I20260709 13:02:05.553514  5156 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:05.583746  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:05.536443  5127 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:05.582791  5083 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.586s	user 0.006s	sys 0.007s
I20260709 13:02:05.587049  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:05.594496  5066 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:05.596994  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:05.601550  5147 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.602061  5158 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.602283  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:05.602975  1551 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:05.603459  4998 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.603927  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:05.604518  1551 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:05.605787  4998 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:05.607064  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:05.610750  1551 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:05.587797  4912 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.589210  4555 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:05.598096  4954 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:05.585781  5030 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:05.621212  1551 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:05.630388  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:05.630996  1674 connection.cc:256] RPC call timeout handler was delayed by -1.8267209519999597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:02:05.631603  1674 connection.cc:256] RPC call timeout handler was delayed by -1.6472547889999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:02:05.632582  1674 connection.cc:256] RPC call timeout handler was delayed by -1.5455547889999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 13:02:05.631302  1558 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:05.632161  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.004s	sys 0.025s
W20260709 13:02:05.636124  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.004s	sys 0.025s
I20260709 13:02:05.596918  5083 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.621248  4555 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.634773  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:05.637730  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 727696us
I20260709 13:02:05.622658  5158 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:02:05.622256  4912 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:05.709995  1552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.004s	sys 0.015s
W20260709 13:02:05.644111  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.967412  1552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.012s	sys 0.022s
W20260709 13:02:05.940383  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 517) took 3871 ms (client timeout 2118 ms). Trace:
W20260709 13:02:05.644677  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.002s	sys 0.018s
W20260709 13:02:05.968066  1629 rpcz_store.cc:269] 0709 13:02:02.069399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.792053 (+722654us) service_pool.cc:224] Handling call
0709 13:02:05.940052 (+3147999us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:05.940285 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369747712}
W20260709 13:02:05.968863  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.013s	sys 0.021s
W20260709 13:02:05.811970  5171 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:02:05.969532  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.05658e+06us
I20260709 13:02:05.680619  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.970407  5044 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:05.678874  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:05.859355  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 491) took 1068 ms. Trace:
W20260709 13:02:05.740429  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.972280  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:05.972960  1558 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:05.724078  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.974357  1701 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:02:05.975274  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.684741  2288 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:05.773710  5110 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:05.636356  4954 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:05.976930  1701 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:05.977433  4954 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.697130  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.981251  1703 raft_consensus.cc:2410] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:05.634428  5147 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.982256  4912 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:05.645126  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:05.696642  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.982561  5147 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:05.983784  1700 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:05.984342  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.984826  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:05.985081  1704 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:05.986296  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.696159  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.789875  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:05.978114  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.970909  1708 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:05.966758  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:05.982360  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:06.034631  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.029s	sys 0.072s
W20260709 13:02:05.724246  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.019s	sys 0.057s
W20260709 13:02:06.057194  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.029s	sys 0.078s
W20260709 13:02:06.057303  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.030s	sys 0.064s
I20260709 13:02:05.971786  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:05.970477  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:05.971755  1556 rpcz_store.cc:276] 0709 13:02:04.790653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.370372 (+579719us) service_pool.cc:224] Handling call
0709 13:02:05.370706 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.529217 (+158511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.858970 (+329753us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54000771d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:05.859261 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347759360}
W20260709 13:02:06.077935  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.23073e+06us
I20260709 13:02:06.078310  1709 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.062922  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:06.074747  5030 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.080245  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:05.993073  5121 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.533s	user 0.016s	sys 0.017s
W20260709 13:02:06.031545  4916 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.014s	user 0.011s	sys 0.007s
I20260709 13:02:05.967036  4555 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:05.753947  4960 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.000s	sys 0.023s
W20260709 13:02:05.710633  5157 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.024s	sys 0.037s
I20260709 13:02:05.843062  5071 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:06.068740  4744 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.575s	user 0.005s	sys 0.006s
I20260709 13:02:06.071954  5179 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:06.074199  5014 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:06.021222  5148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.005s	sys 0.025s
W20260709 13:02:05.933923  5116 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.459s	user 0.000s	sys 0.003s
I20260709 13:02:06.081131  1709 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:06.097927  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.099047  1709 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:02:06.084472  5083 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.071981  5004 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
W20260709 13:02:06.096925  5148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.006s	sys 0.028s
I20260709 13:02:06.089488  4916 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.128412  5147 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:05.988091  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.117s	user 0.023s	sys 0.018s
W20260709 13:02:06.190191  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.319s	user 0.031s	sys 0.018s
I20260709 13:02:06.065693  1551 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.203290  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1589) took 2255 ms. Trace:
I20260709 13:02:06.204001  1552 rpcz_store.cc:276] 0709 13:02:03.947704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.007190 (+1059486us) service_pool.cc:224] Handling call
0709 13:02:05.007569 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.012417 (+  4848us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:05.967989 (+955572us) log.cc:844] Serialized 14 byte log entry
0709 13:02:06.201886 (+233897us) log.cc:844] Serialized 14 byte log entry
0709 13:02:06.202291 (+   405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.202372 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:06.202525 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.202674 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.203014 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":286398,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":1184276,"threads_started":2,"wal-append.queue_time_us":175846}
I20260709 13:02:06.207382  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:06.209087  1552 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:06.071866  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 475) took 3792 ms. Trace:
I20260709 13:02:06.211920  1553 rpcz_store.cc:276] 0709 13:02:02.279091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.279242 (+   151us) service_pool.cc:224] Handling call
0709 13:02:02.279567 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.279979 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.280015 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.283791 (+  3776us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.973517 (+689726us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:03.014794 (+ 41277us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.210874 (+196080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.210983 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.069163 (+2858180us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b180047c560. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:06.071540 (+  2377us) raft_consensus.cc:1685] finished
0709 13:02:06.071642 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.071807 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44786,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":188911,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":54996,"spinlock_wait_cycles":27703296,"thread_start_us":818289,"threads_started":2,"wal-append.queue_time_us":725879}
I20260709 13:02:06.215811  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.217001  1553 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.224869  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.226147  1556 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:06.227120  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:06.228657  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:06.229563  1551 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:06.230609  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:06.231963  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.233016  1551 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.071336  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2214) took 2495 ms. Trace:
W20260709 13:02:06.233798  1524 connection.cc:256] RPC call timeout handler was delayed by -1.0994480040000099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23730000000000004s for a response
I20260709 13:02:06.068257  1558 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:06.236302  1557 rpcz_store.cc:276] 0709 13:02:03.592769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.863952 (+1271183us) service_pool.cc:224] Handling call
0709 13:02:04.879657 (+ 15705us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:05.025156 (+145499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.025190 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:05.025793 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 13:02:05.537042 (+511249us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:05.551105 (+ 14063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.551460 (+   355us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.071097 (+519637us) raft_consensus.cc:1685] finished
0709 13:02:06.071117 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.071273 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":848416,"raft.queue_time_us":135047,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":790805,"thread_start_us":510070,"threads_started":1,"wal-append.queue_time_us":784763}
I20260709 13:02:06.238371  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.239136  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.240006  1557 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.240468  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:06.241549  1558 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:06.242848  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.243741  1558 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:06.236785  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.249043  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 169923us
I20260709 13:02:06.066346  1704 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 5.
I20260709 13:02:06.059306  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.251144  1708 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.081983  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.252996  1703 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:06.257691  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 194684us
I20260709 13:02:05.966421  4852 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.090144  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.260177  4852 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.260857  2293 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:06.261201  4852 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:05.987140  4895 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.087378  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:06.089684  5157 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.033s	sys 0.050s
I20260709 13:02:06.264832  1701 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:06.066177  1705 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:06.285286  5189 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:06.083528  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.288313  5148 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:06.288733  2292 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:06.289005  5148 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:06.088285  4960 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.008s	sys 0.035s
I20260709 13:02:06.058463  5110 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:06.291442  4916 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:06.292707  5110 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:06.293288  5110 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:06.082423  5121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.622s	user 0.020s	sys 0.020s
W20260709 13:02:06.263473  4653 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.885s	user 0.014s	sys 0.003s
I20260709 13:02:06.295595  4974 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:06.296994  4653 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:06.298550  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.299436  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 489) took 1608 ms. Trace:
I20260709 13:02:06.300585  1555 rpcz_store.cc:276] 0709 13:02:04.690936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.741129 (+ 50193us) service_pool.cc:224] Handling call
0709 13:02:06.298944 (+1557815us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003fa054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.299234 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335705984}
I20260709 13:02:06.301396  4919 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:06.301972  4919 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:06.302085  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.303112  1555 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:06.304760  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:06.305094  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 691) took 3167 ms (client timeout 3139 ms). Trace:
I20260709 13:02:06.305485  2291 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:02:06.305541  2296 rpcz_store.cc:269] 0709 13:02:03.137411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.312464 (+1175053us) service_pool.cc:224] Handling call
0709 13:02:06.304702 (+1992238us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b54005186d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.304877 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15577344}
I20260709 13:02:06.306720  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:06.307658  1555 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:02:06.308837  1701 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:06.308851  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.247850  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.096026  4744 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.325871  5020 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.326501  4744 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.328629  5077 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:06.328819  5067 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:06.329417  5077 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.329675  5067 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.329938  5077 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:06.280822  4852 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.289598  5148 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:06.320168  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 349572us
I20260709 13:02:06.321472  4955 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:06.322881  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51204.1us
W20260709 13:02:06.323372  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 533446us
I20260709 13:02:06.336048  1557 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:06.336724  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:06.344646  1708 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:02:06.347208  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:06.292737  1702 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:06.352267  1703 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:06.336133  4955 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:06.351052  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.357191  1551 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:06.363366  1552 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:06.303320  4919 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:06.364753  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.366272  5071 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:06.368714  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1539) took 1538 ms. Trace:
I20260709 13:02:06.369046  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.368570  4955 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:06.370467  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93221us
I20260709 13:02:06.370572  1553 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:02:06.371502  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 288300us
I20260709 13:02:06.372370  1556 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:06.376672  1552 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:06.377584  1558 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:06.377709  1675 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:06.295085  5110 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.358671  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.331452  5067 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.386732  1675 leader_election.cc:403] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:06.389541  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.389830  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.390589  4955 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.391956  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.392377  4852 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:06.385298  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.368039  5079 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:06.396404  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:06.400063  4895 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.400704  4895 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.401868  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.402513  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.402945  2291 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:06.403450  2295 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:06.404219  2292 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:06.404387  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.381964  5071 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:06.404206  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.408869  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.408747  1557 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:02:06.405045  5110 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.380429  1706 rpcz_store.cc:276] 0709 13:02:04.830068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.830283 (+   215us) service_pool.cc:224] Handling call
0709 13:02:04.837892 (+  7609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.838124 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:05.483650 (+645526us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.691366 (+207716us) log.cc:844] Serialized 14 byte log entry
0709 13:02:06.319240 (+627874us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b18004174c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:06.357951 (+ 38711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.359301 (+  1350us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:06.359412 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.359539 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.368440 (+  8901us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34269,"prepare.queue_time_us":316797,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":98,"raft.queue_time_us":642308,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":293,"spinlock_wait_cycles":164332288,"thread_start_us":837562,"threads_started":2,"wal-append.queue_time_us":618641}
I20260709 13:02:06.399020  5079 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.414409  1558 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:06.414840  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:06.412014  1553 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:06.418184  4744 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:06.418740  4744 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.418892  4653 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.419224  4744 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:06.419828  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70572.9us
I20260709 13:02:06.423882  5189 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:06.424507  5189 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:06.424911  5189 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:06.426087  4919 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.428217  2296 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:06.423357  5105 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 2.946s	user 0.004s	sys 0.010s
I20260709 13:02:06.428577  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:06.431757  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:06.430886  1673 connection.cc:256] RPC call timeout handler was delayed by -1.07857144999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
I20260709 13:02:06.408646  4852 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:06.436127  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.444051  4852 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.444598  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:06.444921  5166 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.541s	user 0.000s	sys 0.003s
I20260709 13:02:06.436889  1552 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:06.446720  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 832) took 1752 ms. Trace:
I20260709 13:02:06.440516  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:06.438668  5126 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.905s	user 0.002s	sys 0.000s
I20260709 13:02:06.447160  2290 rpcz_store.cc:276] 0709 13:02:04.694361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.721738 (+ 27377us) service_pool.cc:224] Handling call
0709 13:02:04.722104 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.738277 (+ 16173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.738313 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.760977 (+ 22664us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.865465 (+104488us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:04.866038 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.866326 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.443917 (+1577591us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:06.446204 (+  2287us) raft_consensus.cc:1685] finished
0709 13:02:06.446225 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.446501 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15695,"raft.queue_time_us":16245,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":26056,"spinlock_wait_cycles":50277248,"thread_start_us":26905,"threads_started":1,"wal-append.queue_time_us":105510}
I20260709 13:02:06.449806  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.450085  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1578) took 3946 ms. Trace:
I20260709 13:02:06.445906  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.451722  2293 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:06.450495  1554 rpcz_store.cc:276] 0709 13:02:02.503492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.943244 (+439752us) service_pool.cc:224] Handling call
0709 13:02:02.943551 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.356657 (+413106us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:03.356700 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.424776 (+ 68076us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.026370 (+601594us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:04.026433 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.026596 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.437576 (+2410980us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:06.449599 (+ 12023us) raft_consensus.cc:1685] finished
0709 13:02:06.449624 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.449889 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127063,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":20478,"spinlock_wait_cycles":8499968,"thread_start_us":803937,"threads_started":2,"wal-append.queue_time_us":889297}
I20260709 13:02:06.451574  5071 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.453858  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1573) took 3952 ms. Trace:
I20260709 13:02:06.476971  4896 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.477147  5072 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.460287  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.478359  2292 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:06.478554  5071 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.461068  5020 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:06.464156  4768 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:06.456491  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.479573  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1543) took 1526 ms. Trace:
I20260709 13:02:06.479856  5020 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:06.467499  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:06.465577  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 88180.8us
I20260709 13:02:06.480041  1702 rpcz_store.cc:276] 0709 13:02:04.953699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.953842 (+   143us) service_pool.cc:224] Handling call
0709 13:02:06.479353 (+1525511us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400166054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.479499 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2883852672}
I20260709 13:02:06.470479  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.481364  1700 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
I20260709 13:02:06.490365  5020 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.480186  2295 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:06.476029  1559 rpcz_store.cc:276] 0709 13:02:02.501947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.872804 (+370857us) service_pool.cc:224] Handling call
0709 13:02:02.889428 (+ 16624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.889798 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.889838 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.890395 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.357249 (+466854us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.357300 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.357388 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.422441 (+3065053us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:06.453331 (+ 30890us) raft_consensus.cc:1685] finished
0709 13:02:06.453354 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.453672 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6171,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":81975168,"thread_start_us":466468,"threads_started":1,"wal-append.queue_time_us":586740}
I20260709 13:02:06.481642  4768 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:06.492425  4653 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:06.481678  4984 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.474462  4555 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.483510  5082 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:06.484754  4916 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:04.643244  1631 rpcz_store.cc:276] 0709 13:02:01.515029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.608466 (+2093437us) service_pool.cc:224] Handling call
0709 13:02:04.245352 (+636886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.245615 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:04.275194 (+ 29579us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.603258 (+328064us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.612270 (+  9012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.612340 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:04.612409 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.612497 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.612637 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37695,"prepare.queue_time_us":45694,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":5128,"raft.queue_time_us":42520,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":8986,"thread_start_us":300514,"threads_started":1,"wal-append.queue_time_us":291599}
I20260709 13:02:06.509172  5122 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:06.509764  5122 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.509898  4916 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:06.510447  4916 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:06.511170  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2197) took 5682 ms (client timeout 1729 ms). Trace:
I20260709 13:02:06.510970  5122 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:06.511610  1627 rpcz_store.cc:269] 0709 13:02:00.828363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.831641 (+  3278us) service_pool.cc:224] Handling call
0709 13:02:06.510680 (+5679039us) spinlock_profiling.cc:243] Waited 3.28 s on lock 0x7b5400343ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.510923 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7550111488}
I20260709 13:02:06.511430  4916 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.512557  5122 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.513496  4916 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.514284  5019 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.514963  5019 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.515578  5185 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:06.516172  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:06.517233  5083 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:06.518685  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:06.440428  5142 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.913s	user 0.010s	sys 0.010s
I20260709 13:02:06.492604  5127 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.484673  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.482638  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 698) took 1652 ms. Trace:
I20260709 13:02:06.520562  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.494081  4768 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:06.497591  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69100.6us
W20260709 13:02:06.497779  5164 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.594s	user 0.000s	sys 0.002s
I20260709 13:02:06.502377  4912 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.502895  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 486) took 3448 ms (client timeout 1554 ms). Trace:
I20260709 13:02:06.521682  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.528317  2288 rpcz_store.cc:276] 0709 13:02:04.830019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.367746 (+537727us) service_pool.cc:224] Handling call
0709 13:02:05.368100 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.976290 (+608190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.467487 (+491197us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54004c1654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:06.482452 (+ 14965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727549184}
I20260709 13:02:06.519754  2291 raft_consensus.cc:2376] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9. Re-sending same reply.
W20260709 13:02:06.533178  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 92895.4us
I20260709 13:02:06.530262  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.520081  5082 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:06.472318  5077 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.525185  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.428670  4895 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.540546  1702 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:06.541762  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:06.530337  1560 rpcz_store.cc:269] 0709 13:02:03.054814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.488098 (+433284us) service_pool.cc:224] Handling call
0709 13:02:06.502409 (+3014311us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002b98d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.502663 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410164480}
W20260709 13:02:06.527930  5119 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 3.069s	user 0.000s	sys 0.011s
I20260709 13:02:06.531144  4768 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.548997  1556 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:06.549886  2296 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:06.558568  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.559518  1703 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.559713  1701 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.515617  5019 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.559394  1704 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:06.562778  5019 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:06.563182  2260 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.554s (SENT)
I20260709 13:02:06.565266  4555 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:04.405107  1626 rpcz_store.cc:276] 0709 13:02:02.989760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.810373 (+820613us) service_pool.cc:224] Handling call
0709 13:02:03.810826 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.811140 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:03.811582 (+   442us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.340430 (+528848us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.347415 (+  6985us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.347521 (+   106us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:04.347591 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.347695 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.347896 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42238,"prepare.queue_time_us":21573,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":100,"raft.queue_time_us":52758,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"spinlock_wait_cycles":42368,"thread_start_us":489565,"threads_started":1,"wal-append.queue_time_us":592966}
I20260709 13:02:06.569934  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.568424  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1541) took 1697 ms. Trace:
I20260709 13:02:06.571134  1707 rpcz_store.cc:276] 0709 13:02:04.870480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.870614 (+   134us) service_pool.cc:224] Handling call
0709 13:02:06.568019 (+1697405us) spinlock_profiling.cc:243] Waited 952 us on lock 0x7b5400167954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.568223 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188544}
I20260709 13:02:06.545837  5139 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:06.576213  2293 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:06.539904  4912 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:06.576815  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 198692us
I20260709 13:02:06.523111  5148 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.580792  4768 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.582120  4768 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.586047  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.587085  1708 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:06.589219  4986 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.594476  1709 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:02:06.578963  2260 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.573s (SENT)
W20260709 13:02:06.596510  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48241.2us
I20260709 13:02:06.599185  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:06.600901  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.602119  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.575896  5139 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.605396  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.614300  1702 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.623709  1556 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.631330  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.632351  1702 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:06.615469  5082 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:06.634725  5082 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.635125  5082 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.639915  4766 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:06.613308  5139 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:06.622205  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.596267  5079 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.650050  4744 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.651532  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.652529  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.670049  1703 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.653628  1705 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:06.654268  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.670929  1707 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 6: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:06.659176  5204 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:02:06.660694  5200 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:02:06.668303  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:06.669183  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44804.8us
W20260709 13:02:06.675978  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53239.1us
I20260709 13:02:06.679733  1556 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:06.681540  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.642791  4766 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.686218  1701 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:06.682225  1702 raft_consensus.cc:2376] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11. Re-sending same reply.
I20260709 13:02:06.687285  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.696717  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.697955  1703 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:06.698006  1708 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:06.688627  4766 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.702955  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.704875  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:06.705714  1702 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:06.706650  5179 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:06.707109  5179 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:06.707365  5003 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:06.708017  5003 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.708151  5179 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.709007  5003 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.712668  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.648940  5067 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:06.713725  5067 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:06.714090  4751 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.714704  4751 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.716301  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:06.717182  4751 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.717530  1627 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:06.717837  4751 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:06.718284  4751 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.718649  4751 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.719182  1631 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:06.719408  5149 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.702209  5021 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.719995  5149 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.720566  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:06.721576  1629 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:06.722410  5149 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.714063  5067 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.725929  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.725660  5067 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.726655  1708 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:06.727022  5067 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.724622  1630 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.729923  5079 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.730028  4744 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.730919  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.732116  5019 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:06.732738  5019 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:06.733837  5019 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.737699  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:06.739921  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:06.740025  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 45353.5us
I20260709 13:02:06.740165  5185 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:06.740741  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.741307  5079 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:06.741971  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:06.744385  5148 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.745486  4912 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.692693  4974 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:06.757913  4974 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.714021  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:06.712723  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:06.753937  1702 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:06.755862  4789 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.760174  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:06.769778  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:06.771571  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:06.780349  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67740.3us
I20260709 13:02:06.720497  5021 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.771739  5083 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:06.760260  4974 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:06.797662  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:06.772260  5077 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:06.801034  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:06.804220  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:06.806671  5004 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.807286  5004 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:06.809453  5004 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.810012  5079 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:06.810467  5079 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:06.810819  5079 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.771529  4766 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.814316  4766 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:06.819957  5148 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:06.820245  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 825) took 3837 ms (client timeout 2161 ms). Trace:
I20260709 13:02:06.820565  5148 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.821033  5148 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:06.820669  2294 rpcz_store.cc:269] 0709 13:02:02.982655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.525604 (+542949us) service_pool.cc:224] Handling call
0709 13:02:06.805769 (+3280165us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54006c02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.820046 (+ 14277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912979840}
W20260709 13:02:06.822078  2260 leader_election.cc:336] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.762s (SENT)
W20260709 13:02:06.823261  2260 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.803s (SENT)
I20260709 13:02:06.824380  4768 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:06.831297  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 117320us
I20260709 13:02:06.805274  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:06.835610  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:06.840188  1629 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:06.840965  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43750.5us
I20260709 13:02:06.845726  5020 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:06.846268  5020 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:06.846693  5020 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.849651  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:06.851369  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.852394  2294 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:06.835529  2293 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:02:06.855674  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52890.3us
I20260709 13:02:06.857088  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:06.858767  4974 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:06.792663  4912 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:06.860023  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.860356  1704 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:06.860921  2294 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:06.861248  5095 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:06.841280  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:06.761389  4852 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:02:06.859359  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:06.860420  4912 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:02:06.886754  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:06.887416  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.887856  1627 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:06.779750  5189 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:06.902680  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:06.903460  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.906883  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47602.8us
I20260709 13:02:06.921063  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:06.921167  5004 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.936987  5004 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:06.937032  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:06.940764  5142 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:06.941174  5142 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.941529  5142 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.942364  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:06.887416  4919 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:06.943818  4919 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:06.944777  4919 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:06.946308  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.946902  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:06.947934  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.948261  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.937394  4998 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:06.973109  2293 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:06.999502  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2274) took 1253 ms. Trace:
I20260709 13:02:06.902679  5095 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.005771  2296 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:06.968415  5122 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:07.015424  5122 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.016801  5072 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:07.014668  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 827) took 4022 ms (client timeout 1866 ms). Trace:
I20260709 13:02:06.961581  4998 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:07.017762  5122 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:07.017405  2297 rpcz_store.cc:269] 0709 13:02:02.990307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.009767 (+ 19460us) service_pool.cc:224] Handling call
0709 13:02:06.961233 (+3951466us) spinlock_profiling.cc:243] Waited 3.16 s on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:07.012470 (+ 51237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7271252224}
I20260709 13:02:07.018316  4998 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.002686  4852 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:07.020215  4852 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:07.021232  4852 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.023885  5095 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.000005  1709 rpcz_store.cc:276] 0709 13:02:05.709496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.099668 (+390172us) service_pool.cc:224] Handling call
0709 13:02:06.591856 (+492188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.595036 (+  3180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.962753 (+367717us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400167950. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:06.962913 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078468864}
W20260709 13:02:07.028051  2260 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.645s (SENT)
I20260709 13:02:07.034567  4789 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:07.035144  4789 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:07.041621  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 133914us
W20260709 13:02:07.042693  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 248079us
I20260709 13:02:07.049727  1629 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:02:07.058087  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 286598us
I20260709 13:02:07.070617  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.072331  1560 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:07.075034  1631 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:07.076071  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.081540  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.088686  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.089511  5148 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.090247  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:07.093731  4916 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.580s	user 0.005s	sys 0.010s
I20260709 13:02:07.093554  4751 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.096352  5021 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.096899  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.097671  1634 raft_consensus.cc:2368] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10.
I20260709 13:02:07.098531  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.099118  5139 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:07.100028  1674 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.950s (SENT) [suppressed 46 similar messages]
I20260709 13:02:07.101948  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.102682  1633 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:07.103298  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:07.104348  5205 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 13:02:07.104612  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.105268  1709 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:07.106218  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.106297  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:07.107816  4766 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:07.108035  5201 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.280s	user 0.004s	sys 0.000s
I20260709 13:02:07.087239  5079 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.108785  1705 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:02:07.108906  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:07.072460  5020 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.109602  4766 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:07.109894  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.110006  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:07.110545  1705 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.111037  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:07.111358  5020 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.107092  1709 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:07.115051  1559 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:07.111675  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.115553  1706 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:07.119020  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.120138  1552 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:07.123023  1627 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:07.125240  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.127413  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.131300  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.125939  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.096403  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.133222  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.138782  1628 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.132526  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.112579  5004 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.157910  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.158676  1707 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:07.146195  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.126490  5021 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.110031  4766 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:07.121007  5067 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:07.168613  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57655.1us
I20260709 13:02:07.206429  5067 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:07.207113  5067 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.141470  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:07.179903  5189 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:07.180135  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.211627  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.212731  1629 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.213768  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.214527  1629 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:02:07.181121  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 140036us
I20260709 13:02:07.181565  5079 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.182049  1704 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.182667  5004 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.182864  1552 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:07.183785  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.184108  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.188870  4766 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.189836  1634 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:07.190512  1632 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:07.190963  1628 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:07.192003  5095 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.083104  4653 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.059s
W20260709 13:02:07.204867  5208 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:02:07.205890  5206 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:02:07.258854  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:07.259704  2293 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:07.229084  1626 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:07.121325  4916 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:07.119851  5019 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:07.215587  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.261320  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.089537  5082 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.249502  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:07.263484  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.140434  5139 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.235090  5021 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:02:07.260470  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:07.255862  1708 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:07.260686  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.248574  5149 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:07.239358  5122 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.264241  1560 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:07.263504  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:07.252837  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.264045  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.268131  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.276643  1628 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.270798  1634 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:07.268476  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.261924  1632 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
W20260709 13:02:07.276333  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 135009us
I20260709 13:02:07.257268  5148 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.273841  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.279376  1552 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:07.284799  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:07.287632  1701 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:07.288067  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.289347  2294 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:07.135346  4789 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.297413  1631 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:02:07.292309  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.290248  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.296104  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.296481  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.296754  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.299125  1700 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:07.278657  5122 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.298188  1627 raft_consensus.cc:2376] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9. Re-sending same reply.
I20260709 13:02:07.264990  4916 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.309984  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.312413  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.313469  1702 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:07.319927  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.321631  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.324869  1628 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.328450  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:07.330315  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.332072  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.328768  1632 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.265065  5019 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:07.313117  4916 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.286275  5004 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.310470  5110 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:07.352981  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.383646  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.360430  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.353746  4919 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:07.412417  4912 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:07.389865  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.414052  1633 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:07.414287  5139 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:07.414987  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.352524  1702 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:07.416260  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63569.2us
I20260709 13:02:07.391932  1627 raft_consensus.cc:2376] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10. Re-sending same reply.
I20260709 13:02:07.417090  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:07.417555  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.416289  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.418529  5179 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.418941  4653 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:07.419528  4653 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:07.420477  4653 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.408215  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:07.356634  1708 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:07.368357  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43939.8us
W20260709 13:02:07.412418  2260 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.844s (SENT)
I20260709 13:02:07.388746  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:07.412839  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.418298  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.433744  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:07.439260  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.386221  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.356352  4789 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:07.359403  5072 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:07.457455  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49324.1us
I20260709 13:02:07.463006  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:07.401234  5019 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:07.464579  1634 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:07.465852  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.467064  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.343616  5082 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.468480  1704 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:07.469416  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 51224.4us
I20260709 13:02:07.469981  1705 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:07.466683  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:07.404019  5110 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.473701  1559 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.474229  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:07.474723  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.477437  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.477425  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65068.2us
W20260709 13:02:07.457158  5142 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.004s
I20260709 13:02:07.479879  1557 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.479897  1627 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:02:07.481634  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.482468  1632 raft_consensus.cc:2376] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 11. Re-sending same reply.
I20260709 13:02:07.458572  4919 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:07.485352  1708 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:07.491598  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.473232  5095 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:07.473142  4744 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:07.473542  5071 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:07.478710  5110 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.516726  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:07.520359  1552 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:07.507908  5139 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:07.523386  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 423357us
I20260709 13:02:07.523622  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.500876  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.495378  5095 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.504076  4744 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:07.479120  5067 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:07.535816  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 75953.8us
I20260709 13:02:07.536067  1632 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.531292  1560 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:07.527789  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.489480  5122 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.541611  1554 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:07.544734  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.545811  1553 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:07.546727  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.548336  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.542073  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:07.547580  1553 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.544709  1552 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:07.551115  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.522955  5139 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.524428  5020 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.553833  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.495793  4555 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:02:07.550369  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.538883  5095 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.487413  5142 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.524852  5071 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:07.564597  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:07.571648  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.591444  1632 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.593485  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:07.595695  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.601490  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:07.606151  1554 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:07.496075  5072 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:02:07.607224  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.514005  4998 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.609470  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.561739  5139 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.611160  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.614017  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:07.577853  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 313851us
I20260709 13:02:07.611425  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.614828  1560 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.614993  1555 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:07.611861  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 277750us
I20260709 13:02:07.606323  1559 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:07.616425  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.612975  1705 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:07.617128  1708 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.563134  4986 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:07.613986  1553 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 4.
I20260709 13:02:07.623466  1635 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.621208  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.618726  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.620005  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.625396  1704 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:02:07.626705  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 62212.5us
I20260709 13:02:07.628623  1554 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:07.633564  1629 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.637334  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:07.627562  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.642354  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.642886  1555 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:07.606807  4955 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:02:07.594208  5020 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.653893  1553 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:07.695889  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 58400.5us
I20260709 13:02:07.685701  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.647154  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.648006  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.744568  1708 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.745931  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.746869  1708 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.715518  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.691836  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:07.688679  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.618638  5189 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:07.616356  5021 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:07.624912  4852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.023s	sys 0.024s
I20260709 13:02:07.715119  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.743531  1704 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:07.760990  1705 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.747881  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:07.692276  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 225669us
W20260709 13:02:07.761919  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 155438us
I20260709 13:02:07.744761  1700 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.766100  1628 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:07.752537  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:07.779275  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 215069us
I20260709 13:02:07.780253  1557 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:07.631102  5004 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.782328  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.786219  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:07.715901  5142 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.787415  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.787532  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:07.787868  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 125043us
I20260709 13:02:07.789359  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.789898  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.790325  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.787264  1552 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
W20260709 13:02:07.759903  4852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.024s	sys 0.024s
I20260709 13:02:07.792636  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:07.714246  4998 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.712658  5067 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.794541  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.804148  1703 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 4.
W20260709 13:02:07.805282  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 85485us
I20260709 13:02:07.805302  1708 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.808116  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.813196  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.641947  4986 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.814116  1704 raft_consensus.cc:2376] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 6. Re-sending same reply.
I20260709 13:02:07.814533  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.721769  5079 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:02:07.715029  4916 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.820641  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.821008  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.820681  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.822356  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.823545  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.828886  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:07.829952  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 77769.1us
I20260709 13:02:07.807627  5004 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:07.622164  4919 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.805627  4955 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:07.833273  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.821308  1707 raft_consensus.cc:2376] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 11. Re-sending same reply.
I20260709 13:02:07.721935  4744 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.840003  1704 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:07.839375  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.840895  1708 raft_consensus.cc:2376] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 11. Re-sending same reply.
I20260709 13:02:07.842098  1704 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:07.819626  5021 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.844718  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.845458  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:07.846278  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:07.848990  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.849820  1631 raft_consensus.cc:2376] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 9. Re-sending same reply.
I20260709 13:02:07.833364  5142 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:07.834847  4998 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:07.856395  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:07.792586  5071 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.816838  5223 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:07.839684  4955 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:07.859964  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.868443  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.869223  4751 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:07.869642  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.870720  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.871459  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:07.880929  4916 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:07.857268  5142 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:07.881891  5142 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.886965  1700 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
W20260709 13:02:07.885062  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 95267.8us
I20260709 13:02:07.855082  5232 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:07.860642  4852 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:07.853597  5021 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:07.895365  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:07.900730  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 108162us
I20260709 13:02:07.900920  5238 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:07.901543  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:07.898030  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:07.901155  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:07.916229  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45337.7us
I20260709 13:02:07.894695  5232 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.945868  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:07.897418  4998 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:07.956434  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.957110  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:07.960768  1632 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:07.965337  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 120704us
I20260709 13:02:07.955945  5004 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:07.972478  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 190210us
I20260709 13:02:07.900058  5110 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:07.979206  1523 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.525s (SENT)
I20260709 13:02:07.979817  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:07.973353  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:07.955170  5232 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:07.985052  5236 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:07.985571  5236 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.985929  5236 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:07.987287  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 85808.9us
I20260709 13:02:07.960707  5223 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:07.987489  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:07.988605  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 103050us
I20260709 13:02:07.993813  2293 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:07.994071  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:07.995795  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:07.957940  5077 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:02:07.983752  5179 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 13:02:08.000190  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:07.992973  1523 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.530s (SENT)
I20260709 13:02:08.003268  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.008378  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:07.981628  5234 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.009461  2292 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:07.997822  5235 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:02:08.012176  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:08.012957  1700 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:08.010183  1632 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:08.009899  2290 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:08.004271  5020 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:07.998106  5021 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:08.023106  5210 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.679s	user 0.003s	sys 0.000s
I20260709 13:02:07.956985  4955 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:07.977306  4852 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:08.039422  1630 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:08.013242  5079 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:08.092475  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.300309  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 297247us
I20260709 13:02:08.094942  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:08.300916  2292 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:08.301623  1702 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.006s
I20260709 13:02:08.301162  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:08.309839  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:08.107911  1700 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:08.310587  1706 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.139973  5230 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.311854  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:08.017002  5248 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.025059  5240 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:08.100140  4789 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.000s
W20260709 13:02:08.309823  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 297352us
I20260709 13:02:08.309974  1702 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:08.323786  1706 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
W20260709 13:02:08.324635  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 453208us
I20260709 13:02:08.325193  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:08.121810  5224 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:02:08.104916  5072 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:08.202291  5139 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.001s
I20260709 13:02:08.332093  1560 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
W20260709 13:02:08.160293  5221 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:02:08.332841  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:08.123492  5229 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:02:08.289712  5110 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.311568  5230 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.123970  4986 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:08.087705  5213 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:02:08.040349  5020 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:08.120910  5247 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:08.335575  1560 pending_rounds.cc:85] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 4
I20260709 13:02:08.122519  5189 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:08.338040  2260 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.826s (SENT)
I20260709 13:02:08.338492  1707 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:08.313390  5079 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.326445  5239 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:08.349756  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 376478us
I20260709 13:02:08.289906  5238 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:08.359925  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:08.326795  4852 consensus_queue.cc:1243] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:02:08.348404  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 321071us
I20260709 13:02:08.362743  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:08.371543  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:08.374213  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.339709  4955 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:08.092208  5095 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.361675  1560 pending_rounds.cc:107] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 10.5
W20260709 13:02:08.414903  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52235.4us
W20260709 13:02:08.415199  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77183.7us
I20260709 13:02:08.400952  1632 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:02:08.415831  1560 raft_consensus.cc:2889] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:08.350111  5019 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.871s	user 0.014s	sys 0.000s
I20260709 13:02:08.401782  5239 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.410606  1555 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:08.419618  5233 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:08.087937  4998 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.424528  5231 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:08.360755  5072 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:08.489261  4686 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:08.433790  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59671.1us
I20260709 13:02:08.489984  4686 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:08.490386  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:08.491806  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.491921  5250 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:08.492087  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2381) took 1806 ms (client timeout 1586 ms). Trace:
I20260709 13:02:08.492594  5250 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:08.492513  1626 rpcz_store.cc:269] 0709 13:02:06.713737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.715882 (+  2145us) service_pool.cc:224] Handling call
0709 13:02:08.491917 (+1776035us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54000dd9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.492018 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336179584}
I20260709 13:02:08.493635  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.452857  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2395) took 1393 ms. Trace:
W20260709 13:02:08.454563  5216 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.644s	user 0.001s	sys 0.000s
I20260709 13:02:08.457583  4766 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:08.416846  1523 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.630s (SENT)
I20260709 13:02:08.498102  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:08.462996  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45947.3us
I20260709 13:02:08.349027  5139 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.330181  5248 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.503582  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2230) took 1212 ms. Trace:
I20260709 13:02:08.124231  5004 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.488946  5072 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.375370  5079 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:08.489403  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:08.441641  5095 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:08.493955  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:08.493706  5250 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.494485  5233 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:08.415309  5247 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:08.497812  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 137987us
I20260709 13:02:08.373199  5189 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.473498  5223 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:08.496930  1635 rpcz_store.cc:276] 0709 13:02:07.059264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.059467 (+   203us) service_pool.cc:224] Handling call
0709 13:02:08.452519 (+1393052us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b540014fd54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.452741 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251809664}
W20260709 13:02:08.508219  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 532) took 1855 ms (client timeout 1708 ms). Trace:
I20260709 13:02:08.509187  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:08.504401  1629 rpcz_store.cc:276] 0709 13:02:07.290948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.299899 (+  8951us) service_pool.cc:224] Handling call
0709 13:02:08.503399 (+1203500us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.503524 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736116096}
I20260709 13:02:08.517326  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.411590  5020 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.494539  5081 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:08.523753  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:08.534953  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.573482  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49826.2us
I20260709 13:02:08.504894  5231 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.538755  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 710) took 1285 ms. Trace:
I20260709 13:02:08.491034  4686 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.536403  5255 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:02:08.411931  5263 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:02:08.560061  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2236) took 1195 ms. Trace:
W20260709 13:02:08.589870  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173051us
I20260709 13:02:08.516183  4974 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:08.560837  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2388) took 1806 ms (client timeout 1830 ms). Trace:
I20260709 13:02:08.506141  5239 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.584815  4919 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:08.593410  1630 rpcz_store.cc:269] 0709 13:02:06.754833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.754980 (+   147us) service_pool.cc:224] Handling call
0709 13:02:08.536470 (+1781490us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003444d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.560775 (+ 24305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871239936}
I20260709 13:02:08.561832  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:08.568765  1701 rpcz_store.cc:269] 0709 13:02:06.653118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.691144 (+ 38026us) service_pool.cc:224] Handling call
0709 13:02:08.507948 (+1816804us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b5400134cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.508128 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338378112}
I20260709 13:02:08.572093  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2233) took 1197 ms. Trace:
I20260709 13:02:08.574409  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:08.603717  1631 rpcz_store.cc:276] 0709 13:02:07.364179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.870484 (+506305us) service_pool.cc:224] Handling call
0709 13:02:08.536868 (+666384us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.561479 (+ 24611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171604864}
I20260709 13:02:08.605186  1631 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:08.605289  4555 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.605746  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 542) took 1403 ms. Trace:
I20260709 13:02:08.605818  5179 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:08.606168  1631 rpcz_store.cc:276] 0709 13:02:07.202622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.604566 (+1401944us) service_pool.cc:224] Handling call
0709 13:02:08.604875 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.605524 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.605695 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.605304  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:08.607242  5248 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.574923  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:08.608034  2292 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.608682  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:08.609256  5245 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:08.609365  1635 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:08.609871  5245 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:08.610056  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2247) took 1114 ms. Trace:
I20260709 13:02:08.610769  5245 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.576596  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 708) took 1324 ms. Trace:
I20260709 13:02:08.612092  5253 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.614624  1701 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:08.581712  2291 rpcz_store.cc:276] 0709 13:02:07.253260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.253491 (+   231us) service_pool.cc:224] Handling call
0709 13:02:07.263102 (+  9611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:07.289468 (+ 26366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.289501 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:07.290053 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:07.414661 (+124608us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b18000f2080. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:07.616340 (+201679us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:07.616378 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.616463 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.523196 (+906733us) raft_consensus.cc:1685] finished
0709 13:02:08.523227 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.538688 (+ 15461us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6907,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":11770,"spinlock_wait_cycles":10231808,"thread_start_us":201500,"threads_started":1,"wal-append.queue_time_us":531052}
I20260709 13:02:08.481210  5019 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.611894  2296 rpcz_store.cc:276] 0709 13:02:07.252566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.252687 (+   121us) service_pool.cc:224] Handling call
0709 13:02:07.263019 (+ 10332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:07.268671 (+  5652us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:07.591910 (+323239us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.815354 (+223444us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:07.835277 (+ 19923us) log.cc:844] Serialized 52 byte log entry
0709 13:02:07.835881 (+   604us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:07.835946 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.836063 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.576293 (+740230us) raft_consensus.cc:1685] finished
0709 13:02:08.576326 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.576516 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":373459,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":22511,"raft.queue_time_us":23524,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":21972,"thread_start_us":486469,"threads_started":2,"wal-append.queue_time_us":244409}
I20260709 13:02:08.585692  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 516) took 1274 ms. Trace:
W20260709 13:02:08.579689  2260 leader_election.cc:336] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.129s (SENT)
I20260709 13:02:08.589622  1634 rpcz_store.cc:276] 0709 13:02:07.365130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.439001 (+ 73871us) service_pool.cc:224] Handling call
0709 13:02:08.537085 (+1098084us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b5400274cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.559988 (+ 22903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56358144}
I20260709 13:02:08.597692  5263 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.579211  4653 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:08.610512  1635 rpcz_store.cc:276] 0709 13:02:07.495364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.608320 (+1112956us) service_pool.cc:224] Handling call
0709 13:02:08.609993 (+  1673us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.599198  5236 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
W20260709 13:02:08.600188  5212 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.005s	user 0.006s	sys 0.002s
I20260709 13:02:08.589488  5095 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.573208  5081 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.603335  1630 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:08.592370  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:08.571048  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61942.8us
W20260709 13:02:08.619622  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48483.8us
W20260709 13:02:08.591636  5232 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.607s	user 0.000s	sys 0.012s
I20260709 13:02:08.592466  4916 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:08.540750  5238 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.621485  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2235) took 1256 ms. Trace:
I20260709 13:02:08.672673  1627 rpcz_store.cc:276] 0709 13:02:07.364861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.417317 (+ 52456us) service_pool.cc:224] Handling call
0709 13:02:08.621260 (+1203943us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b5400296dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.621421 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141293312}
I20260709 13:02:08.617626  1558 rpcz_store.cc:276] 0709 13:02:07.311063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.316735 (+  5672us) service_pool.cc:224] Handling call
0709 13:02:07.336270 (+ 19535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:07.336729 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.336768 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:07.337517 (+   749us) log.cc:844] Serialized 52 byte log entry
0709 13:02:07.566387 (+228870us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:07.792898 (+226511us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.804073 (+ 11175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.804154 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.453778 (+649624us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:08.573029 (+119251us) raft_consensus.cc:1685] finished
0709 13:02:08.585486 (+ 12457us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.585635 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10856,"prepare.queue_time_us":302169,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":63494,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":7568,"spinlock_wait_cycles":23505280,"thread_start_us":393953,"threads_started":2,"wal-append.queue_time_us":358919}
I20260709 13:02:08.607553  1631 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:08.619954  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102697us
W20260709 13:02:08.617667  1523 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.664s (SENT)
W20260709 13:02:08.675135  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57499.2us
W20260709 13:02:08.675704  1523 leader_election.cc:336] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.583s (SENT)
I20260709 13:02:08.676115  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:08.626844  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2240) took 1260 ms (client timeout 1664 ms). Trace:
I20260709 13:02:08.628180  1635 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:02:08.626901  2292 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:08.593215  4974 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:08.650098  5233 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:08.652329  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 158792us
I20260709 13:02:08.648240  5254 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:08.639187  5081 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.590528  5223 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:08.649923  5232 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.621922  5263 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:08.636451  4653 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:08.692931  4653 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.639473  5231 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.695977  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.696106  5231 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.696693  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:08.696924  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:08.697921  5223 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.698855  5231 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:08.660246  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 540) took 1434 ms. Trace:
I20260709 13:02:08.699586  1630 rpcz_store.cc:276] 0709 13:02:07.201932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.602631 (+1400699us) service_pool.cc:224] Handling call
0709 13:02:08.602953 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.636482 (+ 33529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.636704 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.651913  5262 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:02:08.680878  5216 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:08.677755  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2239) took 1311 ms. Trace:
I20260709 13:02:08.624224  4919 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:08.672734  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 150115us
I20260709 13:02:08.674376  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 546) took 1470 ms. Trace:
I20260709 13:02:08.715041  1631 rpcz_store.cc:276] 0709 13:02:07.204086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.607034 (+1402948us) service_pool.cc:224] Handling call
0709 13:02:08.607300 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.674100 (+ 66800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.674319 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.676776  1633 rpcz_store.cc:269] 0709 13:02:07.366341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.416030 (+ 49689us) service_pool.cc:224] Handling call
0709 13:02:08.626625 (+1210595us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400297cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:08.626779 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2642970368}
I20260709 13:02:08.654162  5079 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.688829  5019 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.688616  4974 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.689067  5246 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:02:08.739316  4919 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.688644  5020 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:08.690627  5228 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:08.690915  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.621331  5247 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:08.704056  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.639410  5236 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:08.752357  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.713244  1523 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.687s (SENT)
I20260709 13:02:08.713090  1635 rpcz_store.cc:276] 0709 13:02:07.366004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.626055 (+1260051us) service_pool.cc:224] Handling call
0709 13:02:08.627928 (+  1873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.677455 (+ 49527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.677670 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.716630  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.684582  4955 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.758175  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:08.756103  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:08.765885  1633 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:08.760669  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.721740  5254 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.726068  5240 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:08.773322  1558 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:08.786567  1634 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 10. Ops: []
I20260709 13:02:08.774066  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:08.776117  5243 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:08.788417  5243 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:08.788491  1560 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.789384  5243 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.790438  5243 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:08.793439  5270 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:02:08.794836  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78271.8us
I20260709 13:02:08.796865  5081 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.799163  5081 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:08.801481  5079 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.802115  5079 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.739341  5139 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:08.794967  4974 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:08.807745  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:08.814622  1628 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11.
I20260709 13:02:08.814822  1554 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:08.814918  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 505) took 1814 ms. Trace:
W20260709 13:02:08.816390  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47695.7us
I20260709 13:02:08.751063  5228 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:08.821507  5095 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:08.824808  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111595us
W20260709 13:02:08.824625  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 133794us
I20260709 13:02:08.815599  1556 rpcz_store.cc:276] 0709 13:02:07.000337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.010778 (+ 10441us) service_pool.cc:224] Handling call
0709 13:02:07.029136 (+ 18358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:07.029682 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:07.029722 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:07.030347 (+   625us) log.cc:844] Serialized 52 byte log entry
0709 13:02:07.381263 (+350916us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:07.381502 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.381603 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.539823 (+1158220us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:08.769213 (+229390us) raft_consensus.cc:1685] finished
0709 13:02:08.769250 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.814846 (+ 45596us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":197,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":3485952,"thread_start_us":350412,"threads_started":1,"wal-append.queue_time_us":507422}
W20260709 13:02:08.820854  5142 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.935s	user 0.012s	sys 0.001s
W20260709 13:02:08.829674  1523 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.911s (SENT)
I20260709 13:02:08.836468  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:08.837502  1523 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.978s (SENT)
I20260709 13:02:08.785341  5240 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:08.837893  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.808696  4916 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:08.851228  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 90623.5us
I20260709 13:02:08.857131  1560 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:08.866935  1525 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.664s (SENT)
I20260709 13:02:08.867507  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.854326  5240 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:08.808285  5230 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.831722  4955 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.711320  5263 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.877959  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:08.879644  1554 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
W20260709 13:02:08.882074  1523 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.049s (SENT)
I20260709 13:02:08.854666  5248 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:08.889060  2260 leader_election.cc:336] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.508s (SENT)
I20260709 13:02:08.877157  5079 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.779036  5254 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:08.895285  1674 leader_election.cc:400] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10.
I20260709 13:02:08.895577  1674 leader_election.cc:403] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:02:08.884588  4852 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:08.856631  5142 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:08.889683  5260 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:02:08.900279  5273 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 13:02:08.901525  5276 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:02:08.906422  1524 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.530s (SENT)
I20260709 13:02:08.925659  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:08.908423  4916 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10.
I20260709 13:02:08.912300  5142 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.912902  1626 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:08.916611  1674 leader_election.cc:400] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:08.916477  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:08.938262  1703 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:02:08.938009  1674 leader_election.cc:403] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:08.919701  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:08.923883  1523 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.108s (SENT)
I20260709 13:02:08.891893  5233 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:08.894454  5257 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:08.941823  1706 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:08.943166  5236 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.949970  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:08.902405  4955 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.954016  5148 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:08.957372  5232 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:08.971678  4916 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:08.967356  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1560) took 1872 ms. Trace:
I20260709 13:02:08.974220  5254 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:08.965036  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 142783us
I20260709 13:02:08.966271  1702 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:08.966837  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50406.2us
I20260709 13:02:08.972899  1709 rpcz_store.cc:276] 0709 13:02:07.094878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.114659 (+ 19781us) service_pool.cc:224] Handling call
0709 13:02:07.134238 (+ 19579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.224643 (+ 90405us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:07.625312 (+400669us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.538628 (+913316us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:08.948277 (+409649us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b1800417520. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287
0709 13:02:08.953350 (+  5073us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.966977 (+ 13627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.967036 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:08.967092 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.967170 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.967292 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27635,"prepare.queue_time_us":488815,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":81,"raft.queue_time_us":120792,"raft.run_cpu_time_us":5047,"raft.run_wall_time_us":609053,"spinlock_wait_cycles":8687232,"thread_start_us":549248,"threads_started":3,"wal-append.queue_time_us":212596}
I20260709 13:02:08.896143  5079 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:08.979372  4555 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.967818  5148 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:09.040831  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.048225  5277 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:09.048003  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 109529us
I20260709 13:02:09.016197  5257 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:09.092065  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:09.144716  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:09.145742  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:09.089479  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.143479  5263 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.151823  5274 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:08.894770  5228 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.147119  5240 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.152803  5228 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:09.153775  4751 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:09.154248  4751 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:09.155025  5079 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.162791  5240 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.164840  1560 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:09.040764  4555 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.167856  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.164708  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2241) took 1607 ms. Trace:
W20260709 13:02:09.171654  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 79734.8us
W20260709 13:02:09.176002  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 252115us
I20260709 13:02:09.068595  4653 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.179860  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.180712  1523 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.167s (SENT)
I20260709 13:02:09.170126  1559 rpcz_store.cc:276] 0709 13:02:07.557378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.623928 (+ 66550us) service_pool.cc:224] Handling call
0709 13:02:07.647321 (+ 23393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.844152 (+196831us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:07.975909 (+131757us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.562124 (+586215us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.562412 (+   288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.562456 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:08.562508 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.910639 (+348131us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b1800416260. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:09.163321 (+252682us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b1800416260. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419
0709 13:02:09.164457 (+  1136us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:09.164500 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.164643 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24937,"prepare.queue_time_us":24739,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":3027,"raft.queue_time_us":29943,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":26653,"spinlock_wait_cycles":459863040,"thread_start_us":667855,"threads_started":3,"wal-append.queue_time_us":503478}
I20260709 13:02:09.016477  5189 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.185403  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:09.187335  1524 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.822s (SENT)
I20260709 13:02:09.170593  5021 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.175045  4555 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.191650  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.192312  5081 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:09.170318  5274 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:09.207015  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.177820  5232 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:09.200474  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 333544us
I20260709 13:02:09.201721  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.155156  4751 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.215977  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.216693  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.190744  5233 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:09.218815  1631 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:09.224848  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.208263  5228 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:09.208410  5230 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.267457  5274 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.300925  1635 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:02:09.207057  5179 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.301330  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.344558  1629 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:09.172991  5148 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:09.302625  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 87750.7us
I20260709 13:02:09.346199  1706 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:09.344903  1627 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:02:09.316550  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.221680  5231 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:02:09.332926  1559 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:09.336345  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 148796us
I20260709 13:02:09.316834  5139 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:09.209245  5021 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.187049  4653 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.334460  4766 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:09.347723  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.349035  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.334679  5081 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:09.353518  5110 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.354486  4789 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:09.354905  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.344349  5230 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.346415  4919 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:09.357108  5239 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.000s
I20260709 13:02:09.350889  5179 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.316159  5232 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:09.422788  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 242106us
W20260709 13:02:09.425803  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.349139  4751 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:09.442308  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 87482.7us
I20260709 13:02:09.436292  5148 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:09.444114  1523 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.367s (SENT)
I20260709 13:02:09.444479  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:09.444950  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.447286  5286 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:09.448287  1523 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.373s (SENT)
W20260709 13:02:09.448616  5290 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:02:09.448698  5021 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:09.436158  5226 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:09.450132  5226 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.429464  4789 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:09.452214  5226 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.452152  4789 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.453447  4789 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.454300  5292 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.454797  5292 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.456182  5081 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.456766  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.457062  5292 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:09.457767  5287 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.389s	user 0.001s	sys 0.000s
I20260709 13:02:09.461187  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.461377  5179 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:09.425590  5220 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.462056  5220 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:09.463380  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.464121  5220 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:09.464200  4974 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:02:09.472731  1631 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.465849  1633 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:09.467594  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.468559  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:09.476538  1675 connection.cc:256] RPC call timeout handler was delayed by -1.2424048370000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:02:09.443356  5288 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.103s	user 0.008s	sys 0.001s
W20260709 13:02:09.477162  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1380048370000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:02:09.484539  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1371048370000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
I20260709 13:02:09.484810  1631 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:02:09.485026  1675 connection.cc:256] RPC call timeout handler was delayed by -1.095704837000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:02:09.477633  5019 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:09.480418  5232 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.481395  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:09.483978  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.485731  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:09.485730  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.445287  5243 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:09.486091  5122 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.333381  5257 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.505807  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.501431  1556 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:09.438333  5110 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:09.508809  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64419.2us
I20260709 13:02:09.513810  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.441320  5239 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:09.491504  4555 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:09.511089  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.10935e+06us
I20260709 13:02:09.516153  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:09.443264  4919 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.521301  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.521952  1705 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.515426  4974 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:02:09.508293  5122 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.486254  5292 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:09.524344  4751 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:09.504638  5233 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:09.529382  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43734.3us
W20260709 13:02:09.531993  1557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.005s	sys 0.017s
W20260709 13:02:09.532361  1557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.005s	sys 0.017s
I20260709 13:02:09.491999  5019 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.536609  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.537289  1629 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:09.537330  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.517036  4986 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:09.538502  1674 connection.cc:256] RPC call timeout handler was delayed by -1.9231096770000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:02:09.527741  5241 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:09.541180  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 518) took 1200 ms. Trace:
I20260709 13:02:09.542474  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.550205  1703 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:09.549450  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63815.8us
I20260709 13:02:09.551663  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.548527  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.552721  1555 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:09.553851  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:09.512837  5243 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.530898  5247 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:09.535389  5239 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:09.531622  5122 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:09.549794  1557 rpcz_store.cc:276] 0709 13:02:08.341378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.341511 (+   133us) service_pool.cc:224] Handling call
0709 13:02:08.371891 (+ 30380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:08.420045 (+ 48154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.420076 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:08.420702 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 13:02:08.693527 (+272825us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b18002b59a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:08.849461 (+155934us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:08.891185 (+ 41724us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.532687 (+641502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.532761 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:09.532803 (+    42us) raft_consensus.cc:1685] finished
0709 13:02:09.540960 (+  8157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.541115 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7128,"prepare.queue_time_us":76653,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":13448,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":6207,"spinlock_wait_cycles":48922880,"thread_start_us":814883,"threads_started":3,"wal-append.queue_time_us":437619}
I20260709 13:02:09.556249  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.557163  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.567435  1559 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:09.560669  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.562551  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:09.568675  1554 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.569033  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.533911  5248 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.566044  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.569815  1558 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:09.544317  4789 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:09.572958  1555 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:09.549422  4555 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.582512  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.582924  1555 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:02:09.583448  1555 pending_rounds.cc:85] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 4
I20260709 13:02:09.583385  1630 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:09.544338  5292 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:09.575723  1705 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:09.579645  1629 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:09.542976  4986 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:09.584616  1555 pending_rounds.cc:107] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:02:09.565115  5247 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:09.593694  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.594166  1555 raft_consensus.cc:2889] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:09.594774  1554 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:09.543267  5240 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:02:09.596993  1557 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:09.604041  5110 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.604810  5233 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:09.606314  5294 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:02:09.571931  5241 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:09.592777  4916 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:09.574553  4789 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:09.581372  4766 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:09.619906  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.636556  1629 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:02:09.636732  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.609517  1555 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:02:09.595700  5289 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:02:09.625023  5300 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:02:09.629065  1557 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:09.599469  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:09.648368  1554 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:09.596328  5122 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.620697  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.657524  1630 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:09.626513  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.658084  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.658572  1551 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:09.639199  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.592564  4986 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.601539  5247 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.659632  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:09.652944  1703 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:09.662317  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.663228  1632 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:09.664424  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:09.666224  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 115049us
I20260709 13:02:09.667511  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.635598  4789 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.670341  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.670413  1555 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:09.669246  1707 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:09.671252  1632 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:09.671734  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:09.672274  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.672443  1557 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.672025  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:09.608348  5292 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.677953  1551 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:09.675800  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:09.676508  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:09.647205  5081 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:09.647430  4653 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.681288  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.682379  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2226) took 2442 ms. Trace:
I20260709 13:02:09.682133  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:09.682801  1634 rpcz_store.cc:276] 0709 13:02:07.238090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.619833 (+1381743us) service_pool.cc:224] Handling call
0709 13:02:08.630643 (+ 10810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.787219 (+156576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.680328 (+893109us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001afc54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:09.680503 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357885696}
I20260709 13:02:09.665748  5122 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.689615  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:09.691104  1552 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:09.694201  5189 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:09.700875  2260 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.654s (SENT)
W20260709 13:02:09.701777  2260 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.708s (SENT)
I20260709 13:02:09.680298  5019 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:09.705206  1632 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:09.705642  1554 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:09.705701  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:09.709460  1629 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:09.677258  5301 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.712035  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 114360us
I20260709 13:02:09.713549  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:09.717541  1634 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:09.677639  5299 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:09.684028  5230 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.743327  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.746519  1701 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.744675  4986 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.748095  1556 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:09.747946  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:09.748981  4986 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.752044  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.752759  1702 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:09.753700  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.754383  1702 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:09.755120  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.719128  5148 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:09.756965  5245 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:09.758142  5245 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:09.758493  5245 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:09.759078  4916 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:09.759958  1704 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:09.760171  5081 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:09.760879  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.762059  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.762562  1709 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.762614  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.763576  5189 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:09.763953  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82748.4us
I20260709 13:02:09.764915  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.765262  1629 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:09.766363  5263 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:09.721092  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.721491  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.735217  1557 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:09.769073  1706 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:09.724929  1633 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:09.708711  5004 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.708074  5304 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:09.773129  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.773393  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.775214  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.775938  1706 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:09.776430  1702 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:09.776780  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.777463  1706 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:09.769073  1705 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:09.777944  1703 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:09.717429  4744 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.780755  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.781845  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.783525  1675 leader_election.cc:400] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9.
I20260709 13:02:09.763988  5303 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.785730  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.793298  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.784744  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.795331  1705 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.784485  5300 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:09.792485  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.792658  2297 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:09.759303  5245 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.713819  5019 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:09.770991  5081 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.839001  5081 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.839504  5081 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.794051  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.840005  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.840736  2293 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:09.841048  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.841636  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.842557  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.843014  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.844326  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.844640  2290 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:09.845516  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.845652  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.846174  2293 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:09.846755  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.847319  2290 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.800103  1701 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:09.848275  5304 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.848644  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.848758  5304 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.849395  1701 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.851121  5304 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:09.794238  5149 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:02:09.847448  2293 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:09.801724  5250 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:09.853678  2293 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:09.855150  2293 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:09.857524  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.858350  2296 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:09.774794  4766 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:09.802083  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.764516  5299 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.802603  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.863473  2292 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:09.803301  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.772526  5004 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.773633  5243 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:09.808323  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.812906  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:09.813712  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:09.815181  1524 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.367s (SENT)
I20260709 13:02:09.872535  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:09.815503  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.816251  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:09.828469  5302 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:02:09.878615  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.829728  5292 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.830657  5303 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.868860  1709 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:09.883363  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.883991  1700 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:09.884202  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.824923  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.884984  1709 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:09.885138  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60319.8us
I20260709 13:02:09.831845  5122 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.832330  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.833675  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.833786  2294 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:09.841709  1702 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:09.772766  4653 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:02:09.865701  1705 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:09.892249  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.873893  1708 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.781227  4744 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.894114  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.894404  4744 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.894712  2295 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:09.895295  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.895676  5243 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.896207  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.896478  5292 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.864780  5299 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:09.897884  4919 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:09.899351  5304 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:09.900105  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 534) took 2712 ms. Trace:
I20260709 13:02:09.900481  1626 rpcz_store.cc:276] 0709 13:02:07.187334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.535335 (+1348001us) service_pool.cc:224] Handling call
0709 13:02:09.899885 (+1364550us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54000cc6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:09.900036 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711878400}
I20260709 13:02:09.871815  5004 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:09.902431  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.888978  5248 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.903273  5248 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.904249  1632 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:09.904896  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.905325  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.906363  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:09.829523  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 549) took 1836 ms (client timeout 1972 ms). Trace:
I20260709 13:02:09.882042  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.886073  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:09.910601  1703 raft_consensus.cc:2376] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
I20260709 13:02:09.720356  5241 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:09.890184  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:09.891080  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.914482  2292 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:09.892366  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 77211.1us
W20260709 13:02:09.909391  1628 rpcz_store.cc:269] 0709 13:02:07.992659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.489080 (+496421us) service_pool.cc:224] Handling call
0709 13:02:09.829333 (+1340253us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54002760d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:09.829463 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670536576}
W20260709 13:02:09.918361  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.919159  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:09.893894  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81081.4us
I20260709 13:02:09.920540  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.924491  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.925732  1706 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:09.934175  1701 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:09.935910  1701 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:09.878858  5231 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:09.892850  5250 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.937261  5250 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.937379  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.937707  5250 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.938674  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:09.939282  2296 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:09.940871  1558 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:09.882753  4766 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.938874  4789 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:09.974586  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.945978  1702 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 6.
I20260709 13:02:09.952320  2288 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:09.968649  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:09.969875  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:09.970543  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50136.9us
I20260709 13:02:09.970795  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:09.972314  2289 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:10.036341  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:10.018556  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.044742  1703 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:10.036652  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:10.045950  1551 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.046150  1708 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:02:10.047008  2292 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:10.056681  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.061744  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93147.9us
I20260709 13:02:10.064895  1700 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:09.720106  5301 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:09.944630  4986 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:09.972985  5228 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
W20260709 13:02:10.075526  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 169205us
I20260709 13:02:10.081197  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:10.069547  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99128.2us
I20260709 13:02:10.082479  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:09.972019  4653 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:10.100895  1707 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:09.971894  5079 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:10.103479  1703 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:02:10.103865  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.103528  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:09.974915  5299 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:10.068181  5240 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:09.964582  5245 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:10.072150  5292 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:10.069213  5300 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:10.125842  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69252.2us
W20260709 13:02:10.121608  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 79538.8us
I20260709 13:02:10.098881  4653 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.106184  5079 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.039857  4766 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.136279  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:10.119700  5122 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.943385  5243 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.178s
I20260709 13:02:10.072357  5238 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:10.116592  5232 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:10.114598  5299 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:10.158164  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54385.5us
I20260709 13:02:10.158430  1551 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:10.165711  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.077343  5297 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:02:10.120230  5240 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.127777  5148 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:10.164387  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:10.127418  5257 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:10.104111  5264 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:10.152099  4653 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:10.159238  5263 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:10.156435  5122 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:10.168208  5299 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:10.207903  5309 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:02:10.208058  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:10.208868  2289 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
W20260709 13:02:10.205708  5314 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:10.208292  5248 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.173046  5240 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.212706  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:10.155393  5079 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:10.219632  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53983us
I20260709 13:02:10.227476  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.223299  2260 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.537s (SENT)
I20260709 13:02:10.178738  5292 raft_consensus.cc:2764] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:10.182444  5300 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:10.163650  5228 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:02:10.215240  5122 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.179553  5243 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:10.178416  5303 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:10.204480  5232 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 13:02:10.251861  5308 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:10.254349  5004 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.224007  5263 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:10.235158  5304 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:10.232304  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 151199us
I20260709 13:02:10.247870  5257 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:10.288437  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:10.289090  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 209798us
I20260709 13:02:10.299629  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.301595  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 61253us
I20260709 13:02:10.303182  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.309666  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:10.263162  4789 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.311512  2290 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:10.314194  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.318653  2294 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:10.319250  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.327672  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 151554us
I20260709 13:02:10.329854  1675 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:10.311681  4986 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:10.301241  5250 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.334266  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.335049  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:10.335719  1631 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:10.267247  5304 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.336812  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.312701  5320 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:10.319150  5243 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:10.338157  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:10.326007  4789 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:10.344257  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.322546  5264 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:10.343271  1631 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:10.324497  5301 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:10.397641  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:10.396862  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:10.402794  1628 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:10.411355  1553 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:10.415625  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:10.335623  5300 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:10.345754  5304 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.347411  5004 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.344995  5257 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:10.467097  4916 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:10.484948  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 171144us
W20260709 13:02:10.481277  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 149316us
I20260709 13:02:10.398726  4789 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:10.490646  1558 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:10.345584  5246 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.147s
W20260709 13:02:10.521520  1523 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.732s (SENT)
I20260709 13:02:10.522017  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:10.399443  5149 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:10.344859  5292 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
I20260709 13:02:10.342527  5148 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:02:10.470328  5297 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.345438  5320 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.340538  5019 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.140s
I20260709 13:02:10.471614  5323 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.525434  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 188672us
I20260709 13:02:10.562949  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:10.410107  5230 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:02:10.471510  4766 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:10.527500  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111984us
I20260709 13:02:10.399269  4919 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:02:10.534576  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.535465  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.544090  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.537658  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.584254  1633 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:10.555195  5081 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.666s	user 0.005s	sys 0.005s
W20260709 13:02:10.524329  5322 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:02:10.549615  1631 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:10.519845  5248 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:10.521524  5244 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:10.523516  4789 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:10.558522  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.412093  5263 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.566696  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.492785  5257 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:10.569499  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:10.616241  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:10.547600  5243 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:10.580798  1705 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:10.516350  5301 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:10.583612  1632 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:10.585538  1634 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:10.593415  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:10.493850  5326 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:02:10.599815  1700 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:10.553650  5149 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:10.555980  5320 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:10.553947  5297 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.600255  5250 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.622092  1709 raft_consensus.cc:2376] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9. Re-sending same reply.
I20260709 13:02:10.610222  5332 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.634251  1703 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:10.540575  4986 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:10.635255  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.636585  1708 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:10.688476  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.688778  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.634421  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.682463  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.623124  5330 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:10.605870  5081 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:10.690037  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 168542us
I20260709 13:02:10.623260  5244 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:10.680316  5300 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.623231  5301 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:10.688481  5332 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.700929  1708 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:10.720973  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.722151  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:10.689808  5243 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:10.723145  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:10.713250  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.582227  5323 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.714596  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:10.729050  1633 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:10.703815  1700 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:10.730872  1700 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:10.719385  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:10.720976  2295 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:10.714052  1709 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:02:10.733594  1709 pending_rounds.cc:85] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 5
I20260709 13:02:10.734009  1709 pending_rounds.cc:107] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:10.734269  1709 raft_consensus.cc:2889] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:10.733474  1708 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:10.714984  5247 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.041s	user 0.011s	sys 0.001s
I20260709 13:02:10.726994  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:10.735718  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.711273  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:10.737679  1706 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:10.741655  1635 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:10.742416  1633 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:02:10.751840  5081 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.755867  1705 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:10.711704  5220 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:10.756428  5338 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.757515  1628 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:10.757237  5338 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.759012  1628 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:10.760354  5336 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:10.760591  1635 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:10.756660  5220 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:10.761436  5337 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.761837  5220 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.724771  5230 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.716259  5148 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.691152  5333 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:02:10.785509  4789 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:10.780090  5334 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:02:10.780638  5244 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:10.781688  5345 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:02:10.783993  5220 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.784864  5338 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:10.786823  5324 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:02:10.713039  5257 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.705873  5301 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:10.787739  5321 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:02:10.710843  5300 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.841946  5300 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:10.788115  5149 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.844127  5257 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.846169  1703 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:02:10.789152  5122 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.004s
W20260709 13:02:10.791950  5327 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:02:10.762020  5337 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.785542  5148 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.858723  5122 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.869599  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 553) took 1138 ms. Trace:
I20260709 13:02:10.869998  1702 rpcz_store.cc:276] 0709 13:02:09.731323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.895045 (+163722us) service_pool.cc:224] Handling call
0709 13:02:10.869406 (+974361us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54001cb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:10.869533 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114820992}
I20260709 13:02:10.870383  5148 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:10.870858  5122 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:10.871939  1458 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 5 to 8, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:10.873613  1626 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:10.875041  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2311) took 1084 ms. Trace:
I20260709 13:02:10.875607  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.876516  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:10.877483  4744 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.983s	user 0.009s	sys 0.001s
I20260709 13:02:10.877909  4744 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.877060  1700 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:10.875416  1707 rpcz_store.cc:276] 0709 13:02:09.797771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.833444 (+ 35673us) service_pool.cc:224] Handling call
0709 13:02:10.874851 (+1041407us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001b7454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:10.874984 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440983296}
I20260709 13:02:10.818277  5230 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.737713  4916 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:10.701996  5248 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:10.727301  5243 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:10.837570  2295 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:02:10.726799  5246 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:02:10.892153  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:10.867286  1708 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:10.883013  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:10.890347  1706 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:10.746620  5247 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:10.879321  5301 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.898883  4744 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.891602  5264 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.902215  5319 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:10.884655  5230 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.903227  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:10.908206  5337 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:10.911474  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 725) took 1002 ms. Trace:
I20260709 13:02:10.912341  5297 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:10.913807  1560 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:02:10.915872  5324 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:10.921006  5248 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.937474  5323 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:10.938905  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:10.949796  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66848.3us
I20260709 13:02:10.984256  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.948371  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 383043us
W20260709 13:02:10.964388  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 566821us
I20260709 13:02:10.968916  2291 rpcz_store.cc:276] 0709 13:02:09.909409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.909575 (+   166us) service_pool.cc:224] Handling call
0709 13:02:09.910009 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.179292 (+269283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.179335 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.179860 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.544040 (+364180us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:10.544100 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.842332 (+298232us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:10.842382 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.911129 (+ 68747us) raft_consensus.cc:1685] finished
0709 13:02:10.911156 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.911403 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15787,"raft.queue_time_us":10265,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":32257,"spinlock_wait_cycles":597902464,"thread_start_us":306210,"threads_started":1,"wal-append.queue_time_us":675734}
W20260709 13:02:10.965369  5240 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.715s	user 0.003s	sys 0.007s
I20260709 13:02:10.981551  1709 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:10.985836  1558 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:10.968614  5079 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.003s
I20260709 13:02:10.993819  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.986891  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.994567  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.992781  5304 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.003793  5241 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:11.004261  5304 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.007290  5300 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.008298  1704 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:11.009413  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.011256  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:10.992568  4653 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.776s	user 0.013s	sys 0.000s
I20260709 13:02:11.015774  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:11.017630  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 272553us
I20260709 13:02:11.029383  1633 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:11.046317  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.050020  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:10.998868  5337 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.000562  5230 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.062827  1560 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:11.001053  5323 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:11.036870  5299 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.830s	user 0.012s	sys 0.000s
I20260709 13:02:11.055249  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 864) took 1139 ms. Trace:
W20260709 13:02:11.061496  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44986.8us
I20260709 13:02:11.065220  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:11.004279  5241 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:10.995621  5246 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:02:11.064050  2293 rpcz_store.cc:276] 0709 13:02:09.915708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.915956 (+   248us) service_pool.cc:224] Handling call
0709 13:02:11.015570 (+1099614us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b5800060c98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:11.055164 (+ 39594us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55547648}
I20260709 13:02:11.030475  5297 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:10.980309  5264 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.072381  5342 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:11.077375  1458 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 4 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:11.078154  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 723) took 1190 ms. Trace:
I20260709 13:02:11.082233  5263 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:11.050823  5343 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:11.085078  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2320) took 1284 ms. Trace:
I20260709 13:02:11.094677  1458 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:11.089700  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:10.996168  5240 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.072374  2295 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:11.099022  1704 rpcz_store.cc:276] 0709 13:02:09.801093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.067303 (+266210us) service_pool.cc:224] Handling call
0709 13:02:10.873197 (+805894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.016413 (+143216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.084272 (+ 67859us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54001cb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:11.084888 (+   616us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54001cb6d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:11.085019 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162664064}
I20260709 13:02:11.092267  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.009845  5303 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:02:10.980408  5319 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.119901  5241 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.068835  5230 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.120877  5319 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.121011  5320 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.121542  5338 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:11.121733  5301 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:11.122517  5245 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.996s	user 0.002s	sys 0.009s
I20260709 13:02:11.067297  5344 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:11.123186  5320 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.123355  5344 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:11.111189  5263 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.007817  5300 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:10.998823  5079 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.049338  4653 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.110730  2295 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:11.147549  2260 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.914s (SENT)
I20260709 13:02:11.071471  5297 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:11.190089  1559 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:02:11.125690  1630 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:11.143270  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:11.110328  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 231339us
I20260709 13:02:11.089376  5148 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:02:11.066030  5337 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.156097  1629 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:11.094370  2297 rpcz_store.cc:276] 0709 13:02:09.887845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.904501 (+ 16656us) service_pool.cc:224] Handling call
0709 13:02:09.973852 (+ 69351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.974151 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:10.221101 (+246950us) log.cc:844] Serialized 14 byte log entry
0709 13:02:10.472503 (+251402us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.077641 (+605138us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b180040e9a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:11.077807 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.077859 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:11.077910 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.077976 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.078096 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7526,"prepare.queue_time_us":90434,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":612116,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":81,"spinlock_wait_cycles":19210112,"thread_start_us":497179,"threads_started":2,"wal-append.queue_time_us":342759}
I20260709 13:02:11.192620  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:11.068897  5299 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.198593  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:11.238998  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 138000us
I20260709 13:02:11.438512  1702 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:11.200265  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:11.225535  1709 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:11.436349  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 220102us
W20260709 13:02:11.440811  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 240646us
I20260709 13:02:11.307186  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:11.239326  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 174181us
I20260709 13:02:11.441864  2288 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:11.442028  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:11.437053  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.443688  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.214260  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:11.227043  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.446702  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:11.104609  5349 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:11.448789  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 234617us
I20260709 13:02:11.136188  5320 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:11.449885  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:11.195079  5297 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:11.459829  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:11.494159  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.408205  5357 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:02:11.296784  5353 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:02:11.502851  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:11.214018  5347 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.145s	user 0.005s	sys 0.000s
W20260709 13:02:11.505041  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45324.7us
I20260709 13:02:11.510759  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.299716  4653 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.448053  5301 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.448206  4744 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:11.384076  5264 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.512707  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.456923  5079 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:11.460724  5356 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:02:11.520282  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.427999  5230 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.523064  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:11.455046  5244 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:02:11.517761  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:11.456555  5240 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.504824  5296 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:11.505049  5343 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:11.317879  4916 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:11.457808  5223 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:11.504323  5253 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:11.504623  5345 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.438278  5257 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:11.225669  4912 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.289s
I20260709 13:02:11.225152  5272 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.290s
I20260709 13:02:11.225457  5189 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.289s
I20260709 13:02:11.437260  5241 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:11.509519  5292 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:11.524396  2293 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:11.545789  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.588515  5233 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:11.591150  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 863) took 1687 ms. Trace:
I20260709 13:02:11.653813  2294 rpcz_store.cc:276] 0709 13:02:09.906295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.939925 (+ 33630us) service_pool.cc:224] Handling call
0709 13:02:10.318455 (+378530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.324018 (+  5563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.590936 (+1266918us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b540040c250. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:11.591081 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869747456}
I20260709 13:02:11.518708  5301 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.598483  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 561) took 1809 ms. Trace:
I20260709 13:02:11.604212  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:11.658675  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 112959us
I20260709 13:02:11.659264  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1644) took 1246 ms. Trace:
I20260709 13:02:11.604501  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.604764  4751 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:11.606281  5220 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:11.609961  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2421) took 1075 ms. Trace:
I20260709 13:02:11.611899  1458 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 5 to 8, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:11.230360   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:11.518556  4766 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:11.506546  5320 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.622316  2288 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:11.637727  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 964) took 1332 ms. Trace:
I20260709 13:02:11.641856  5238 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:11.515690  4986 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:02:11.545776  4653 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:02:11.648886  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1642) took 1237 ms. Trace:
I20260709 13:02:11.528453  5248 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:11.527804  5323 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:11.567294  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.680584  1556 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:11.653131  1706 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:11.198305  5344 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.659731  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.658296  1701 rpcz_store.cc:276] 0709 13:02:09.789555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.965579 (+176024us) service_pool.cc:224] Handling call
0709 13:02:10.877435 (+911856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.877786 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.877820 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.878304 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.968952 (+ 90648us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:11.068260 (+ 99308us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.068941 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.069017 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.141123 (+ 72106us) raft_consensus.cc:1685] finished
0709 13:02:11.141156 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.598229 (+457073us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54001b7450. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:11.598396 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":385141,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":3219,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":8187,"spinlock_wait_cycles":1871405952,"thread_start_us":165339,"threads_started":2,"wal-append.queue_time_us":193004}
I20260709 13:02:11.659693  1554 rpcz_store.cc:276] 0709 13:02:10.413193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.413413 (+   220us) service_pool.cc:224] Handling call
0709 13:02:11.604044 (+1190631us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:11.659189 (+ 55145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9983744}
I20260709 13:02:11.663546  1630 rpcz_store.cc:276] 0709 13:02:10.534143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.534268 (+   125us) service_pool.cc:224] Handling call
0709 13:02:11.609749 (+1075481us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54002747d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:11.609894 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247540992}
I20260709 13:02:11.509225  4919 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:02:11.657567  5233 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:11.525801  5230 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:11.665940  2289 rpcz_store.cc:276] 0709 13:02:10.305284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.796412 (+491128us) service_pool.cc:224] Handling call
0709 13:02:10.796826 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.088136 (+291310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.088164 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.088615 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.516208 (+427593us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:11.516273 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.516454 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.637378 (+120924us) raft_consensus.cc:1685] finished
0709 13:02:11.637415 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.637639 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290902,"raft.queue_time_us":291945,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":426798,"threads_started":1,"wal-append.queue_time_us":532406}
I20260709 13:02:11.662637  5240 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:11.647536  5343 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:11.651019  5296 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:11.675583  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.678270  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 228451us
I20260709 13:02:11.664706  4916 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.667703  4766 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.667198  5220 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:11.676913  1555 rpcz_store.cc:276] 0709 13:02:10.411116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.411225 (+   109us) service_pool.cc:224] Handling call
0709 13:02:10.413358 (+  2133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.677361 (+264003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.677412 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.678052 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.874441 (+196389us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4800002958. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:10.945139 (+ 70698us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:10.945189 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.946209 (+  1020us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.641017 (+694808us) raft_consensus.cc:1685] finished
0709 13:02:11.641044 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.648805 (+  7761us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66719,"raft.queue_time_us":66810,"raft.run_cpu_time_us":4309,"raft.run_wall_time_us":390270,"spinlock_wait_cycles":12283648,"thread_start_us":358272,"threads_started":2,"wal-append.queue_time_us":305116}
W20260709 13:02:11.690063  2262 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.515s (SENT)
I20260709 13:02:11.666739  5292 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:11.675825  5362 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.224s	user 0.001s	sys 0.003s
I20260709 13:02:11.668853  5238 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:11.648511  5223 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.696197  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:11.699208  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.707614  1555 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.650154  5345 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.704295  4986 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:11.706297  5264 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.667855  4751 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.710187  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.703629  2288 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:11.719729  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 216945us
I20260709 13:02:11.679852  5367 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:11.717278  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.722185  1554 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:11.722456  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.663192  5257 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.678821  5349 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:11.700515  2262 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.573s (SENT)
I20260709 13:02:11.679665  5248 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.706066  4916 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.686787  5323 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.722945  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 223389us
I20260709 13:02:11.726161  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.723651  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:11.692724  5228 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:11.737151  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.716750  5263 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:11.758836  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 244545us
I20260709 13:02:11.758934  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.760574  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:11.733120  5248 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:11.751471  5337 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.739017  5272 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:11.771903  5343 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.779188  5332 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:11.725237  5264 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.781700  5264 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.781625  5349 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:11.749691  5364 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:02:11.763654  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63187.3us
I20260709 13:02:11.760895  5371 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:02:11.763450  5253 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:11.763335  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:11.786623  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.766723  5345 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.768160  4916 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:11.736971  5323 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:11.772374  5272 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:11.782464  1552 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:11.725741  4986 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.762967  2288 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:11.752032  5230 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:11.784955  2262 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.645s (SENT)
I20260709 13:02:11.808954  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.765601  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 963) took 1461 ms. Trace:
I20260709 13:02:11.809809  2296 rpcz_store.cc:276] 0709 13:02:10.304227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.630567 (+326340us) service_pool.cc:224] Handling call
0709 13:02:10.689090 (+ 58523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.156062 (+466972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.190298 (+ 34236us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.190755 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.645939 (+455184us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:11.645974 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.646027 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.765372 (+119345us) raft_consensus.cc:1685] finished
0709 13:02:11.765392 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.765538 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482714,"raft.queue_time_us":499935,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":320085,"threads_started":1,"wal-append.queue_time_us":440762}
I20260709 13:02:11.788511  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:11.789597  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:11.792097  1555 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 5.
I20260709 13:02:11.813453  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.814229  1555 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:11.815294  1551 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:11.734802  4912 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:11.793309  5368 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.754396  5244 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:11.795419  5223 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.798074  5300 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:11.798954  5326 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:11.799566  1557 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:11.723415  5233 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.845131  5233 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:11.802786  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1646) took 1388 ms. Trace:
I20260709 13:02:11.788527  5220 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.847931  1552 rpcz_store.cc:276] 0709 13:02:10.414136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.414244 (+   108us) service_pool.cc:224] Handling call
0709 13:02:11.782087 (+1367843us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:11.782285 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.802530 (+ 20245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.802731 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9758080}
I20260709 13:02:11.706712  5079 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:02:11.848874  5220 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:11.849336  5079 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:11.849699  5079 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:11.850468  5079 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.782917  5365 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:11.794360  4751 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:11.822113  5292 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.822641  5342 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:11.762744  5238 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:11.837914  5360 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.573s	user 0.003s	sys 0.007s
I20260709 13:02:11.840798  5369 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:11.841845  5253 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:02:11.842820  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57915.7us
W20260709 13:02:11.894850  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.842948  5323 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.895865  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.843477  5370 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:11.844429  5326 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:11.731379  5367 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.854712  1556 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:11.788878  5189 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:11.859527  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:11.868557  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:11.930008  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:11.930784  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:11.873905  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:11.874504  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:11.851929  5365 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:11.876632  1704 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:11.851583  5079 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:11.876209  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:11.932972  1702 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.881501  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1643) took 1470 ms. Trace:
I20260709 13:02:11.933504  1559 rpcz_store.cc:276] 0709 13:02:10.411409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.413307 (+  1898us) service_pool.cc:224] Handling call
0709 13:02:10.483612 (+ 70305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.194379 (+710767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.879614 (+685235us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:11.881433 (+  1819us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624165120}
W20260709 13:02:11.882813  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.890498  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1607) took 1207 ms. Trace:
I20260709 13:02:11.897310  1630 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:11.857967  5300 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.899290  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:11.900503  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:11.856686  5368 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:11.875943  5240 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:11.852025  5320 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:11.895576  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.914845  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 215706us
W20260709 13:02:11.915228  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 162177us
I20260709 13:02:11.917657  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.968143  2295 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:11.953100  5228 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:11.969106  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:11.970137  5228 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.970944  5228 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.971907  4986 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:11.977278  5326 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:11.915563  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:11.924510  1553 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:11.927584  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:11.980330  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52854.4us
I20260709 13:02:11.925959  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:11.981173  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55296.3us
I20260709 13:02:11.981359  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:11.934747  1705 rpcz_store.cc:276] 0709 13:02:10.683620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.689752 (+  6132us) service_pool.cc:224] Handling call
0709 13:02:10.690023 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.756328 (+ 66305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.890254 (+1133926us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b5400166054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:11.890440 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118358400}
I20260709 13:02:11.897823  4912 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:11.984843  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69372us
I20260709 13:02:11.988588  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.919130  5231 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:02:11.964560  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43795.4us
I20260709 13:02:11.961094  1702 raft_consensus.cc:2368] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:11.970211  5366 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:11.988620  2288 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:11.990775  1703 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:11.899014  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:11.992188  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.993065  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.946238  1708 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:11.993933  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2426) took 1443 ms. Trace:
I20260709 13:02:12.015216  2290 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:12.015372  1629 rpcz_store.cc:276] 0709 13:02:10.550661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.550809 (+   148us) service_pool.cc:224] Handling call
0709 13:02:10.581807 (+ 30998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.445240 (+863433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.993649 (+548409us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54002756d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:11.993873 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316771200}
I20260709 13:02:11.995020  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:11.998873  4974 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:11.999051  5332 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:11.999790  5338 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:12.000470  5236 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:11.911427  5304 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:02:12.001734  5122 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:12.006688  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1638) took 1597 ms. Trace:
I20260709 13:02:12.008282  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.981905  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:12.010471  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 121390us
I20260709 13:02:11.991461  5326 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:11.926273  5369 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:12.031345  2295 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:12.036335  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.001389  5228 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:12.038095  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:12.039397  1558 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:11.801455  5296 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:11.963073  5301 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:12.042721  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.064885  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:12.065630  2291 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.046717  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 962) took 1743 ms. Trace:
I20260709 13:02:12.067966  2292 rpcz_store.cc:276] 0709 13:02:10.302948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.613668 (+310720us) service_pool.cc:224] Handling call
0709 13:02:10.613992 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.085668 (+471676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.085698 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.086164 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.110952 (+ 24788us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:11.110994 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.111062 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.837199 (+726137us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:12.036326 (+199127us) raft_consensus.cc:1685] finished
0709 13:02:12.036349 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.046637 (+ 10288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":581802,"raft.queue_time_us":473616,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":7248128,"thread_start_us":24406,"threads_started":1,"wal-append.queue_time_us":178500}
I20260709 13:02:12.017705  2289 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:02:12.047924  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.050335  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 68507.9us
I20260709 13:02:12.073794  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.074637  1702 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:11.983722  5330 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:12.075513  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.076237  1702 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:12.076464  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.076980  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.077693  1702 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:12.077615  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.078516  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.078699  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.079073  1702 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:12.079217  1706 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:11.896529  5342 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.080107  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:12.080672  1705 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:12.044366  1551 rpcz_store.cc:276] 0709 13:02:10.409790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.410532 (+   742us) service_pool.cc:224] Handling call
0709 13:02:11.776760 (+1366228us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:11.815074 (+ 38314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.815843 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.006479 (+190636us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b540036bfd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:12.006621 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97113728}
I20260709 13:02:12.083072  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:12.083597  1551 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:12.084467  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:12.084998  1551 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:02:12.085736  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:12.086674  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:12.080096  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:12.087430  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:12.089119  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:12.090159  1551 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:12.091137  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:12.092576  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:12.057955  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:12.027232  5332 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.027951  5019 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:12.094226  1556 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:12.122278  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84256.8us
W20260709 13:02:12.121079  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 126597us
W20260709 13:02:12.096233  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127737us
I20260709 13:02:12.096542  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:12.057678  1554 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:12.097558  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.126012  1706 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 5: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:12.127876  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:12.128588  2296 raft_consensus.cc:2376] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
I20260709 13:02:12.091754  1551 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.101438  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:12.130110  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:12.130719  1705 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:12.102401  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 55256us
W20260709 13:02:12.058549  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 57706.3us
I20260709 13:02:12.119268  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:12.119879  1559 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:02:12.133211  1559 pending_rounds.cc:85] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:02:12.133459  1559 pending_rounds.cc:107] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:02:12.133801  1559 raft_consensus.cc:2889] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:11.990275  5300 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.135286  1598 leader_election.cc:400] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:12.135746  1598 leader_election.cc:403] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:12.131989  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:12.060093  5257 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:12.145094  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.601s (SENT) [suppressed 37 similar messages]
I20260709 13:02:12.145949  1551 raft_consensus.cc:2368] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:12.146847  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:12.018276  5374 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:02:12.119836  5263 raft_consensus.cc:2764] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:12.121599  5332 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:12.058007  5345 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:02:12.174785  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:11.954962  5272 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.176430  5272 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:12.122776  5378 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:02:12.180128  5236 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:12.159746  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:12.161512  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.140645  5371 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:12.188868  1630 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:12.170660  1703 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:12.172164  1556 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:12.184383  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.185101  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.186092  1626 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:02:12.040050  5338 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.186326  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.119998  5189 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:12.194209  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.200799  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.200953  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.202232  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.204013  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.180881  5236 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:12.167994  5072 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:12.213824  2291 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:12.213945  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.138751  5004 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:02:12.209334  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.210292  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.211861  1634 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:12.137861  5232 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:12.144565  5244 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:12.225301  1628 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:12.228673  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.140383  5366 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:02:12.188943  5371 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.232568  5371 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:12.212606  5241 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.004s
W20260709 13:02:12.229564  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83555.8us
I20260709 13:02:12.197063  5228 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.237751  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 78084.3us
I20260709 13:02:12.238575  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.153056  5248 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:12.247684  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115802us
I20260709 13:02:12.248037  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.252867  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.260891  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.254240  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.255532  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.264977  1635 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:12.256909  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.266746  1626 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:12.292543  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.293478  1631 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:12.293999  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.294543  1599 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:12.297101  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58610.5us
I20260709 13:02:12.238760  5378 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:12.236641  5382 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:12.298600  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.301975  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101100us
I20260709 13:02:12.082320  5342 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:12.306488  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:12.319020  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.298882  5236 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:12.303475  5241 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:12.303754  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.263438  5323 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:12.302637  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.307999  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:12.166913  5368 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.259047  5228 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.307124  5385 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:12.333352  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.356400  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.368991  1626 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:12.346401  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 253951us
W20260709 13:02:12.371826  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.373950  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:12.325527  5343 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:12.324460  5236 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:12.393492  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:12.381407  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48149.9us
W20260709 13:02:12.396544  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 102088us
I20260709 13:02:12.394671  1631 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:12.330675  5241 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:12.354362  5323 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:12.322335  5366 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:02:12.316323  5382 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.413164  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.413874  2289 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:12.269119  5369 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.322494  5374 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:02:12.351230  5244 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:12.402465  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.351181  5231 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:12.404480  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:12.434170  2293 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:12.391417  5189 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.437592  1626 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:12.350984  5230 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:02:12.330199  5391 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:02:12.404057  5367 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:12.181823  5373 consensus_queue.cc:1175] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b2c3934407d422495728f84374c6b53" responder_term: 10 status { last_received { term: 7 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.9-9.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:12.350415  5400 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:02:12.417250  5004 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.447974  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45571.3us
W20260709 13:02:12.434067  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.435070  5300 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:12.471572  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:12.432504  5342 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.407369  5257 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:12.417344  5019 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:12.417520  5232 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:02:12.405004  5384 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:02:12.451977  4916 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:12.449076  5231 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:12.538205  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86532.1us
W20260709 13:02:12.549082  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 152702us
I20260709 13:02:12.554020  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:12.545100  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:12.513391  5344 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.560868  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 425708us
I20260709 13:02:12.506489  5383 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:12.593592  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76439.3us
W20260709 13:02:12.597091  1701 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.003s	sys 0.011s
W20260709 13:02:12.597539  1701 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.003s	sys 0.011s
W20260709 13:02:12.598199  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 442095us
I20260709 13:02:12.602741  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.604476  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:12.541996  5004 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.517936  5241 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.553695  5342 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.609963  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:12.642059  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47477.5us
I20260709 13:02:12.979830  2290 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:12.788336  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.558658  5231 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.790876  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 728) took 1352 ms. Trace:
W20260709 13:02:12.764008  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.006s
W20260709 13:02:12.993921  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.003s	sys 0.011s
I20260709 13:02:12.993793  2293 rpcz_store.cc:276] 0709 13:02:11.438040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.664861 (+226821us) service_pool.cc:224] Handling call
0709 13:02:11.672642 (+  7781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.451255 (+778613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.790602 (+339347us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400771ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:12.790805 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750763392}
W20260709 13:02:12.994630  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 747041us
W20260709 13:02:12.792583  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2254) took 1738 ms (client timeout 1673 ms). Trace:
W20260709 13:02:12.996125  1553 rpcz_store.cc:269] 0709 13:02:11.054694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.604241 (+549547us) service_pool.cc:224] Handling call
0709 13:02:12.792373 (+1188132us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001edcd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:12.792500 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606858624}
I20260709 13:02:12.607071  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:12.998440  1634 raft_consensus.cc:2376] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
I20260709 13:02:12.984859  1632 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:12.796129  1700 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:13.000396  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1619) took 1127 ms. Trace:
I20260709 13:02:12.843629  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 560) took 2126 ms. Trace:
I20260709 13:02:13.001967  1633 rpcz_store.cc:276] 0709 13:02:10.717324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.747746 (+ 30422us) service_pool.cc:224] Handling call
0709 13:02:10.748252 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.045723 (+297471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.843328 (+1797605us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54002760d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:12.843559 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645804928}
I20260709 13:02:12.795511  1552 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:12.606416  1635 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:13.000916  1700 rpcz_store.cc:276] 0709 13:02:11.872522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.872653 (+   131us) service_pool.cc:224] Handling call
0709 13:02:12.795744 (+923091us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:12.795961 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.000084 (+204123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.000327 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3556224}
I20260709 13:02:13.009876  1553 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:12.975944  5388 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.520s	user 0.001s	sys 0.000s
I20260709 13:02:12.762856  5343 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:13.025287  5220 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.176s	user 0.005s	sys 0.006s
W20260709 13:02:12.650918  5393 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:12.766826  5232 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.028292  5232 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:12.609036  5257 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:12.651934  5397 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:02:12.648660  5367 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.384s
I20260709 13:02:13.034040  5367 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.034266  5297 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.521s	user 0.010s	sys 0.002s
I20260709 13:02:13.034572  5367 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.034710  5297 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:12.650169  5398 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:12.978173  5387 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.438s	user 0.001s	sys 0.001s
I20260709 13:02:12.954927  5228 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:12.979867  5272 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.780s	user 0.011s	sys 0.000s
I20260709 13:02:12.973344  5407 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:12.662369  4916 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.037879  4912 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.035s	user 0.016s	sys 0.000s
I20260709 13:02:13.038672  4912 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:13.038486  4916 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:12.649207  5380 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.411s	user 0.001s	sys 0.001s
I20260709 13:02:12.561360  5296 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.481s
I20260709 13:02:12.659451  4974 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:13.037945  5407 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.037262  5233 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.192s	user 0.011s	sys 0.000s
I20260709 13:02:13.025049  5241 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:13.041527  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:13.040791  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 883) took 1049 ms. Trace:
W20260709 13:02:13.014001  5072 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.779s	user 0.004s	sys 0.009s
W20260709 13:02:12.658776  5079 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 13:02:13.037405  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1626) took 1109 ms. Trace:
I20260709 13:02:12.751181  5385 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:12.606782  5248 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:12.662115  5019 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.593067  5344 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:12.789182  4789 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.154558  1707 rpcz_store.cc:276] 0709 13:02:11.927731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.966014 (+ 38283us) service_pool.cc:224] Handling call
0709 13:02:11.966326 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.966671 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.966702 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.991337 (+ 24635us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.172906 (+181569us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:12.172948 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.173019 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.037129 (+864110us) raft_consensus.cc:1685] finished
0709 13:02:13.037153 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.037312 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12746,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":26526,"thread_start_us":131036,"threads_started":1,"wal-append.queue_time_us":197195}
I20260709 13:02:12.651855  5122 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:13.138470  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 143034us
I20260709 13:02:12.795101  5021 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:13.148110  2292 rpcz_store.cc:276] 0709 13:02:11.991901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.070483 (+ 78582us) service_pool.cc:224] Handling call
0709 13:02:12.070946 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.120486 (+ 49540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.120538 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.136034 (+ 15496us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.555068 (+419034us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:12.555116 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.555190 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.040515 (+485325us) raft_consensus.cc:1685] finished
0709 13:02:13.040552 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.040713 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82497,"raft.queue_time_us":92044,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":368066,"threads_started":1,"wal-append.queue_time_us":827000}
I20260709 13:02:13.040874  1560 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.152714  2296 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:13.179500  5122 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:13.103636  5408 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:13.125579  5413 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:02:13.131484  5412 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.123428  5422 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:13.148344  5407 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:13.171025  2291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.002s	sys 0.022s
W20260709 13:02:13.125579  5378 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.820s	user 0.008s	sys 0.000s
I20260709 13:02:13.173763  1555 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:13.151047  5232 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:13.146974  5371 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.914s	user 0.012s	sys 0.001s
I20260709 13:02:13.084002  4974 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:13.147303  5361 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:02:13.039726  5342 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.136976  5297 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:13.118948  5228 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.169551  5072 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:13.036271  5398 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:13.176631  5350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.006s	sys 0.011s
W20260709 13:02:13.213888  5332 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.009s	user 0.004s	sys 0.006s
I20260709 13:02:13.178758  5248 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.180128  5292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.003s	sys 0.013s
I20260709 13:02:13.125125  5414 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:13.176455  5019 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:13.643451  5326 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.549s	user 0.005s	sys 0.008s
I20260709 13:02:13.228103  4916 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.419709  5386 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:13.251493  5323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.013s	sys 0.020s
I20260709 13:02:13.667586  1707 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:13.667869  5414 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.670679  5414 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:13.295712  5292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.003s	sys 0.018s
I20260709 13:02:13.383633  4912 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.403703  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.383e+06us
I20260709 13:02:13.238530  5344 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:13.672624  4912 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.280726  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:13.674368  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 393686us
I20260709 13:02:13.441039  1556 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:13.236863  5004 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.434098  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2257) took 2341 ms. Trace:
W20260709 13:02:13.284505  5350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.632s	user 0.007s	sys 0.021s
W20260709 13:02:13.528280  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.011s	sys 0.011s
I20260709 13:02:13.407179  1560 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:13.215586  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:13.440625  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 972) took 1268 ms. Trace:
W20260709 13:02:13.695963  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 315158us
I20260709 13:02:13.675438  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:13.664341  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:13.413820  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2263) took 2218 ms (client timeout 1974 ms). Trace:
I20260709 13:02:13.701799  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:13.252116  2291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.002s	sys 0.022s
W20260709 13:02:13.702001  1554 rpcz_store.cc:269] 0709 13:02:11.195545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.738549 (+543004us) service_pool.cc:224] Handling call
0709 13:02:13.413551 (+1675002us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540030a7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:13.413718 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501160832}
I20260709 13:02:13.703176  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 976) took 1380 ms. Trace:
I20260709 13:02:13.703379  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:13.704020  1705 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:13.703617  2291 rpcz_store.cc:276] 0709 13:02:12.323157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.323319 (+   162us) service_pool.cc:224] Handling call
0709 13:02:12.323614 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.323857 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:12.404634 (+ 80777us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.702706 (+1298072us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.702894 (+   188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:13.702943 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.703010 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.703122 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":317781,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":38319,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":7311,"thread_start_us":1367601,"threads_started":2}
W20260709 13:02:13.575290  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.680017  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.014s	sys 0.011s
I20260709 13:02:13.699965  2296 rpcz_store.cc:276] 0709 13:02:12.171821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.173826 (+  2005us) service_pool.cc:224] Handling call
0709 13:02:12.174206 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.245093 (+1070887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.440268 (+195175us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:13.440536 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249173888}
W20260709 13:02:13.444808  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.469870  2292 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:02:13.708686  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 236943us
I20260709 13:02:13.706058  1706 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:13.699296  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.012s	sys 0.016s
I20260709 13:02:13.699055  1558 rpcz_store.cc:276] 0709 13:02:11.092276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.841806 (+749530us) service_pool.cc:224] Handling call
0709 13:02:11.842138 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.060931 (+218793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.433743 (+1372812us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540036d3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:13.434005 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431995648}
W20260709 13:02:13.707928  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.014s	sys 0.011s
I20260709 13:02:13.643625  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1662) took 1502 ms. Trace:
W20260709 13:02:13.713204  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.10477e+06us
I20260709 13:02:13.713483  1551 rpcz_store.cc:276] 0709 13:02:12.141640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.655878 (+514238us) service_pool.cc:224] Handling call
0709 13:02:12.656178 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.656402 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:13.136565 (+480163us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.642748 (+506183us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.643042 (+   294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.643109 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:13.643203 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.643349 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.643540 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":516041,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":98,"raft.queue_time_us":31932,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":29286,"spinlock_wait_cycles":2118912,"thread_start_us":492747,"threads_started":1}
I20260709 13:02:13.577984  2293 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:13.383872  5296 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:13.717535  1554 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:02:13.718703  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2267) took 1558 ms. Trace:
W20260709 13:02:13.258596  5369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.007s	sys 0.016s
I20260709 13:02:13.720001  1554 rpcz_store.cc:276] 0709 13:02:12.160530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.716976 (+1556446us) service_pool.cc:224] Handling call
0709 13:02:13.717314 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.718449 (+  1135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.718646 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.717653  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 502148us
I20260709 13:02:13.279563  5398 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:13.667656  5323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.021s	sys 0.035s
I20260709 13:02:13.444446  5428 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.395884  5409 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.286933  5394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.001s	sys 0.031s
I20260709 13:02:13.257920  5378 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:13.729681  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:13.470364  5421 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.731168  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:13.236109  5416 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:13.727144  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:13.348263  5384 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:13.260114  5371 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:13.216075  5385 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.281373  5332 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:13.280637  5231 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:13.772125  5394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.015s	sys 0.032s
I20260709 13:02:13.559478  5241 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.735268  1558 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:13.232813  5367 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.224188  5408 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:13.802145  1630 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:13.346393  5247 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:13.697187  4986 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.716475  5344 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:13.264391  4974 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:13.726119  5369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.007s	sys 0.031s
I20260709 13:02:13.715476  5326 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:13.781558  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50505.3us
I20260709 13:02:13.667625  5386 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:13.279670  5228 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.730355  5296 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.283555  5248 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:13.735266  5292 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.797186  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 542) took 2914 ms (client timeout 1885 ms). Trace:
W20260709 13:02:13.849774  1557 rpcz_store.cc:269] 0709 13:02:10.883007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.902864 (+ 19857us) service_pool.cc:224] Handling call
0709 13:02:13.796947 (+2894083us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54002f9254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:13.797103 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913195904}
I20260709 13:02:13.850968  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:13.735288  5323 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:13.797992  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 547) took 1477 ms. Trace:
I20260709 13:02:13.807325  5433 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.732060  5424 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:13.737802  5079 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.732362  5337 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:13.576712  5407 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:13.800958  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71418.4us
W20260709 13:02:13.802445  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93607us
I20260709 13:02:13.771634  1705 raft_consensus.cc:2955] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:13.216607  5122 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:13.215255  5021 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:13.773339  5416 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:13.816128  5439 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:13.817426  5019 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:13.780306  5421 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.425824  5236 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.401290  5243 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.785113  5398 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:13.228622  5412 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.785357  5414 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.346745  5343 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.829977  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:13.804520  5427 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:02:13.675741  4912 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:13.796789  5426 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.775254  5409 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.851783  1557 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:13.809836  5378 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.805888  5408 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:13.853878  1558 rpcz_store.cc:276] 0709 13:02:12.320740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.731581 (+1410841us) service_pool.cc:224] Handling call
0709 13:02:13.734911 (+  3330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.797695 (+ 62784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.797928 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:13.798899  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:13.857677  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.317940  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:14.319193  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 520530us
I20260709 13:02:13.862906  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:14.320238  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:14.320878  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 458029us
I20260709 13:02:13.772307  5428 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.839403  5004 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:13.810225  5241 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:13.877561  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 150528us
I20260709 13:02:14.327435  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:13.878568  5431 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:13.879181  5231 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:13.879413  5419 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:13.864810  5433 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:13.785866  5384 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:13.868528  5425 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.351s	user 0.001s	sys 0.000s
I20260709 13:02:13.842629  5326 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.863523  5443 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:13.861892  5337 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:14.009388  5296 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:13.889417  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64893.7us
W20260709 13:02:13.880990  5348 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.665s	user 0.011s	sys 0.011s
I20260709 13:02:13.922677  5238 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:13.229046  5422 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.257364  5452 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:13.868310  5079 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.865258  5424 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.321861  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:14.160678  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.323698  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 342107us
I20260709 13:02:14.327468  5378 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:14.450814  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.888967  5122 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:13.982726  1629 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:13.931008  5368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.014s	sys 0.022s
I20260709 13:02:13.919960  5394 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:13.995749  5423 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:02:14.331221  1601 leader_election.cc:400] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9.
I20260709 13:02:13.925662  5373 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:14.489403  5373 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:14.332155  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:14.384918  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:13.983489  5439 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.491178  5439 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.491899  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:13.923998  5451 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:14.493170  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:14.493350  5424 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:14.001199  5435 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:02:13.988066  5416 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:14.387631  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.050s	sys 0.087s
W20260709 13:02:14.399780  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.000s	sys 0.035s
I20260709 13:02:14.011188  5407 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.221841  5299 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.317302  5228 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.275336  5421 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.404320  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:14.280228  5332 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.232102  5371 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.325641  5350 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.519134  5416 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:13.993419  5432 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.141373  5019 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.520560  5019 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.522684  5019 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.332599  4912 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:14.141373  5223 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.523628  5223 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:14.395030  5240 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.006s	sys 0.021s
I20260709 13:02:14.528541  5422 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.493369  5439 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.529039  5326 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.332912  5233 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:14.529700  5233 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.530148  5233 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:14.530509  5433 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:13.857909  5292 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.979935  4916 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.327589  5248 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.386584  5459 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:14.388567  5448 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.421732  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 94260.3us
I20260709 13:02:13.921187  5414 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.389250  5458 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:14.426302  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97778.1us
I20260709 13:02:14.399016  5081 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:02:14.409231  5443 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.933059  5021 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:13.993012  5323 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.385987  5004 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.401326  5231 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:02:14.487449  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158432us
W20260709 13:02:14.480676  5368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.040s	sys 0.048s
W20260709 13:02:14.481556  5348 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.482s	user 0.012s	sys 0.011s
I20260709 13:02:14.498382  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:14.000988  5454 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:14.501194  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.050s	sys 0.088s
W20260709 13:02:14.501286  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.000s	sys 0.035s
W20260709 13:02:14.484640  5389 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.498s	user 0.004s	sys 0.000s
I20260709 13:02:14.517344  1558 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:14.525867  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 121619us
I20260709 13:02:14.524034  5447 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:14.528398  5223 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.493398  5338 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:13.981377  5426 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.806941  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1650) took 2754 ms. Trace:
I20260709 13:02:14.807399  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 548) took 2486 ms. Trace:
I20260709 13:02:14.405431  5337 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.519313  5350 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.845624  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 549) took 2524 ms. Trace:
I20260709 13:02:14.528848  5365 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.809368  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:14.770187  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 184797us
W20260709 13:02:14.528872  5240 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.006s	sys 0.021s
I20260709 13:02:14.811686  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:14.856849  5343 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:14.812311  5448 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.813270  5004 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.813867  5021 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:14.815232  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.816438  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:14.807950  1558 rpcz_store.cc:276] 0709 13:02:12.321032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.494076 (+2173044us) service_pool.cc:224] Handling call
0709 13:02:14.517176 (+ 23100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.807118 (+289942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.807340 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.817410  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2295) took 2215 ms. Trace:
I20260709 13:02:14.818423  5414 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.699230  5433 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:14.807466  1552 rpcz_store.cc:276] 0709 13:02:12.052374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.093963 (+ 41589us) service_pool.cc:224] Handling call
0709 13:02:12.795271 (+701308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.003461 (+208190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.806662 (+1803201us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54001edcd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:14.806870 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533802240}
W20260709 13:02:14.697916  5435 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:14.820070  5081 consensus_queue.cc:1243] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:14.820401  5469 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:14.826249  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.495746  5452 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.888942  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:14.890756  1552 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
W20260709 13:02:14.856923  5455 consensus_queue.cc:1175] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 9 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:14.808017  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:14.534426  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:14.841097  5385 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.841192  2290 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:14.899441  1629 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:14.899561  1706 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:14.848031  1557 rpcz_store.cc:276] 0709 13:02:12.321299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.498108 (+2176809us) service_pool.cc:224] Handling call
0709 13:02:14.498465 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:14.507445 (+  8980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.513935 (+  6490us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:14.514529 (+   594us) log.cc:844] Serialized 52 byte log entry
0709 13:02:14.845170 (+330641us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:14.845223 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.845317 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.845371 (+    54us) raft_consensus.cc:1685] finished
0709 13:02:14.845388 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.845543 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":10801,"thread_start_us":76591,"threads_started":1,"wal-append.queue_time_us":289696}
I20260709 13:02:14.900450  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:14.846213  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:14.901070  1629 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:14.866917  5241 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.857607  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:14.903923  1632 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:14.858198  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:14.856947  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:14.806211  5457 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 13:02:14.805759  5463 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:02:14.511953  5412 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.493640  5451 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:14.863231  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46859.1us
I20260709 13:02:14.846561  4916 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.533160  5424 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:14.528438  5079 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.912379  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1628) took 2892 ms. Trace:
I20260709 13:02:14.839218  5458 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:14.867273  5365 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:14.879071  5433 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:14.866255  5228 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:14.519979  5432 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:14.901818  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55751.1us
I20260709 13:02:14.812194  5454 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:14.928597  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.927994  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:14.905823  5466 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:02:14.906003  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:14.944617  1706 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:14.906503  5081 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:14.905153  1634 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:14.866434  5409 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.909315  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:14.863680  1627 rpcz_store.cc:276] 0709 13:02:12.601845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.658842 (+ 56997us) service_pool.cc:224] Handling call
0709 13:02:14.498234 (+1839392us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b58001f6898. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:14.817312 (+319078us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16982272}
I20260709 13:02:14.910919  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:14.872551  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:14.912799  1708 rpcz_store.cc:276] 0709 13:02:12.019939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.021043 (+  1104us) service_pool.cc:224] Handling call
0709 13:02:12.021487 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.022288 (+   801us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:12.022329 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.027916 (+  5587us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.245516 (+217600us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:12.245579 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.245654 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.014678 (+1769024us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b24001d4878. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb93609d 00007f70ce56810e 00007f70ce57f88c 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221
0709 13:02:14.903383 (+888705us) raft_consensus.cc:1685] finished
0709 13:02:14.903405 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.912295 (+  8890us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4942,"raft.queue_time_us":385,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":11986,"spinlock_wait_cycles":211334912,"thread_start_us":124816,"threads_started":1,"wal-append.queue_time_us":358068}
I20260709 13:02:14.911806  5385 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.818920  5443 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.948452  1558 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:14.950911  5432 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.945078  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:14.831190  5233 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:14.940415  5426 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.941792  5452 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.942171  4916 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:14.943164  5448 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:14.926703  4686 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:14.898519  5428 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:14.954226  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.917969  5458 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:14.915424  5378 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:14.958204  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48992.2us
I20260709 13:02:14.957867  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 551) took 1221 ms. Trace:
I20260709 13:02:15.009186  1635 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:15.013422  1632 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:14.968348  5409 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:15.118865  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.119544  5409 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.119920  5246 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.120131  5409 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:15.067431  5337 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.120429  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 878) took 3819 ms. Trace:
I20260709 13:02:15.120505  5246 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.121034  2297 rpcz_store.cc:276] 0709 13:02:11.300921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.645915 (+344994us) service_pool.cc:224] Handling call
0709 13:02:11.646220 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.733950 (+ 87730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.733980 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.734019 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:11.734050 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.120148 (+3386098us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:15.120201 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.120336 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201326976}
I20260709 13:02:15.122507  5337 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:15.122879  5246 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:14.951742  5238 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.002s
I20260709 13:02:15.003964  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1629) took 2972 ms. Trace:
I20260709 13:02:15.060292  1706 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:15.135177  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1632) took 1416 ms. Trace:
I20260709 13:02:14.965971  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:15.133249  1703 rpcz_store.cc:276] 0709 13:02:12.031330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.033498 (+  2168us) service_pool.cc:224] Handling call
0709 13:02:12.033805 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.189676 (+155871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.003664 (+2813988us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54003e2454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:15.003892 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313835392}
I20260709 13:02:15.114112  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:15.119609  5472 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:02:15.087705  5454 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:15.125392  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.116415  1558 rpcz_store.cc:276] 0709 13:02:13.736289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.956884 (+1220595us) service_pool.cc:224] Handling call
0709 13:02:14.957225 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.957462 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.957500 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.957556 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:14.957596 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.957659 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.957776 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.127379  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1627) took 3148 ms. Trace:
I20260709 13:02:15.022490  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 568) took 1041 ms. Trace:
I20260709 13:02:14.993407  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.118916  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.135641  1706 rpcz_store.cc:276] 0709 13:02:13.719079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.134294 (+1415215us) service_pool.cc:224] Handling call
0709 13:02:15.134641 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.134828 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.134857 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.134903 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:15.134937 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.135000 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.135115 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:15.020134  5464 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:02:14.955263  5223 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.177s
I20260709 13:02:15.023931  4974 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.149331  5338 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:15.139377  5323 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.088828  5385 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:14.952082  5231 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.200s
I20260709 13:02:15.140128  5246 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.370304  1634 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:15.370779  5246 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.371356  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.371901  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:15.372454  1558 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:15.263737  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2341) took 1884 ms. Trace:
I20260709 13:02:15.068053  5350 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.373548  5246 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:15.186203  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.370087  1709 rpcz_store.cc:276] 0709 13:02:11.978930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.982831 (+  3901us) service_pool.cc:224] Handling call
0709 13:02:12.027682 (+ 44851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.040520 (+ 12838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.040554 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.049877 (+  9323us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.304467 (+254590us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.304514 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.304588 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.119391 (+2814803us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b2400414d88. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:02:15.126966 (+  7575us) raft_consensus.cc:1685] finished
0709 13:02:15.126996 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.127288 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1660,"raft.queue_time_us":1482,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":44952,"spinlock_wait_cycles":206385536,"thread_start_us":165502,"threads_started":1,"wal-append.queue_time_us":393498}
I20260709 13:02:15.376989  1635 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:15.378247  1635 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:15.376602  1709 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:15.381525  1632 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:15.374542  1628 rpcz_store.cc:276] 0709 13:02:13.980759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.980906 (+   147us) service_pool.cc:224] Handling call
0709 13:02:13.992907 (+ 12001us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:14.482956 (+490049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.482986 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:14.496064 (+ 13078us) log.cc:844] Serialized 78 byte log entry
0709 13:02:14.861393 (+365329us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:14.861429 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.861492 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.022239 (+160747us) raft_consensus.cc:1685] finished
0709 13:02:15.022260 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.022423 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2068,"raft.queue_time_us":333675,"spinlock_wait_cycles":9581952,"thread_start_us":314680,"threads_started":2,"wal-append.queue_time_us":464791}
W20260709 13:02:15.368767  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 402860us
W20260709 13:02:15.319936  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.374722  1458 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:15.364459  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1653) took 3305 ms. Trace:
I20260709 13:02:15.387802  1556 rpcz_store.cc:276] 0709 13:02:12.059208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.209095 (+149887us) service_pool.cc:224] Handling call
0709 13:02:13.440849 (+1231754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.676322 (+235473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.364145 (+1687823us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540030a7d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:15.364368 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832007424}
W20260709 13:02:15.369851  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.966471  5228 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:15.010052  5402 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:15.405813  1706 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:15.259696  5300 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:15.382905  1632 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:15.152617  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2266) took 2992 ms. Trace:
I20260709 13:02:15.409436  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.158946  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2264) took 2999 ms. Trace:
I20260709 13:02:15.168233  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:15.385798  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 271810us
I20260709 13:02:15.392431  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.373281  1707 rpcz_store.cc:276] 0709 13:02:13.379467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.379607 (+   140us) service_pool.cc:224] Handling call
0709 13:02:13.379932 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.668302 (+288370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.263397 (+1595095us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:15.263660 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660662528}
I20260709 13:02:15.403276  2293 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:15.021404  5233 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.414311  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.415544  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.416218  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:15.252544  5408 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.866s	user 0.015s	sys 0.000s
I20260709 13:02:14.963922  5421 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:15.411104  2291 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:15.418272  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:15.409157  1553 rpcz_store.cc:276] 0709 13:02:12.160286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.011086 (+850800us) service_pool.cc:224] Handling call
0709 13:02:13.137803 (+126717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.168611 (+ 30808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.168662 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.169887 (+  1225us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.425863 (+1255976us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b180042c760. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:14.495212 (+ 69349us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:14.495249 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.495315 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.152246 (+656931us) raft_consensus.cc:1685] finished
0709 13:02:15.152269 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.152531 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37984,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":35165,"spinlock_wait_cycles":47919360,"thread_start_us":69165,"threads_started":1,"wal-append.queue_time_us":1562740}
I20260709 13:02:14.983400  5292 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:15.153762  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:15.156467  5460 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.674s	user 0.002s	sys 0.001s
I20260709 13:02:15.417842  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.443873  2292 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:15.157214  5443 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:15.422303  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.422670  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.419407  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:15.448434  2293 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:15.152247  5451 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:15.151932  5382 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:15.427371  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.459950  2289 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:15.390090  5424 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:15.379251  5433 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:15.461463  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.461557  5243 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:15.461953  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.462910  5365 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.003s
I20260709 13:02:15.463917  5326 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.464277  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:15.464496  5326 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.465471  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.466122  5122 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.466599  5326 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:15.468645  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.442903  5238 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:15.470360  5232 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.470507  5228 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:15.470893  5232 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.410357  1559 rpcz_store.cc:276] 0709 13:02:12.159692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.987358 (+827666us) service_pool.cc:224] Handling call
0709 13:02:13.797653 (+810295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.815189 (+1017536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.815217 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.815676 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.876646 (+ 60970us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:14.876704 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.157821 (+281117us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002f9254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:15.157874 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.157925 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:15.157940 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.158753 (+   813us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002f9250. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:15.158882 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1112,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":386,"spinlock_wait_cycles":4081808256,"thread_start_us":60531,"threads_started":1,"wal-append.queue_time_us":60977}
I20260709 13:02:15.472681  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 574) took 1767 ms. Trace:
I20260709 13:02:15.472970  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.446312  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:15.474229  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.412711  5300 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:15.450635  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.451123  5443 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:15.448458  5240 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:15.147661  4686 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:15.456471  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1649) took 3404 ms (client timeout 3363 ms). Trace:
I20260709 13:02:15.418722  5402 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:15.457669  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.1358e+06us
I20260709 13:02:15.410302  5338 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:15.470773  5453 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:15.473047  1705 rpcz_store.cc:276] 0709 13:02:13.705195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.706820 (+  1625us) service_pool.cc:224] Handling call
0709 13:02:13.707102 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.711716 (+  4614us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:13.984132 (+272416us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.423824 (+439692us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.258191 (+834367us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:15.453012 (+194821us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b18003aa920. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70d28b858f 00007f70cb8f7571 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd
0709 13:02:15.456081 (+  3069us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.456631 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.456815 (+   184us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:15.456858 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.456941 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.472472 (+ 15531us) raft_consensus.cc:1685] finished
0709 13:02:15.472490 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.472625 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1052918,"prepare.queue_time_us":26009,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":1136246,"raft.run_cpu_time_us":25287,"raft.run_wall_time_us":1438678,"spinlock_wait_cycles":67307648,"thread_start_us":365951,"threads_started":1,"wal-append.queue_time_us":538377}
I20260709 13:02:15.513767  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:15.429462  5408 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.475692  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.515269  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.517669  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.518599  1632 raft_consensus.cc:2410] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:15.476467  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.443969  1553 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:15.520126  1635 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:15.520314  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 552) took 1783 ms. Trace:
I20260709 13:02:15.519475  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.520751  1553 rpcz_store.cc:276] 0709 13:02:13.736650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.443174 (+1706524us) service_pool.cc:224] Handling call
0709 13:02:15.443572 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.520050 (+ 76478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.520256 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.521921  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.478848  1558 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:15.481530  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.523121  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:15.523397  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.482455  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 567) took 1502 ms. Trace:
I20260709 13:02:15.484970  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.490386  5238 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:15.442705  5344 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:02:15.491205  5343 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:15.493396  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:15.494450  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.494302  5300 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.428519  5421 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:15.496253  1560 rpcz_store.cc:269] 0709 13:02:12.052017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.052123 (+   106us) service_pool.cc:224] Handling call
0709 13:02:15.456225 (+3404102us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:15.456371 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087143808}
I20260709 13:02:15.496774  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 555) took 1759 ms. Trace:
I20260709 13:02:15.504458  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:15.458226  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.504912  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1633) took 1785 ms. Trace:
I20260709 13:02:15.554637  1703 rpcz_store.cc:276] 0709 13:02:13.722598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.136631 (+1414033us) service_pool.cc:224] Handling call
0709 13:02:15.385102 (+248471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.504487 (+119385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.504526 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.504572 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:15.504603 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.504713 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.504849 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119033,"raft.queue_time_us":119940,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6712}
I20260709 13:02:15.556744  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.557538  1703 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:15.505113  2293 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:15.469434  5458 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.515273  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.560717  1705 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:02:15.475140  1633 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:15.410559  5323 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.524309  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.566900  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.552578  1559 rpcz_store.cc:276] 0709 13:02:13.737640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.479679 (+1742039us) service_pool.cc:224] Handling call
0709 13:02:15.491480 (+ 11801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.491689 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.491715 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.496444 (+  4729us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:15.496504 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.496577 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.496703 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.529721  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.569326  1632 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:15.512725  5433 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:15.591387  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.592041  1554 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:15.520751  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:15.528676  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:15.533746  5424 raft_consensus.cc:2764] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:02:15.496371  5443 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:15.598940  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.600198  5455 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:15.600751  1627 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:02:15.600816  5455 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.601334  1627 pending_rounds.cc:85] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 7
I20260709 13:02:15.601640  1627 pending_rounds.cc:107] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:02:15.539182  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.602365  1628 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:15.539968  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.537731  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:15.604669  1635 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:15.466794  5122 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.609970  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.612609  2297 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:15.562693  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.614816  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:15.570482  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.545291  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:15.490697  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.617887  2292 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:15.572805  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.537813  4653 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:15.593406  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64872.9us
I20260709 13:02:15.524829  1631 rpcz_store.cc:276] 0709 13:02:13.980454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.980616 (+   162us) service_pool.cc:224] Handling call
0709 13:02:13.992770 (+ 12154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.996616 (+  3846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.996664 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.016653 (+ 19989us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.459170 (+1442517us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b1800061960. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1
0709 13:02:15.459370 (+   200us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:15.459416 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.459495 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.459544 (+    49us) raft_consensus.cc:1685] finished
0709 13:02:15.459559 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.482390 (+ 22831us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":332803,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":56674,"spinlock_wait_cycles":47268608,"thread_start_us":105800,"threads_started":1,"wal-append.queue_time_us":287709}
I20260709 13:02:15.637768  1556 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:15.601504  5455 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:15.638576  5345 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:15.639181  5345 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.592482  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.639668  5345 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:15.605865  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 101486us
I20260709 13:02:15.641693  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:15.606490  1627 raft_consensus.cc:2889] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:15.636541  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 91365.6us
I20260709 13:02:15.506156  5382 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.567449  5460 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:15.541286  5459 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:15.558568  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.619300  1558 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:15.610818  2289 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:15.552280  5469 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:15.551498  4974 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:02:15.569079  5021 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:02:15.620146  2295 raft_consensus.cc:2376] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 9. Re-sending same reply.
I20260709 13:02:15.643750  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:15.537634  5233 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:02:15.541666  5365 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:02:15.591696  5300 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:15.644840  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:15.565984  5421 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.660274  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.598955  4766 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.608366  5231 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:15.595075  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74560.9us
I20260709 13:02:15.541486  5453 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.635202  5304 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.636229  5238 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:15.645668  4653 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.670086  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:15.678611  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 207998us
I20260709 13:02:15.597436  5402 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.681083  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:15.655226  5409 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.004s
I20260709 13:02:15.647641  5232 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.687143  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.700743  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.748100  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:15.748723  1703 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:15.704144  5469 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:15.661319  5382 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.701980  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 571) took 2000 ms. Trace:
I20260709 13:02:15.604614  5433 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.595866  5230 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:02:15.663270  5426 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.668396  5470 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.668102  5296 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:15.679353  4912 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.679069  5385 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.704665  5449 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:15.701928  5326 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:15.732824  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:15.754838  5019 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:15.551991  5373 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:15.740288  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 265374us
W20260709 13:02:15.749248  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68306.7us
I20260709 13:02:15.714099  5481 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:15.701449  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 109033us
I20260709 13:02:15.769511  5402 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.752965  1700 rpcz_store.cc:276] 0709 13:02:13.701159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.701292 (+   133us) service_pool.cc:224] Handling call
0709 13:02:13.701626 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.710317 (+  8691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.710351 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.710824 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.930704 (+219880us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:14.126072 (+195368us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.309871 (+183799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.309980 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.155696 (+845716us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:15.700874 (+545178us) raft_consensus.cc:1685] finished
0709 13:02:15.701724 (+   850us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.701899 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45562,"prepare.queue_time_us":55523,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":286405,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":20358,"spinlock_wait_cycles":4688000,"thread_start_us":390499,"threads_started":2,"wal-append.queue_time_us":670153}
I20260709 13:02:15.753865  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.725018  5430 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:15.765174  1703 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:15.731889  5442 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.734818  5231 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:15.738386  5223 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:15.738641  4653 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:15.751163  4916 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.647735  5441 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:15.837707  5382 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:15.794680  4912 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:15.840848  5488 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:02:15.657534  5338 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.841823  4912 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.811553  4686 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:15.843900  5408 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:15.843990  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 576) took 2138 ms. Trace:
W20260709 13:02:15.844985  5491 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:15.747192  5452 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:15.846138  5493 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:15.845800  5452 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.732663  5079 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:15.743557  5448 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:15.797852  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:15.723193  5412 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.814011  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.854113  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:15.814349  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60577us
I20260709 13:02:15.815169  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.856269  1703 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:15.857375  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.858086  1703 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:15.859105  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:15.815554  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.795245  5449 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.836886  5122 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:15.846951  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 550) took 3525 ms. Trace:
I20260709 13:02:15.773237  5426 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.812495  5326 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:15.802085  5019 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:15.860339  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.908190  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.812237  4974 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.855471  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:15.913026  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57670.6us
I20260709 13:02:15.914494  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.865351  1552 rpcz_store.cc:276] 0709 13:02:12.321563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.916438 (+2594875us) service_pool.cc:224] Handling call
0709 13:02:14.925626 (+  9188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.925905 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.925943 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.926474 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.370997 (+444523us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:15.371033 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.371103 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.846721 (+475618us) raft_consensus.cc:1685] finished
0709 13:02:15.846740 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.846888 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":404655,"threads_started":1,"wal-append.queue_time_us":682817}
I20260709 13:02:15.750177  5409 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.815076  5481 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:15.795455  5296 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.844430  1702 rpcz_store.cc:276] 0709 13:02:13.705741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.705845 (+   104us) service_pool.cc:224] Handling call
0709 13:02:13.724366 (+ 18521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.784391 (+ 60025us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:13.784423 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.803702 (+ 19279us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.467982 (+1664280us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b1800287800. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:15.723165 (+255183us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:15.723204 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.723271 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.843626 (+120355us) raft_consensus.cc:1685] finished
0709 13:02:15.843649 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.843922 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8553,"raft.queue_time_us":110205,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":9906,"spinlock_wait_cycles":7299968,"thread_start_us":314403,"threads_started":2,"wal-append.queue_time_us":1995551}
I20260709 13:02:15.894680  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.829332  5304 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:15.800297  5385 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.898947  1700 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:15.791282  5470 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.894023  5240 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:15.899266  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.853551  5458 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:15.864888  5449 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.836190  5430 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.930716  1703 raft_consensus.cc:2410] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:15.771823  5459 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.835490  5442 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.938759  1559 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:15.941809  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:15.912271  5019 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:15.939626  1633 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:15.872890  5412 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.933070  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1641) took 1379 ms. Trace:
I20260709 13:02:15.945125  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:15.837733  4916 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.951877  2288 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:15.849836  5469 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:15.925163  5122 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:15.850924  5223 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.925025  5232 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:15.955601  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:15.861205  5448 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.930124  5481 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.975468  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2446) took 1165 ms. Trace:
I20260709 13:02:16.041909  1552 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:16.047489  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102439us
I20260709 13:02:16.046986  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:16.048234  1634 rpcz_store.cc:276] 0709 13:02:14.810233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.372320 (+562087us) service_pool.cc:224] Handling call
0709 13:02:15.372666 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.386414 (+ 13748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.386448 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.415583 (+ 29135us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.796220 (+380637us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:15.796282 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.796379 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.975237 (+178858us) raft_consensus.cc:1685] finished
0709 13:02:15.975264 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.975407 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14678,"raft.queue_time_us":18159,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":6832,"thread_start_us":315389,"threads_started":1,"wal-append.queue_time_us":493640}
I20260709 13:02:16.035979  1702 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:16.050827  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 558) took 2174 ms. Trace:
I20260709 13:02:16.053220  1559 rpcz_store.cc:276] 0709 13:02:13.876824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.911976 (+2035152us) service_pool.cc:224] Handling call
0709 13:02:15.938511 (+ 26535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.050532 (+112021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.050765 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.047959  1632 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:16.051512  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:16.037305  1707 rpcz_store.cc:276] 0709 13:02:14.553939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.411599 (+857660us) service_pool.cc:224] Handling call
0709 13:02:15.412056 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.412314 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:15.706128 (+293814us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.932620 (+226492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.932762 (+   142us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:15.932825 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.932905 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.933015 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16155,"prepare.queue_time_us":448894,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":61,"raft.queue_time_us":12899,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7329,"thread_start_us":492540,"threads_started":2}
I20260709 13:02:16.049649  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:16.057025  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:15.973203  5494 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:02:16.058357  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 134655us
I20260709 13:02:16.056226  5332 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:16.077418  5332 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.077849  5332 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:16.078855  5480 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.016s	user 0.000s	sys 0.001s
W20260709 13:02:16.080621  5506 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:02:16.084097  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1638) took 1586 ms. Trace:
I20260709 13:02:16.084933  5441 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:16.084477  1706 rpcz_store.cc:276] 0709 13:02:14.497416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.407038 (+909622us) service_pool.cc:224] Handling call
0709 13:02:15.407492 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.505489 (+ 97997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.505517 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.516792 (+ 11275us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.750411 (+233619us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:15.750460 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.750552 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.083872 (+333320us) raft_consensus.cc:1685] finished
0709 13:02:16.083894 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.084036 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96628,"raft.queue_time_us":92884,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":21948,"thread_start_us":147392,"threads_started":1,"wal-append.queue_time_us":296157}
I20260709 13:02:16.060807  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.926944  4974 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.050212  5232 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.097572  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:16.097887  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.101140  1559 pending_rounds.cc:85] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 4
I20260709 13:02:16.101735  1559 pending_rounds.cc:107] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:16.102167  1559 raft_consensus.cc:2889] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:16.107177  1703 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:16.074156  5448 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:16.075095  5496 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:15.952883  5299 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:02:15.991063  5345 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:15.970289  5304 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:16.076706  5333 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.076845  5470 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.088568  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:15.990304  5272 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.001106  5243 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
I20260709 13:02:16.060501  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.128094  1702 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:16.098654  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:16.058383  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:16.085887  5500 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:02:16.119441  5442 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.124431  1633 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:16.121140  5333 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.131997  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1613) took 4294 ms. Trace:
I20260709 13:02:16.145573  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.141657  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.113046  5232 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:16.108947  5367 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.147382  1706 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:15.942188  5231 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:16.153678  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104089us
I20260709 13:02:16.120776  5021 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:16.050031  5326 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.079463  5412 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.145148  1701 rpcz_store.cc:276] 0709 13:02:11.829440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.829636 (+   196us) service_pool.cc:224] Handling call
0709 13:02:11.840272 (+ 10636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.011134 (+170862us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:12.616673 (+605539us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.290621 (+673948us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.378327 (+ 87706us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.378411 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:13.378476 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.527015 (+1148539us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b1800417940. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:16.123033 (+1596018us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:16.123089 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.123225 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":941406,"prepare.queue_time_us":116281,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":3552,"raft.queue_time_us":206150,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":15857,"spinlock_wait_cycles":496581888,"thread_start_us":1220410,"threads_started":3,"wal-append.queue_time_us":615943,"wal-append.run_cpu_time_us":3610,"wal-append.run_wall_time_us":1757701}
I20260709 13:02:16.126482  5433 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:16.140755  5263 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.134646  5448 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:16.112668  5149 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:02:16.157968  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.150233  1632 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:16.170253  1706 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:16.161441  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.142719  5462 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.153069  5333 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.174914  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.167479  1703 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:16.142654  5517 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.175813  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.159514  5122 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:16.138412  5272 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.184734  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:16.162151  5367 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:16.191103  5505 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:02:16.192253  5517 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.197325  1701 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:16.168069  5448 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:16.177947  5426 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:16.206787  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:16.178385  5409 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:16.206676  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.210836  1702 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:16.211719  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.213012  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:16.214061  5513 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:16.215668  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:16.170889  5263 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.221349  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:16.222944  5504 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:16.225270  5508 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:16.225804  5508 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.223970  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:16.228503  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1630) took 2510 ms. Trace:
I20260709 13:02:16.229106  1708 rpcz_store.cc:276] 0709 13:02:13.718523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.955663 (+1237140us) service_pool.cc:224] Handling call
0709 13:02:14.956102 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.471408 (+515306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.471443 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.471930 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.703844 (+231914us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:15.703889 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.703959 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.228201 (+524242us) raft_consensus.cc:1685] finished
0709 13:02:16.228230 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.228417 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":514837,"raft.queue_time_us":523901,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":7711,"thread_start_us":231472,"threads_started":1,"wal-append.queue_time_us":344331}
I20260709 13:02:16.232789  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:16.204224  5296 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.234526  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.231555  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.239622  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.241389  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.242834  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.252836  1708 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:16.254081  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:16.223999  5433 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:16.252206  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:16.226184  5508 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.239187  5338 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:16.260401  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:16.205243  5453 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.253424  5079 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:16.201261  5304 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.225929  5517 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:16.260422  5511 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:02:16.253253  5240 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.262948  5228 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:16.274099  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.245882  5504 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:16.263456  5433 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:16.269183  5079 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:16.279246  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:16.198457  5462 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.291622  5471 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:16.257407  5122 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.277345  5408 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:16.256194  5448 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.278697  5504 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:16.278987  5228 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:16.281276  5338 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:16.292285  5079 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:16.306389  5421 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.005s
I20260709 13:02:16.301064  5238 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.292089  4974 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:16.310726  1632 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:16.299486  5382 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:16.306115  5443 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.000s
I20260709 13:02:16.317934  4653 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.307294  5471 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:16.316457  5228 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:16.312677  5504 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:16.342655  1558 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:16.367122  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 570) took 1136 ms. Trace:
I20260709 13:02:16.367646  1558 rpcz_store.cc:276] 0709 13:02:15.230255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.330114 (+1099859us) service_pool.cc:224] Handling call
0709 13:02:16.342469 (+ 12355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.366711 (+ 24242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.367026 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.299940  5384 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.351734  5272 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:16.332311  5512 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:02:16.340106  5433 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.353658  1632 pending_rounds.cc:85] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 8
I20260709 13:02:16.383986  1632 pending_rounds.cc:107] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:16.384367  1632 raft_consensus.cc:2889] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:16.307734  5408 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.358667  5523 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.051s	user 0.002s	sys 0.001s
I20260709 13:02:16.358515  5443 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.304397  4916 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.365015  1635 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:16.353121  5421 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:16.331722  5516 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:02:16.374648  5471 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.356945  5448 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.371322  5382 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.320101  5338 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.354869  5238 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.386831  5408 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:16.401881  2296 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:16.399562  2289 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:16.430311  5519 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:02:16.400628  5338 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:16.405189  5471 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:16.399833  5382 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.455475  5238 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:16.404589  5451 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:16.494246  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.508589  5240 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:16.431898  5332 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:16.536841  5523 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:16.565902  5462 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.537838  5019 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.594053  1708 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:02:16.593441  5455 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.955s	user 0.015s	sys 0.001s
I20260709 13:02:16.837344  5455 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.653111  5508 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.842410  5508 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:16.676658  5531 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:16.845098  1709 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:16.846172  5332 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:16.593474  5449 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.005s
I20260709 13:02:16.863421  5449 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:16.869697  5079 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.002s
I20260709 13:02:16.870453  5079 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.873484  5079 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:16.592213  5402 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.001s
W20260709 13:02:16.901852  5481 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.839s	user 0.000s	sys 0.012s
I20260709 13:02:16.902483  5481 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:16.661552  5522 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
W20260709 13:02:16.593768  5300 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.891s	user 0.004s	sys 0.009s
I20260709 13:02:16.637681  5382 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:16.901901  5448 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.005s
I20260709 13:02:16.895309  5421 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.934017  5338 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:16.936774  5333 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.005s
I20260709 13:02:16.937136  5338 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.937335  5333 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:16.939836  5333 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.940399  5443 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:16.941277  5449 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:16.945513  5481 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:16.947562  5408 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.002s
I20260709 13:02:16.948202  5408 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:16.950917  5408 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:17.179785  5228 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.003s
W20260709 13:02:17.034044  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.001s	sys 0.001s
I20260709 13:02:17.035271  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 565) took 2073 ms. Trace:
I20260709 13:02:17.038446  1631 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:17.116852  1701 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:17.118470  1702 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:17.118896  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1631) took 3399 ms. Trace:
W20260709 13:02:17.119666  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.182060  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.001s	sys 0.001s
I20260709 13:02:17.119887  1707 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:16.951638  5455 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:16.955370  5504 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 13:02:17.178848  5333 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:17.205607  5333 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.184484  5338 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.182260  5228 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:17.196571  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2441) took 4986 ms (client timeout 3041 ms). Trace:
W20260709 13:02:16.913480  5231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.017s
I20260709 13:02:17.205348  5504 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:17.008508  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.005s	sys 0.009s
I20260709 13:02:17.200634  1626 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:17.208606  1630 rpcz_store.cc:269] 0709 13:02:12.208703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.209022 (+   319us) service_pool.cc:224] Handling call
0709 13:02:17.191190 (+4982168us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540019c654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.194512 (+  3322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3635804544}
I20260709 13:02:17.217756  5332 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.199993  5449 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:16.592491  5232 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.218516  5332 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.219954  5338 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.248677  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1664) took 1739 ms. Trace:
I20260709 13:02:17.182642  1559 rpcz_store.cc:276] 0709 13:02:14.961709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.059714 (+1098005us) service_pool.cc:224] Handling call
0709 13:02:16.100824 (+ 41110us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:16.130723 (+ 29899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.130767 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:16.157677 (+ 26910us) log.cc:844] Serialized 78 byte log entry
0709 13:02:16.260003 (+102326us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.260044 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.260113 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.034317 (+774204us) raft_consensus.cc:1685] finished
0709 13:02:17.034402 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.034993 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4888,"raft.queue_time_us":9268,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":16522,"thread_start_us":80953,"threads_started":1,"wal-append.queue_time_us":131010}
I20260709 13:02:17.259173  1706 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:17.183375  1704 rpcz_store.cc:276] 0709 13:02:13.718846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.090291 (+1371445us) service_pool.cc:224] Handling call
0709 13:02:15.090729 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.512665 (+421936us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:15.512696 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.533884 (+ 21188us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.351613 (+817729us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b1800417f40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:16.400300 (+ 48687us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:16.400340 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.400482 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.117735 (+717253us) raft_consensus.cc:1685] finished
0709 13:02:17.117803 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.118494 (+   691us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":471818,"raft.queue_time_us":423577,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7615,"spinlock_wait_cycles":25626368,"thread_start_us":44925,"threads_started":1,"wal-append.queue_time_us":957116}
I20260709 13:02:17.251231  1558 rpcz_store.cc:276] 0709 13:02:15.507054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.378812 (+871758us) service_pool.cc:224] Handling call
0709 13:02:16.379318 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.379735 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.379810 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.385626 (+  5816us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.198653 (+813027us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:17.211983 (+ 13330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.221120 (+  9137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.223957 (+  2837us) raft_consensus.cc:1685] finished
0709 13:02:17.223974 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.246169 (+ 22195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76488,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"thread_start_us":782747,"threads_started":1,"wal-append.queue_time_us":531737}
W20260709 13:02:17.255820  5231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.000s	sys 0.018s
I20260709 13:02:17.262573  1447 maintenance_manager.cc:419] P 8c0fa710867745f189aafd0e7d84fa64: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033354
I20260709 13:02:17.252617  5448 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:17.270210  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.006s	sys 0.010s
I20260709 13:02:17.271095  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 554) took 3533 ms. Trace:
W20260709 13:02:17.274639  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.514s (SENT) [suppressed 10 similar messages]
I20260709 13:02:17.275368  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2314) took 1426 ms. Trace:
I20260709 13:02:17.275892  1633 rpcz_store.cc:276] 0709 13:02:15.848407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.058440 (+210033us) service_pool.cc:224] Handling call
0709 13:02:17.274887 (+1216447us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b5400343ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.275139 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84631808}
W20260709 13:02:17.268354  1557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.006s	sys 0.018s
I20260709 13:02:17.281369  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 977) took 1887 ms. Trace:
I20260709 13:02:17.281899  2289 rpcz_store.cc:276] 0709 13:02:15.393588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.750653 (+357065us) service_pool.cc:224] Handling call
0709 13:02:15.771000 (+ 20347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.402870 (+631870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.276754 (+873884us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:17.281055 (+  4301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445009664}
I20260709 13:02:17.271747  1554 rpcz_store.cc:276] 0709 13:02:13.737310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.489090 (+1751780us) service_pool.cc:224] Handling call
0709 13:02:15.534102 (+ 45012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.592579 (+ 58477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.270248 (+1677669us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54003fb954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:17.270776 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132650752}
I20260709 13:02:17.287889  5408 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.288669  5408 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.297834  1554 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:17.298693  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1667) took 1790 ms. Trace:
I20260709 13:02:17.299175  5338 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.280156  5332 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:17.316689  5408 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:17.268101  5449 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.299062  1554 rpcz_store.cc:276] 0709 13:02:15.507923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.296914 (+1788991us) service_pool.cc:224] Handling call
0709 13:02:17.297521 (+   607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.298408 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.298614 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.334682  1559 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:17.335496  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1665) took 1828 ms. Trace:
I20260709 13:02:17.335899  1559 rpcz_store.cc:276] 0709 13:02:15.507315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.267639 (+1760324us) service_pool.cc:224] Handling call
0709 13:02:17.334392 (+ 66753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.335166 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.335411 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727104}
I20260709 13:02:17.338337  1559 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:17.337869  5386 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.339154  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1672) took 1829 ms. Trace:
I20260709 13:02:17.345340  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2356) took 1597 ms. Trace:
I20260709 13:02:17.345824  1707 rpcz_store.cc:276] 0709 13:02:15.747387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.058127 (+310740us) service_pool.cc:224] Handling call
0709 13:02:17.345003 (+1286876us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54003c0d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.345194 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273562368}
I20260709 13:02:17.346633  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.348753  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.350135  5449 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.350682  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2350) took 1604 ms. Trace:
I20260709 13:02:17.350792  2289 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:02:17.351256  2289 pending_rounds.cc:85] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 9
I20260709 13:02:17.333367  1554 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:17.287329  1557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.007s	sys 0.019s
I20260709 13:02:17.304349  5448 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.354053  1707 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:17.371481  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 561) took 3358 ms. Trace:
I20260709 13:02:17.401310  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1671) took 1892 ms. Trace:
I20260709 13:02:17.401886  1554 rpcz_store.cc:276] 0709 13:02:15.509059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.332471 (+1823412us) service_pool.cc:224] Handling call
0709 13:02:17.333044 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.400521 (+ 67477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.401241 (+   720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.351145  1701 rpcz_store.cc:276] 0709 13:02:15.745966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.199874 (+453908us) service_pool.cc:224] Handling call
0709 13:02:16.213090 (+ 13216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.213265 (+1000175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.350401 (+137136us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54003afcd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:17.350625 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076517120}
I20260709 13:02:17.352926  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2296) took 4750 ms. Trace:
W20260709 13:02:17.396677  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122203us
I20260709 13:02:17.351609  2289 pending_rounds.cc:107] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:17.412411  2289 raft_consensus.cc:2889] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:17.359071  5238 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.415716  5238 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.404462  1626 rpcz_store.cc:276] 0709 13:02:12.602182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.791835 (+189653us) service_pool.cc:224] Handling call
0709 13:02:13.245683 (+453848us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b58001f6898. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:17.191476 (+3945793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.220120 (+ 28644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.352653 (+132533us) spinlock_profiling.cc:243] Waited 3.95 s on lock 0x7b540019c650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.352822 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9082225792}
I20260709 13:02:17.413674  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1649) took 1598 ms. Trace:
I20260709 13:02:17.339646  1559 rpcz_store.cc:276] 0709 13:02:15.509324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.337140 (+1827816us) service_pool.cc:224] Handling call
0709 13:02:17.337748 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.338784 (+  1036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.339060 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.419996  1707 rpcz_store.cc:276] 0709 13:02:15.810674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.347226 (+1536552us) service_pool.cc:224] Handling call
0709 13:02:17.353696 (+  6470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.404437 (+ 50741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.409340 (+  4903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.401194  1552 rpcz_store.cc:276] 0709 13:02:14.012847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.927756 (+1914909us) service_pool.cc:224] Handling call
0709 13:02:15.942716 (+ 14960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.048671 (+105955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.371034 (+1322363us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b54000771d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:17.371325 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227504384}
I20260709 13:02:17.360435  5385 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:17.357221  5333 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.354970  5504 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.352458  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.355079  5019 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.424489  5375 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:17.425691  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 546) took 5105 ms. Trace:
I20260709 13:02:17.426009  5238 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:17.404958  5338 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.379251  5382 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:17.369836  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:17.360129  5240 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:17.374697  5386 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:17.371019  1599 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.526s (SENT)
I20260709 13:02:17.431434  1551 rpcz_store.cc:276] 0709 13:02:12.320426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.725286 (+1404860us) service_pool.cc:224] Handling call
0709 13:02:13.725739 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.455870 (+730131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.455900 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.486441 (+ 30541us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.716025 (+229584us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:14.716082 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.413009 (+2696927us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:17.418978 (+  5969us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.425062 (+  6084us) raft_consensus.cc:1685] finished
0709 13:02:17.425081 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.425448 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678533888,"thread_start_us":196852,"threads_started":1,"wal-append.queue_time_us":352828,"wal-append.run_cpu_time_us":1916,"wal-append.run_wall_time_us":1005952}
I20260709 13:02:17.448238  5375 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.449120  5240 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.450875  5375 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:17.452498  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1666) took 1944 ms. Trace:
I20260709 13:02:17.452916  1558 rpcz_store.cc:276] 0709 13:02:15.507565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.299857 (+1792292us) service_pool.cc:224] Handling call
0709 13:02:17.387707 (+ 87850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.452059 (+ 64352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.452098 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.452166 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.452210 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.452287 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.452413 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15104}
I20260709 13:02:17.451468  5240 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:17.460949  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 174420us
I20260709 13:02:17.463390  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:17.464195  5338 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.468313  5382 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.469447  5448 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:17.470917  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101526us
I20260709 13:02:17.471323  5382 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:17.477272  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 569) took 2387 ms (client timeout 2368 ms). Trace:
W20260709 13:02:17.477722  1556 rpcz_store.cc:269] 0709 13:02:15.089305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.410490 (+321185us) service_pool.cc:224] Handling call
0709 13:02:17.476753 (+2066263us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54000ea454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.477016 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510931712}
I20260709 13:02:17.464692  5385 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:17.453356  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:17.459348  2297 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:17.467500  5338 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:17.479825  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2256) took 6418 ms (client timeout 4723 ms). Trace:
W20260709 13:02:17.494477  1555 rpcz_store.cc:269] 0709 13:02:11.060862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.814931 (+754069us) service_pool.cc:224] Handling call
0709 13:02:17.479340 (+5664409us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002b98d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:17.479593 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854006656}
W20260709 13:02:17.505585  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53732.2us
I20260709 13:02:17.507325  5542 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:17.507709  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 136974us
W20260709 13:02:17.508811  1599 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.577s (SENT)
I20260709 13:02:17.509521  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:17.510726  5542 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:17.511485  5391 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.512329  5391 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.515128  5391 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:17.515380  1559 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:17.523952  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1676) took 2013 ms. Trace:
I20260709 13:02:17.526230  1559 rpcz_store.cc:276] 0709 13:02:15.510460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.432857 (+1922397us) service_pool.cc:224] Handling call
0709 13:02:17.505273 (+ 72416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.522720 (+ 17447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.523871 (+  1151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:17.526923  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 80775.8us
W20260709 13:02:17.504537  5428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.017s	sys 0.024s
I20260709 13:02:17.544199  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:17.544636  5428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.584s	user 0.017s	sys 0.025s
I20260709 13:02:17.557204  5391 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.561133  1599 connection.cc:256] RPC call timeout handler was delayed by -3.2650548249999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
I20260709 13:02:17.561115  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:17.568811  5366 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:17.570319  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2306) took 2061 ms. Trace:
I20260709 13:02:17.574525  5391 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.577584  5391 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.578444  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:17.570969  1628 rpcz_store.cc:276] 0709 13:02:15.508744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.602946 (+ 94202us) service_pool.cc:224] Handling call
0709 13:02:15.603248 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.603448 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:15.603759 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.358350 (+1754591us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b18003e4400. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240
0709 13:02:17.382042 (+ 23692us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18003e4400. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db
0709 13:02:17.569251 (+187209us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18003e4400. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287
0709 13:02:17.569444 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.569529 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:17.569675 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.569805 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.570057 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30495,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":112079,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7231,"spinlock_wait_cycles":27737856,"thread_start_us":436511,"threads_started":2,"wal-append.queue_time_us":371014}
W20260709 13:02:17.579249  1524 connection.cc:256] RPC call timeout handler was delayed by -1.1899712820000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2523s for a response
I20260709 13:02:17.580542  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:17.580994  5455 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.579360  5338 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.591154  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 80528.1us
I20260709 13:02:17.592677  1556 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:17.593395  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1650) took 1777 ms. Trace:
I20260709 13:02:17.599997  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.585850  5391 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.602514  2289 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:17.602974  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.602658  5391 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:17.606915  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78998.9us
I20260709 13:02:17.613189  5385 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:17.614627  5385 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:17.615126  5385 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:17.616055  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.616590  5385 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.618165  5385 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.621531  5370 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:17.627987  5391 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.630759  5323 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.631912  5455 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:17.597043  1701 rpcz_store.cc:276] 0709 13:02:15.815960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.410847 (+1594887us) service_pool.cc:224] Handling call
0709 13:02:17.411545 (+   698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.415975 (+  4430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.416004 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.416635 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.517938 (+101303us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.518172 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.518261 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.593162 (+ 74901us) raft_consensus.cc:1685] finished
0709 13:02:17.593179 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.593318 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5238,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":5706,"thread_start_us":100791,"threads_started":1,"wal-append.queue_time_us":150801}
I20260709 13:02:17.640779  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1651) took 1807 ms. Trace:
I20260709 13:02:17.648525  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.650939  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:17.653261  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.656169  1555 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:17.657491  1556 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:17.635704  5455 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.662276  4653 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.661083  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:17.669168  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:17.667351  2297 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:17.672263  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1678) took 2161 ms. Trace:
I20260709 13:02:17.711642  1552 rpcz_store.cc:276] 0709 13:02:15.511272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.427339 (+1916067us) service_pool.cc:224] Handling call
0709 13:02:17.427871 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.428333 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:17.534813 (+106480us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.671926 (+137113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.671975 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:17.672023 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.672090 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.672198 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":975,"prepare.queue_time_us":162470,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2806,"raft.queue_time_us":8968,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"thread_start_us":176317,"threads_started":2}
I20260709 13:02:17.713734  1701 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:17.650020  5370 consensus_queue.cc:1243] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:02:17.725668  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.720269  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:17.714706  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 580) took 1556 ms. Trace:
I20260709 13:02:17.740468  1701 rpcz_store.cc:276] 0709 13:02:16.157832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.639415 (+1481583us) service_pool.cc:224] Handling call
0709 13:02:17.713187 (+ 73772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.714324 (+  1137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.714610 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.714541  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:17.722520  5428 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:17.760699  5428 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:17.762540  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.765040  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:17.768043  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.770030  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:17.770242  2289 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:17.770740  1558 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.771291  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.772157  5424 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.773123  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.774068  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.777657  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.659613  5338 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:17.779413  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64442us
I20260709 13:02:17.913544  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:17.915608  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.812844  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.826982  4653 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.803895  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.875016  1552 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:17.909759  1558 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:17.920130  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.920750  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.921537  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:17.924383  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.925873  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.929376  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.929945  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.933933  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.933286  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.934976  2289 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:17.932214  1556 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:17.937750  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:17.934904  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:17.941445  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.920424  5555 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.942237  1556 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:17.920145  5424 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.947086  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.948199  1555 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:17.949198  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.942690  1552 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:17.953712  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:17.960187  1701 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.958316  4653 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:17.962519  1558 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:17.965313  2291 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:17.943210  5428 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:17.979642  2291 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:17.980587  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:17.997578  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:18.015700  1599 connection.cc:256] RPC call timeout handler was delayed by -1.0470474609999769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3168s for a response
I20260709 13:02:17.961354  5555 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.023239  1558 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:02:18.052934  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44140.1us
I20260709 13:02:18.070885  5424 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:18.058027  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.021010  5454 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.073872  4653 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:18.074156  1555 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:18.073099  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.076778  1558 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:02:18.078976  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57350.8us
I20260709 13:02:18.022013  5428 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.080679  2289 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:18.085052  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:18.085906  5555 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:18.086321  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46082.7us
I20260709 13:02:18.086961  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 578) took 2277 ms. Trace:
I20260709 13:02:18.087169  5448 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:18.087369  1632 rpcz_store.cc:276] 0709 13:02:15.809824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.170358 (+360534us) service_pool.cc:224] Handling call
0709 13:02:16.170702 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.384878 (+214176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.086492 (+1701614us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:18.086784 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320813312}
I20260709 13:02:18.088380  5338 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:18.086784  5454 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.095376  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.099957  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:18.088425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62765.9us
I20260709 13:02:18.105482  5428 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.113538  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:18.113248  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 133585us
I20260709 13:02:18.179661  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 893) took 4309 ms. Trace:
I20260709 13:02:18.160384  1558 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:18.162555  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.226230  1701 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:18.115337  5338 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:02:18.179009  5429 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.232043  5414 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.233152  5230 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.234092  5371 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:18.234531  5454 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:18.227027  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54148.5us
W20260709 13:02:18.184238  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2348) took 2438 ms (client timeout 1802 ms). Trace:
I20260709 13:02:18.251205  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:18.180856  5558 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:02:18.115231  5428 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:18.181802  5501 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 2.259s	user 0.014s	sys 0.020s
I20260709 13:02:18.235566  2290 rpcz_store.cc:276] 0709 13:02:13.869706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.869886 (+   180us) service_pool.cc:224] Handling call
0709 13:02:13.870256 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.902994 (+1032738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.178970 (+3275976us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:18.179405 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232170240}
I20260709 13:02:18.218801  5236 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:18.251209  1700 rpcz_store.cc:269] 0709 13:02:15.745501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.853810 (+108309us) service_pool.cc:224] Handling call
0709 13:02:18.183707 (+2329897us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:18.184017 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100676992}
I20260709 13:02:18.256052  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:18.258075  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:18.251945  5465 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:02:18.264825  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.268036  5244 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:02:18.274549  1555 raft_consensus.cc:2376] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14. Re-sending same reply.
I20260709 13:02:18.285017  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:18.285920  5448 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:18.310518  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56179.8us
I20260709 13:02:18.342141  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:18.347162  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 568) took 3261 ms. Trace:
I20260709 13:02:18.355461  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.348304  1700 raft_consensus.cc:2376] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 9. Re-sending same reply.
W20260709 13:02:18.354908  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63999.9us
W20260709 13:02:18.357867  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 79994.5us
I20260709 13:02:18.359898  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:18.355238  1701 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:18.359598  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:18.485882  5428 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:18.513669  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 439838us
I20260709 13:02:18.513967  1558 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:18.256709  5429 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:18.518333  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158812us
W20260709 13:02:18.519534  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.521530  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 890) took 5054 ms. Trace:
I20260709 13:02:18.521688  5433 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:18.522662  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49031.4us
I20260709 13:02:18.522037  2294 rpcz_store.cc:276] 0709 13:02:13.466793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.467043 (+   250us) service_pool.cc:224] Handling call
0709 13:02:13.467527 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.923377 (+455850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.923428 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.419396 (+1495968us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b2c00047568. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:02:15.427111 (+  7715us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.735416 (+308305us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:15.735474 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.735601 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.180907 (+2445306us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:18.521067 (+340160us) raft_consensus.cc:1685] finished
0709 13:02:18.521088 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.521350 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3002,"raft.queue_time_us":96722,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":10176,"spinlock_wait_cycles":17617024,"thread_start_us":698232,"threads_started":2,"wal-append.queue_time_us":362022}
I20260709 13:02:18.349635  1557 rpcz_store.cc:276] 0709 13:02:15.085590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.271634 (+1186044us) service_pool.cc:224] Handling call
0709 13:02:16.271936 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.272115 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:16.321034 (+ 48919us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.022419 (+1701385us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.244170 (+221751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.249015 (+  4845us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:18.256316 (+  7301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.289091 (+ 32775us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.347083 (+ 57992us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1125,"prepare.queue_time_us":206344,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":3496,"raft.queue_time_us":77420,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":6343,"thread_start_us":1402633,"threads_started":5,"wal-append.queue_time_us":253765,"wal-append.run_cpu_time_us":1574,"wal-append.run_wall_time_us":1069211}
I20260709 13:02:18.524966  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:18.525265  1674 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.842s (SENT)
W20260709 13:02:18.526365  1674 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.943s (SENT)
I20260709 13:02:18.526623  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.516230  5428 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.354894  5373 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.528348  5428 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.568257  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.569080  1558 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:18.571857  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:18.567306  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49620us
I20260709 13:02:18.548660  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.570550  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.578475  1700 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:18.580646  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.547716  5557 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:18.581586  1557 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:18.653020  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.653774  1558 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 5.
I20260709 13:02:18.579607  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:18.547912  5338 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:18.678099  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.679078  1557 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:18.681253  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:18.682056  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 102645us
I20260709 13:02:18.653831  5557 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.692910  1555 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:18.689109  1524 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.708s (SENT)
I20260709 13:02:18.699504  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:18.701308  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:18.695427  5557 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:18.713702  5338 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:18.715041  5223 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:18.719933  5304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.284s	user 0.009s	sys 0.042s
W20260709 13:02:18.720379  5304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.320s	user 0.009s	sys 0.043s
I20260709 13:02:18.724391  5253 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:18.724898  5253 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:18.725980  5253 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.727169  5253 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:18.732182  1559 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:18.733156  1555 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:18.733542  1555 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:18.740481  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.744720  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.744688  1700 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:18.749150  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:18.750765  5577 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:18.737959  5414 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.768399  1559 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:02:18.704218  5371 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:18.776189  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:18.779232  1700 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.797012  5574 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:18.797971  5574 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:18.800942  5574 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:18.820529  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 113265us
I20260709 13:02:18.775736  5414 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.817713  1555 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:18.825333  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:18.820858  5555 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.829056  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 98254.5us
W20260709 13:02:18.834288  5568 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:02:18.834841  2260 connection.cc:256] RPC call timeout handler was delayed by -1.888867380000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
W20260709 13:02:18.869619  2260 connection.cc:256] RPC call timeout handler was delayed by -1.8538079860000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
W20260709 13:02:18.870100  2260 connection.cc:256] RPC call timeout handler was delayed by -1.49609250900005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2368s for a response
W20260709 13:02:18.844180  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.844731  5382 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:18.848023  5429 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:18.868431  1555 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:18.876906  1701 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:18.871579  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:18.874590  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.879477  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65134.3us
I20260709 13:02:18.883303  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2370) took 1111 ms. Trace:
I20260709 13:02:18.852252  5414 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:18.869891  5577 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:18.883857  1701 rpcz_store.cc:276] 0709 13:02:17.765954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.257761 (+491807us) service_pool.cc:224] Handling call
0709 13:02:18.877744 (+619983us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:18.886870  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 229632us
I20260709 13:02:18.888975  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:18.893599  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:18.898417  1524 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.965s (SENT)
I20260709 13:02:18.893301  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1682) took 1513 ms. Trace:
I20260709 13:02:18.879796  5366 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:18.899144  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:18.905704  2294 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:02:18.908954  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.909024  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:18.893280  5244 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:18.915988  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:18.918887  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:18.898332  5580 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:18.886914  5555 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:18.930326  1672 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44866 dropped due to backpressure: service queue is full with 50 items
I20260709 13:02:18.931046  1700 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:18.996016  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2373) took 1229 ms. Trace:
I20260709 13:02:18.996441  1700 rpcz_store.cc:276] 0709 13:02:17.766746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.758662 (+991916us) service_pool.cc:224] Handling call
0709 13:02:18.995949 (+237287us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:18.935286  5236 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.001s
I20260709 13:02:18.935845  5414 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:18.937603  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:19.001132  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63593.7us
I20260709 13:02:18.939075  5366 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:18.939802  2295 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:18.939985  5567 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:18.940573  5429 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:18.940764  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:18.942539  5576 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:18.986640  2290 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:18.988560  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:18.989369  1701 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:18.991521  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 96743.9us
I20260709 13:02:18.917306  5244 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.999325  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 122491us
I20260709 13:02:19.000072  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:19.001703  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:18.943171  5583 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:18.987751  5580 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:19.009686  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 111313us
I20260709 13:02:19.009894  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2365) took 1245 ms. Trace:
I20260709 13:02:19.010387  5378 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.010571  5369 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.011013  5576 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.015887  5343 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:19.019539  5590 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:19.020265  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:18.899312  5578 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 13:02:19.023478  5452 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.044229  1555 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:02:19.039706  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.006860  5230 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:19.023979  5429 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.082662  5378 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.093138  1555 pending_rounds.cc:85] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:02:19.095207  1555 pending_rounds.cc:107] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 13:02:19.054728  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56303.7us
I20260709 13:02:19.095559  1555 raft_consensus.cc:2889] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.037603  1701 rpcz_store.cc:276] 0709 13:02:17.764413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.893279 (+1128866us) service_pool.cc:224] Handling call
0709 13:02:19.009832 (+116553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.029388  5244 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:19.101863  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.029294  5414 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:19.024822  5296 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.087744  1700 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:19.090075  5343 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.107522  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.110920  1632 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:19.112838  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.114459  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.115885  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.117408  5244 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.054365  5567 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:19.077365  5382 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:19.120096  5555 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.120564  5559 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:19.124348  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2375) took 1357 ms. Trace:
I20260709 13:02:19.123494  1634 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:19.131834  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.132200  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.096468  5230 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.133153  5343 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.096129  5448 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:19.039372  5576 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.130652  5567 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.133249  5244 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.137075  1700 rpcz_store.cc:276] 0709 13:02:17.767292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.000746 (+1233454us) service_pool.cc:224] Handling call
0709 13:02:19.124260 (+123514us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:19.145480  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 120589us
W20260709 13:02:19.146273  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.146664  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1683) took 1371 ms. Trace:
I20260709 13:02:19.146981  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.154186  1557 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:19.156317  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:19.157986  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50591.2us
I20260709 13:02:19.147408  1558 rpcz_store.cc:276] 0709 13:02:17.775235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.674165 (+898930us) service_pool.cc:224] Handling call
0709 13:02:18.687407 (+ 13242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.703002 (+ 15595us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:18.761016 (+ 58014us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.077161 (+316145us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.145995 (+ 68834us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.146282 (+   287us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:19.146338 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.146412 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.146609 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120934,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":1357,"raft.queue_time_us":54737,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":13920000,"thread_start_us":360129,"threads_started":3,"wal-append.queue_time_us":327173}
I20260709 13:02:19.159919  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:19.073834  5369 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.161278  5429 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:19.161881  5586 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:19.161500  5414 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:19.159348  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1646) took 3534 ms. Trace:
I20260709 13:02:19.163851  1705 rpcz_store.cc:276] 0709 13:02:15.475474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.571737 (+ 96263us) service_pool.cc:224] Handling call
0709 13:02:15.572050 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.618202 (+ 46152us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:15.920121 (+301919us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.161461 (+241340us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.726922 (+2565461us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b2c000b1c58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:02:18.727637 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.889719 (+162082us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:18.920556 (+ 30837us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.943480 (+ 22924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.943723 (+   243us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.993511 (+ 49788us) raft_consensus.cc:1685] finished
0709 13:02:18.993548 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.998121 (+  4573us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88779,"prepare.queue_time_us":341007,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":5976,"raft.queue_time_us":52954,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":19105,"spinlock_wait_cycles":8663424,"thread_start_us":588721,"threads_started":3,"wal-append.queue_time_us":453751,"wal-append.run_cpu_time_us":3238,"wal-append.run_wall_time_us":1268249}
W20260709 13:02:19.167186  5428 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 13:02:19.167935  5591 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:19.170125  5557 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.038175  5577 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.172684  1627 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:02:19.173075  1627 pending_rounds.cc:85] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 8
I20260709 13:02:19.173285  1627 pending_rounds.cc:107] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:02:19.173506  1627 raft_consensus.cc:2889] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.174633  1630 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:19.002617  1672 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44866 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2384)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1656)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1663)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 590)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1652)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1653)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1654)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1662)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1666)
I20260709 13:02:19.175701  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:19.176151  5581 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:02:19.176365  1630 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.140148  5382 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:19.177719  5382 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.140009  5230 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.155583  5448 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.161585  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.182215  1632 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:02:19.177253  1672 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44866 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2384)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1656)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1670)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1663)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1671)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 590)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1652)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1653)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1654)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1662)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1666)
I20260709 13:02:19.136315  5559 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:19.144834  5567 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.187693  5580 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:19.194013  5503 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 3.193s	user 0.017s	sys 0.042s
I20260709 13:02:19.193626  5559 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:19.194831  5559 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.195640  1628 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:19.186017  1633 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:19.200218  5430 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.202394  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1648) took 3392 ms. Trace:
I20260709 13:02:19.202795  1704 rpcz_store.cc:276] 0709 13:02:15.810391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.284367 (+1473976us) service_pool.cc:224] Handling call
0709 13:02:17.285073 (+   706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.435551 (+150478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.435614 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.436620 (+  1006us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.598193 (+161573us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.598233 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.202016 (+1603783us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:19.202066 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.202110 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:19.202123 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.202335 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332499840,"thread_start_us":160937,"threads_started":1,"wal-append.queue_time_us":186049,"wal-append.run_cpu_time_us":2067,"wal-append.run_wall_time_us":1015322}
I20260709 13:02:19.206851  5366 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.206930  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.207858  1704 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:19.208256  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:19.208365  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2376) took 1440 ms. Trace:
I20260709 13:02:19.208851  1704 rpcz_store.cc:276] 0709 13:02:17.767556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.206592 (+1439036us) service_pool.cc:224] Handling call
0709 13:02:19.208285 (+  1693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.209210  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.209888  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:19.210420  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 63262.1us
I20260709 13:02:19.210613  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2381) took 1188 ms. Trace:
I20260709 13:02:19.210937  1704 rpcz_store.cc:276] 0709 13:02:18.024933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.209574 (+1184641us) service_pool.cc:224] Handling call
0709 13:02:19.210532 (+   958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.211239  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:19.208879  1632 raft_consensus.cc:2376] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11. Re-sending same reply.
I20260709 13:02:19.212774  5371 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:19.213495  5371 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:19.214515  5378 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.215077  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.220902  1630 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:19.135819  5296 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:18.910094  1551 rpcz_store.cc:276] 0709 13:02:17.379446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.448663 (+ 69217us) service_pool.cc:224] Handling call
0709 13:02:18.524455 (+1075792us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:18.524650 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.524895 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:18.745266 (+220371us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.892481 (+147215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.892532 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:18.892577 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.892640 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.892734 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21266,"prepare.queue_time_us":283296,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1548,"raft.queue_time_us":5562,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":15962,"spinlock_wait_cycles":11525376,"thread_start_us":315262,"threads_started":2}
I20260709 13:02:19.167683  5428 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.231491  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.229786  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:19.233779  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 113102us
I20260709 13:02:19.222594  5244 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:19.237133  1704 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:02:19.250932  1635 raft_consensus.cc:2368] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9.
W20260709 13:02:19.251044  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61091.1us
W20260709 13:02:19.252913  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 250851us
W20260709 13:02:19.254050  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.253624  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:19.239156  5378 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:19.255347  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.262367  5432 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:19.262792  5432 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.261696  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:19.264258  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:19.268249  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.269327  1704 raft_consensus.cc:2410] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:19.265987  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2345) took 3518 ms. Trace:
I20260709 13:02:19.276859  1706 rpcz_store.cc:276] 0709 13:02:15.744779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.185864 (+441085us) service_pool.cc:224] Handling call
0709 13:02:16.186218 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.259863 (+1073645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.262980 (+2003117us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400165654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:19.263219 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464328960}
I20260709 13:02:19.278136  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.278573  5349 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:19.278815  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:19.279052  5349 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.279431  5349 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:19.263162  5432 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.269351  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:19.284613  5429 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:19.285454  5560 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:19.285890  5429 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:19.286089  5578 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:19.286360  5429 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:19.286648  5578 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.287087  5578 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.287945  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.288887  1706 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:19.289269  5454 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:19.289784  5454 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:19.290261  1628 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:19.290652  5454 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.292520  5296 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.269429  5448 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:19.294219  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:19.294723  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2362) took 3542 ms. Trace:
I20260709 13:02:19.295192  1708 rpcz_store.cc:276] 0709 13:02:15.752369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.254826 (+502457us) service_pool.cc:224] Handling call
0709 13:02:16.255225 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.836550 (+581325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.294267 (+2457717us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:19.294551 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778395520}
W20260709 13:02:19.296365  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44125.9us
I20260709 13:02:19.296646  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.297287  1708 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:19.297427  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:19.299147  2288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 985) took 3853 ms (client timeout 2374 ms). Trace:
W20260709 13:02:19.299515  2288 rpcz_store.cc:269] 0709 13:02:15.445918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.465207 (+ 19289us) service_pool.cc:224] Handling call
0709 13:02:19.298759 (+3833552us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54005d02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:19.298946 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099164032}
I20260709 13:02:19.305912  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:19.306591  2290 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:19.306879  5369 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.307850  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:19.308202  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:19.308601  2290 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:19.309753  5263 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.310344  5373 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:19.310804  5373 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.311206  5373 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.311702  5345 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.295302  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:19.289367  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2382) took 1219 ms. Trace:
I20260709 13:02:19.315282  1706 rpcz_store.cc:276] 0709 13:02:18.069797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.279697 (+1209900us) service_pool.cc:224] Handling call
0709 13:02:19.289284 (+  9587us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:19.271121  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.329108  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.329588  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.330073  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2367) took 1519 ms. Trace:
I20260709 13:02:19.331030  1704 rpcz_store.cc:276] 0709 13:02:17.765067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.248696 (+1483629us) service_pool.cc:224] Handling call
0709 13:02:19.283990 (+ 35294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.328595  1672 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44866 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1656)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1672)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1670)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1663)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1671)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2383)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47210 (request call id 28)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 590)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1652)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1653)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1654)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1662)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1674)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1675)
I20260709 13:02:19.265792  5557 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:19.351699  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.352452  2290 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:19.307660  5369 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:19.359975  5369 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:19.360052  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:19.360395  5369 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.364125  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.363962  5596 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:19.364867  1706 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:19.365391  5596 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:19.365697  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.366798  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.367385  1706 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:19.356876  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:19.370828  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.372152  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.372349  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.373229  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:19.374692  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:19.375463  2290 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:19.285917  5560 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:19.339288  5230 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.395831  1704 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:19.400804  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 105612us
I20260709 13:02:19.401008  1708 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:19.404309  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.407794  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.408645  1558 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:19.371917  5454 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:19.411897  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.392253  5366 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:19.417631  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 154041us
I20260709 13:02:19.419334  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.421639  5595 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:19.395529  5557 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:19.442028  1706 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:19.287597  5429 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.372095  5576 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.454509  5373 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.459399  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.460196  2288 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:19.460592  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.461055  5373 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.461272  1706 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:19.462152  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.462232  5576 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.462688  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:19.450585  5557 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.463665  5557 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:19.427034  5378 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:19.464006  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44804.5us
W20260709 13:02:19.464959  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48497.3us
I20260709 13:02:19.465301  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 988) took 3933 ms. Trace:
I20260709 13:02:19.465718  2292 rpcz_store.cc:276] 0709 13:02:15.531870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.807169 (+275299us) service_pool.cc:224] Handling call
0709 13:02:15.831284 (+ 24115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.831791 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.831832 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.832412 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.335435 (+1503023us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b180041b9c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:17.587107 (+251672us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:17.605563 (+ 18456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.639089 (+ 33526us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.464544 (+1825455us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5400626720. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce959dbe 00007f70ce964930 00007f70ce9648d8 00007f70ce964858 00007f70ce964813 00007f70ce963a9c 00007f70d484f626 00007f70d4849dd8 00007f70ce9a07cb 00007f70ce9a32c0
0709 13:02:19.464964 (+   420us) raft_consensus.cc:1685] finished
0709 13:02:19.464984 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.465127 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5714,"spinlock_wait_cycles":69208064,"thread_start_us":153956,"threads_started":1,"wal-append.queue_time_us":1736440}
I20260709 13:02:19.468983  5232 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:19.470268  5349 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:19.454155  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51059.4us
I20260709 13:02:19.471297  5223 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:19.472625  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:19.472743  5349 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.473626  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.457636  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.477612  1708 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:19.479022  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2371) took 1712 ms. Trace:
I20260709 13:02:19.479844  1708 rpcz_store.cc:276] 0709 13:02:17.766210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.297886 (+1531676us) service_pool.cc:224] Handling call
0709 13:02:19.478963 (+181077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.474138  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.481153  5378 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.481762  2292 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:19.410564  5228 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.483911  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.496932  1708 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 7: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:19.498492  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:19.450985  5595 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.503741  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:19.499795  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:19.503364  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.511034  5230 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:19.512997  1598 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44866 dropped due to backpressure: service queue is full with 50 items
I20260709 13:02:19.487320  5366 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:19.485776  5378 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:19.511662  5595 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.471246  5557 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.497560  5428 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:19.530807  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2351) took 3784 ms (client timeout 2070 ms). Trace:
W20260709 13:02:19.531279  1703 rpcz_store.cc:269] 0709 13:02:15.746203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.050936 (+304733us) service_pool.cc:224] Handling call
0709 13:02:19.522977 (+3472041us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b540016c954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:19.530635 (+  7658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120378624}
I20260709 13:02:19.531356  5369 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.532577  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.533351  1703 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:19.533694  5369 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:17.641252  1707 rpcz_store.cc:276] 0709 13:02:15.833590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.437469 (+1603879us) service_pool.cc:224] Handling call
0709 13:02:17.451806 (+ 14337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.452338 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:17.499458 (+ 47120us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.640058 (+140600us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.640264 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.640313 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:17.640361 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.640424 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.640721 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":75137,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":7340,"raft.queue_time_us":14783,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":106834,"threads_started":2}
I20260709 13:02:19.527053  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.536352  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.538308  1706 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:02:19.529462  5326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.069s	user 0.038s	sys 0.054s
I20260709 13:02:19.527923  5560 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.543280  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:19.544567  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:19.523149  5378 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.539059  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.547828  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.549081  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:19.551577  2292 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:19.539108  5326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.161s	user 0.039s	sys 0.054s
I20260709 13:02:19.538899  5223 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:19.554343  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83965.9us
I20260709 13:02:19.557106  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2364) took 1896 ms. Trace:
I20260709 13:02:19.557673  1707 rpcz_store.cc:276] 0709 13:02:17.671179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.535752 (+1864573us) service_pool.cc:224] Handling call
0709 13:02:19.557016 (+ 21264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.539480  5567 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:19.565685  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.566082  1708 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:19.567025  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.566936  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:19.567808  1706 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:19.568216  1630 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.561277  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.569139  1555 raft_consensus.cc:2376] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9. Re-sending same reply.
I20260709 13:02:19.555058  5428 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.572707  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:19.570554  5223 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:19.573777  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.574600  1708 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 7.
I20260709 13:02:19.574326  5223 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.575452  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.570619  5432 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.576535  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.575667  5567 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.577097  1708 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:19.577847  5432 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.577978  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.578639  1708 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:19.558781  5369 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:19.580075  5369 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.580440  5369 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.582875  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.573850  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.583869  5366 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:19.584092  1707 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:19.585058  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.585950  1707 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.586957  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.588074  5549 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:19.579691  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.587877  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2374) took 1820 ms. Trace:
I20260709 13:02:19.589178  1708 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:19.589564  5366 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:19.590225  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.590634  5366 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.592483  1708 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:19.589443  1707 rpcz_store.cc:276] 0709 13:02:17.767016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.586670 (+1819654us) service_pool.cc:224] Handling call
0709 13:02:19.587823 (+  1153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.601233  5567 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:19.609208  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.609743  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:19.610216  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2372) took 1843 ms. Trace:
I20260709 13:02:19.612067  1706 rpcz_store.cc:276] 0709 13:02:17.766468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.592674 (+1826206us) service_pool.cc:224] Handling call
0709 13:02:19.610139 (+ 17465us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.612957  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.613883  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.616086  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:19.616290  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.617013  1629 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:19.617363  1555 raft_consensus.cc:2376] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9. Re-sending same reply.
I20260709 13:02:19.613977  1707 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:19.622876  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.624424  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.629395  5443 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:19.627650  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.633064  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.634258  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2366) took 1869 ms. Trace:
I20260709 13:02:19.634646  1708 rpcz_store.cc:276] 0709 13:02:17.764755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.632608 (+1867853us) service_pool.cc:224] Handling call
0709 13:02:19.634204 (+  1596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.636231  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.636868  1708 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.637847  1630 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:19.640332  5557 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:19.640746  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 903) took 2267 ms (client timeout 1521 ms). Trace:
W20260709 13:02:19.641184  5608 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:02:19.645318  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:19.635320  1560 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:19.646119  1633 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:19.647845  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:19.641186  2297 rpcz_store.cc:269] 0709 13:02:17.373536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.560362 (+186826us) service_pool.cc:224] Handling call
0709 13:02:19.640546 (+2080184us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:19.640655 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146414336}
I20260709 13:02:19.634586  5578 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.656550  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.640748  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.660390  5557 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:19.665373  5557 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:19.666692  5557 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.689110  5369 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.694350  5578 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.700189  5595 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.701450  5369 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.701303  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.702287  1552 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.702415  5595 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.703104  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.704053  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.704270  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.704810  1552 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.705358  5367 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:19.704764  5378 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.705492  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.705742  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.706333  1559 raft_consensus.cc:2410] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:19.706884  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.707603  5378 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.707967  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.708570  5565 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:19.708784  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.709100  5565 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:19.709693  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.710191  1552 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.710203  5565 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.711090  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.711704  5565 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.712173  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.712795  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:19.713095  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1635) took 5260 ms. Trace:
I20260709 13:02:19.713579  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.713564  1709 rpcz_store.cc:276] 0709 13:02:14.452547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.382010 (+929463us) service_pool.cc:224] Handling call
0709 13:02:15.414541 (+ 32531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.845579 (+1431038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.712633 (+2867054us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:19.712889 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3289441664}
I20260709 13:02:19.714337  1552 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:19.715209  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.715343  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.716470  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.707147  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.719347  1709 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:02:19.720098  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.720803  1552 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:19.721539  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2383) took 1198 ms. Trace:
I20260709 13:02:19.722011  1709 rpcz_store.cc:276] 0709 13:02:18.523592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.714921 (+1191329us) service_pool.cc:224] Handling call
0709 13:02:19.721455 (+  6534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.724162  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.725430  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.726455  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.727533  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.728890  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:02:19.729589  1709 raft_consensus.cc:606] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:19.729918  1709 raft_consensus.cc:614] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:19.729748  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.731424  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.739193  1559 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:19.742657  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.743316  1559 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:19.744253  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.745396  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.746027  1559 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:19.738435  1552 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:19.747030  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.747840  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.748974  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.753779  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:19.754534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.755267  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.755748  1559 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.756512  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.757493  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:19.758028  1559 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:19.758795  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:19.759783  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:19.766211  1674 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 3.041s (SENT)
I20260709 13:02:19.766894  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:19.769801  5367 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:19.781900  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 584) took 3615 ms. Trace:
I20260709 13:02:19.782423  1709 rpcz_store.cc:276] 0709 13:02:16.166237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.730993 (+3564756us) service_pool.cc:224] Handling call
0709 13:02:19.731307 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.731550 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:19.779813 (+ 48263us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.781487 (+  1674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.781538 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:19.781588 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.781677 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.781828 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48365,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":278,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":86272,"thread_start_us":48549,"threads_started":2,"wal-append.queue_time_us":1200}
I20260709 13:02:19.784819  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.785876  1709 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:19.786820  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.788436  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.789155  5366 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.789549  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.789762  5366 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:19.790271  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:19.791668  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.792243  5366 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:19.792326  1709 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:19.793408  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.794054  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:19.795257  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.796343  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:19.797482  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.812629  1551 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:19.828110  5451 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:19.828750  5451 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:19.829540  5514 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 3.840s	user 0.019s	sys 0.081s
W20260709 13:02:19.832913  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.837801  5451 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:19.840667  5451 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:19.894826  1703 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:19.895638  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 586) took 3724 ms. Trace:
I20260709 13:02:19.896023  1703 rpcz_store.cc:276] 0709 13:02:16.171823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.894225 (+3722402us) service_pool.cc:224] Handling call
0709 13:02:19.894647 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.895327 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.895552 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.897699  1703 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:19.898275  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 587) took 3712 ms. Trace:
I20260709 13:02:19.898538  1703 rpcz_store.cc:276] 0709 13:02:16.185477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.896982 (+3711505us) service_pool.cc:224] Handling call
0709 13:02:19.897489 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.898089 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.898234 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.900266  1703 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:19.900750  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 590) took 3694 ms. Trace:
I20260709 13:02:19.900962  1703 rpcz_store.cc:276] 0709 13:02:16.206515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.899659 (+3693144us) service_pool.cc:224] Handling call
0709 13:02:19.899999 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.900543 (+   544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.900710 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.901969  1703 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:19.902431  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 591) took 3692 ms. Trace:
I20260709 13:02:19.902626  1703 rpcz_store.cc:276] 0709 13:02:16.210403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.901600 (+3691197us) service_pool.cc:224] Handling call
0709 13:02:19.901846 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.902263 (+   417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.902380 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.903612  1703 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:19.904140  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1652) took 2524 ms. Trace:
I20260709 13:02:19.904348  1703 rpcz_store.cc:276] 0709 13:02:17.380310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.903231 (+2522921us) service_pool.cc:224] Handling call
0709 13:02:19.903457 (+   226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.903977 (+   520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.904106 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.905400  1703 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:19.905921  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2380) took 1928 ms. Trace:
I20260709 13:02:19.906217  1703 rpcz_store.cc:276] 0709 13:02:17.977460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.904905 (+1927445us) service_pool.cc:224] Handling call
0709 13:02:19.905111 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.905682 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.905847 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:19.910588  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.911064  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.911643  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.911882  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1653) took 1127 ms. Trace:
W20260709 13:02:19.912222  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.912343  1703 rpcz_store.cc:276] 0709 13:02:18.784465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.906836 (+1122371us) service_pool.cc:224] Handling call
0709 13:02:19.907161 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:19.907462 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.907479 (+    17us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:19.908025 (+   546us) log.cc:844] Serialized 52 byte log entry
0709 13:02:19.908799 (+   774us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:19.908825 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.908871 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.910993 (+  2122us) raft_consensus.cc:1685] finished
0709 13:02:19.911013 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.911202 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"thread_start_us":499,"threads_started":1,"wal-append.queue_time_us":851}
I20260709 13:02:19.913946  1703 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:19.914426  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1654) took 1129 ms. Trace:
I20260709 13:02:19.914672  1703 rpcz_store.cc:276] 0709 13:02:18.784902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.913576 (+1128674us) service_pool.cc:224] Handling call
0709 13:02:19.913818 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.914261 (+   443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.914387 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.915578  1703 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:02:19.916059  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2386) took 1101 ms. Trace:
I20260709 13:02:19.916311  1703 rpcz_store.cc:276] 0709 13:02:18.815156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.915194 (+1100038us) service_pool.cc:224] Handling call
0709 13:02:19.915425 (+   231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.915878 (+   453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.916021 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.917344  1703 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:19.917820  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 597) took 1047 ms. Trace:
I20260709 13:02:19.918063  1703 rpcz_store.cc:276] 0709 13:02:18.870899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.916935 (+1046036us) service_pool.cc:224] Handling call
0709 13:02:19.917227 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.917670 (+   443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.917784 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.928915  1703 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:19.930181  1703 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:19.935407  1703 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:19.950281  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:19.950884  1628 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.958246  1628 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:19.958750  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:19.959249  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:19.959573  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2369) took 2193 ms (client timeout 2226 ms). Trace:
W20260709 13:02:19.960034  1700 rpcz_store.cc:269] 0709 13:02:17.765666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.958444 (+2192778us) service_pool.cc:224] Handling call
0709 13:02:19.959514 (+  1070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.961064  5223 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:19.961649  5223 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:19.962687  5223 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.011054  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 583) took 1012 ms. Trace:
I20260709 13:02:20.011561  1626 rpcz_store.cc:276] 0709 13:02:18.998816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.004209 (+  5393us) service_pool.cc:224] Handling call
0709 13:02:19.019704 (+ 15495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.086842 (+ 67138us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:19.160245 (+ 73403us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.370701 (+210456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.370756 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:19.370803 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.010607 (+639804us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5400274cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:20.010673 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.010848 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3024,"prepare.queue_time_us":105234,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":119,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":16423,"spinlock_wait_cycles":28832512,"thread_start_us":210011,"threads_started":2,"wal-append.queue_time_us":253110}
I20260709 13:02:20.015025  5560 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.019023  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.031883  2289 raft_consensus.cc:2376] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 8. Re-sending same reply.
I20260709 13:02:20.028430  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1673) took 4518 ms. Trace:
I20260709 13:02:20.032581  1560 rpcz_store.cc:276] 0709 13:02:15.509620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.372978 (+1863358us) service_pool.cc:224] Handling call
0709 13:02:17.373539 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.635065 (+2261526us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b540010af20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:02:19.646218 (+ 11153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.028131 (+381913us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54002bacd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:20.028349 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161769216}
I20260709 13:02:20.034073  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.034682  1560 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.022051  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:20.037752  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.039498  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.047703  1700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:02:20.048525  1700 raft_consensus.cc:606] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:20.048954  1700 raft_consensus.cc:614] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:20.047875  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.050899  2292 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:20.058228  1700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:02:20.058888  1700 raft_consensus.cc:606] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:20.053413  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.064317  1551 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.068351  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.050025  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.072206  1708 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:02:20.075510  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:20.076241  1700 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:20.076418  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2379) took 2124 ms (client timeout 1802 ms). Trace:
W20260709 13:02:20.076828  1705 rpcz_store.cc:269] 0709 13:02:17.954463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.076222 (+2121759us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:20.076332 (+   110us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:20.077647  1709 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.078583  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 589) took 3875 ms (client timeout 2368 ms). Trace:
W20260709 13:02:20.079064  1702 rpcz_store.cc:269] 0709 13:02:16.203427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.229543 (+ 26116us) service_pool.cc:224] Handling call
0709 13:02:20.078241 (+3848698us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:20.078481 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984290688}
I20260709 13:02:20.054836  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.081462  2289 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:20.085938  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:20.084908  1709 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:20.081950  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.084776  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:20.087363  1707 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:20.085664  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:20.086553  1702 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.092063  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:20.093865  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.102181  2292 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.095489  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1704) took 1223 ms. Trace:
I20260709 13:02:20.103479  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:20.103631  1558 rpcz_store.cc:276] 0709 13:02:18.872519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.158709 (+286190us) service_pool.cc:224] Handling call
0709 13:02:19.216719 (+ 58010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.409919 (+193200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.095226 (+685307us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000e8154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:20.095429 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433358592}
I20260709 13:02:20.104918  5232 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.105214  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.105386  5232 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:20.105851  1558 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.084256  1700 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:20.095176  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.104029  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.105863  5232 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.107651  2297 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.099969  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:20.115477  1626 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:20.119653  1628 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:02:20.122217  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.123178  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.122483  5232 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.123699  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.126922  5455 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:20.127473  1634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.010s	sys 0.016s
I20260709 13:02:20.127625  5455 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.127804  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.128755  5455 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:20.130244  5338 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:20.131099  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76348.7us
I20260709 13:02:20.131212  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.126989  1626 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:20.132107  1633 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 6: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:20.132121  5428 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.132817  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.133033  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.133711  1626 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:20.134277  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.134541  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.135289  1626 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:20.135921  1630 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:20.136164  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:20.136780  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 750) took 4008 ms (client timeout 2328 ms). Trace:
I20260709 13:02:20.137233  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:20.140595  5598 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.140646  2291 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:20.141184  5598 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:20.141696  5598 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.144439  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2310) took 4427 ms. Trace:
I20260709 13:02:20.144894  1631 rpcz_store.cc:276] 0709 13:02:15.716601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.717930 (+  1329us) service_pool.cc:224] Handling call
0709 13:02:15.745801 (+ 27871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.213340 (+1467539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.144072 (+2930732us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002747d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:20.144372 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2970851584}
I20260709 13:02:20.146317  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.147073  1631 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:20.137353  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.149187  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.149621  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.150403  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:20.127874  1634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.010s	sys 0.016s
I20260709 13:02:20.154268  1634 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:20.154680  1634 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.137117  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.135488  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.156770  1626 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:20.160998  2297 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:20.161988  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.162539  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.163159  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.163473  1630 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:20.164160  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.165359  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.165426  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.165670  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:20.150645  1631 raft_consensus.cc:2368] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:02:20.166350  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.166450  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:20.163372  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.167665  1700 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:20.161914  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.151693  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.107647  2289 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:20.169327  1560 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:20.187491  1628 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:02:20.103487  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.170687  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.187915  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.127032  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.194967  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.179922  1558 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6.
I20260709 13:02:20.181042  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:20.185915  1709 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:20.186573  1523 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.802s (SENT)
I20260709 13:02:20.198422  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.100756  1551 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:20.189774  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.190505  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.193269  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.171264  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2309) took 4455 ms. Trace:
W20260709 13:02:20.137188  2295 rpcz_store.cc:269] 0709 13:02:16.127974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.215435 (+ 87461us) service_pool.cc:224] Handling call
0709 13:02:20.136531 (+3921096us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54007bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:20.136715 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6262855552}
I20260709 13:02:20.194330  1705 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:20.196493  2290 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:20.196833  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69881.9us
I20260709 13:02:20.199818  1627 rpcz_store.cc:276] 0709 13:02:15.716217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.717829 (+  1612us) service_pool.cc:224] Handling call
0709 13:02:15.729349 (+ 11520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.174191 (+3444842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.170925 (+996734us) spinlock_profiling.cc:243] Waited 3.44 s on lock 0x7b540024fc54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:20.171199 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7917520256}
I20260709 13:02:20.199604  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.201349  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.202587  1551 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:20.198752  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:20.206276  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.208009  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.208393  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:20.209101  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.209676  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.209731  2297 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:20.210831  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.206276  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.208437  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.211599  2297 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.212278  1628 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:20.212646  1631 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.193800  5454 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:02:20.216791  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.218413  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.219218  1560 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.220324  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.514s (SENT)
I20260709 13:02:20.215726  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.227424  2295 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:20.215270  5367 raft_consensus.cc:2764] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:20.228416  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.228711  5367 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:20.229032  2295 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.268442  1675 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.574s (SENT)
I20260709 13:02:20.269428  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.318588  2290 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:20.319885  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.346138  2290 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:20.296993  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.350471  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.352387  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 912) took 1011 ms. Trace:
I20260709 13:02:20.352968  2297 rpcz_store.cc:276] 0709 13:02:19.340927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.351323 (+1010396us) service_pool.cc:224] Handling call
0709 13:02:20.351692 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.351949 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.351971 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.352041 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:20.352094 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.352169 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.352300 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.309334  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81552.6us
I20260709 13:02:20.300109  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:20.356101  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 56085.7us
W20260709 13:02:20.356478  1675 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.738s (SENT)
W20260709 13:02:20.273839  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.004s (SENT)
W20260709 13:02:20.292395  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76250.5us
I20260709 13:02:20.357995  5640 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:20.358006  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.290112  1633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9df64a488003414aac04146e56487c81"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:20.360060  1633 raft_consensus.cc:606] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:20.360173  5559 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:20.360662  1633 raft_consensus.cc:614] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:20.361371  1560 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:20.305429  1634 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:20.349833  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1674) took 4839 ms. Trace:
I20260709 13:02:20.368841  1554 rpcz_store.cc:276] 0709 13:02:15.509903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.422659 (+1912756us) service_pool.cc:224] Handling call
0709 13:02:18.231553 (+808894us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:18.231741 (+   188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.232201 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.232242 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.232868 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.250166 (+ 17298us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:18.250466 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.250548 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.679233 (+1428685us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b2400158648. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:02:20.349494 (+670261us) raft_consensus.cc:1685] finished
0709 13:02:20.349532 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.349742 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1973,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":17476608,"thread_start_us":15909,"threads_started":1,"wal-append.queue_time_us":110324}
I20260709 13:02:20.371111  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.371691  1554 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.372675  5647 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:20.373135  5647 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.345995  5600 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:20.336048  1704 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:20.375814  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2388) took 1490 ms. Trace:
I20260709 13:02:20.376178  1704 rpcz_store.cc:276] 0709 13:02:18.885389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.615328 (+729939us) service_pool.cc:224] Handling call
0709 13:02:20.375724 (+760396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:20.358544  5640 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:20.376577  5641 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:20.376762  5576 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:20.377131  5641 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.269008  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.378820  1627 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:20.378382  5641 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.368670  5648 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.380086  1634 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.364369  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc"
dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:47210
I20260709 13:02:20.332664  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:20.363381  5567 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.384416  1558 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:20.384363  1708 raft_consensus.cc:606] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:20.385311  1708 raft_consensus.cc:614] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:20.377225  5576 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.380105  5648 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.387169  5648 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.387001  5576 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:20.388365  5576 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.356487  2297 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:02:20.392385  1558 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:20.393673  1558 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:20.393882  5559 consensus_queue.cc:1175] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 11 status { last_received { term: 8 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:20.395141  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.829s (SENT)
I20260709 13:02:20.377017  5640 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.395639  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.398721  1560 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:20.399993  5019 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.373993  5647 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.367470  5232 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:20.409190  5647 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.368021  5454 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.379653  5641 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.411737  5454 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.414137  5640 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.366778  1551 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:20.406347  5378 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.297883  5367 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.419061  1634 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
W20260709 13:02:20.423056  1675 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.835s (SENT)
I20260709 13:02:20.451498  1634 pending_rounds.cc:85] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:02:20.451632  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:20.425143  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.436480  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:20.439285  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.453581  1554 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:20.433907  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:20.422899  1560 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:20.450198  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.455133  2289 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.427173  1708 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:20.455746  2289 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:02:20.455802  1674 leader_election.cc:336] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.602s (SENT)
I20260709 13:02:20.456463  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.451809  1634 pending_rounds.cc:107] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:02:20.458757  1634 raft_consensus.cc:2889] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.407544  2290 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
W20260709 13:02:20.465258  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.466149  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 192358us
I20260709 13:02:20.443730  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.480576  2290 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:20.481168  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.481693  2289 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.435577   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:02:20.487306  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43690.7us
W20260709 13:02:20.487366  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.368s (SENT)
I20260709 13:02:20.488214  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.489909  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:20.490533  1709 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.490892  1709 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:02:20.492919  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.494074  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.479357  5378 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:20.495713  1629 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:20.495965  5378 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.498309  5378 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:20.500000  5651 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:02:20.503513  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:20.504268  1555 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.512269  1560 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:20.513957  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.514330  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 91315.4us
I20260709 13:02:20.463897  5019 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.518494  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:20.515472  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.842s (SENT)
I20260709 13:02:20.446359  5454 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:20.520021  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.483579  5232 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.522315  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:20.519999  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.524127  1629 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:20.524923  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.525837  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:20.526261  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.526515  1706 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:20.527057  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1681) took 1106 ms. Trace:
I20260709 13:02:20.527341  1706 rpcz_store.cc:276] 0709 13:02:19.420693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.515336 (+1094643us) service_pool.cc:224] Handling call
0709 13:02:20.527011 (+ 11675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:20.486573  5642 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:20.506413  5590 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:20.528955  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 916) took 1009 ms. Trace:
I20260709 13:02:20.530097  2294 rpcz_store.cc:276] 0709 13:02:19.519721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.663102 (+143381us) service_pool.cc:224] Handling call
0709 13:02:20.528875 (+865773us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:20.443493  5367 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.531616  2289 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:20.533172  2289 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:20.529876  5590 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:20.535002  5646 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:20.535451  5646 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.536481  5367 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:20.536316  5646 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.537225  1707 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:20.537693  5646 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.540031  2295 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:20.541962  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.543274  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:20.545166  1675 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.883s (SENT)
I20260709 13:02:20.545735  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.530956  5232 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:20.535050  5590 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.540885  5640 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:20.551874  5642 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:20.558447  5652 raft_consensus.cc:2764] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:20.558992  5366 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:20.560937  5303 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.559316  5652 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:20.544256  5645 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:20.583828  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.584822  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.555891  5232 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:20.607810  5560 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.002s
I20260709 13:02:20.608858  1552 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:20.570837  5645 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:20.609750  5645 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.610010  5232 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.610522  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2316) took 1076 ms. Trace:
I20260709 13:02:20.611922  1552 rpcz_store.cc:276] 0709 13:02:19.533989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.748756 (+214767us) service_pool.cc:224] Handling call
0709 13:02:20.610304 (+861548us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b5400128ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:20.610460 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1975484672}
I20260709 13:02:20.583701  5640 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:20.613565  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68434.5us
I20260709 13:02:20.614506  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 588) took 1216 ms. Trace:
W20260709 13:02:20.619876  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77984.9us
I20260709 13:02:20.618654  1558 rpcz_store.cc:276] 0709 13:02:19.397588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.394627 (+997039us) service_pool.cc:224] Handling call
0709 13:02:20.394963 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.395193 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.395217 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.420071 (+ 24854us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.524551 (+104480us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:20.524585 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.524667 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.614205 (+ 89538us) raft_consensus.cc:1685] finished
0709 13:02:20.614235 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.614424 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":104119,"threads_started":1,"wal-append.queue_time_us":130335}
W20260709 13:02:20.621621  1675 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.893s (SENT)
I20260709 13:02:20.621428  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:20.622583  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.943s (SENT)
I20260709 13:02:20.588235  5652 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:20.623945  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.622329  5649 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:20.626360  5646 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.634516  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:20.644199  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:20.644649  1629 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:20.644934  1627 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:20.642359  5648 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:20.648809  5646 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.642634  5559 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:20.649343  5648 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:20.649071  1458 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 5 to 6, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 6 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:20.649359  5646 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.652545  5646 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:20.660602  5559 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.661772  5559 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.662422  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1008) took 1081 ms. Trace:
I20260709 13:02:20.662814  2297 rpcz_store.cc:276] 0709 13:02:19.581182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.395749 (+814567us) service_pool.cc:224] Handling call
0709 13:02:20.396097 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.505879 (+109782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.505911 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.530486 (+ 24575us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.615570 (+ 85084us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:20.615634 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.615735 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.662214 (+ 46479us) raft_consensus.cc:1685] finished
0709 13:02:20.662238 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.662366 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48662,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":174029,"threads_started":2,"wal-append.queue_time_us":92083}
I20260709 13:02:20.636644  5640 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.667332  5557 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:20.651008  1458 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 7, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 7 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:20.668746  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.669628  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.669683  5557 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:20.672585  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:20.673221  1702 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.674528  5230 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:20.674973  5230 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.675309  5230 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:20.644569  5645 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.677193  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:20.680390  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.681928  1702 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
W20260709 13:02:20.685014  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55020.7us
I20260709 13:02:20.685829  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.685976  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:20.687225  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.690459  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.703912  5223 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.704375  5223 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.704727  5223 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.708228  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 589) took 1594 ms (client timeout 1546 ms). Trace:
W20260709 13:02:20.708796  1635 rpcz_store.cc:269] 0709 13:02:19.113821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.114036 (+   215us) service_pool.cc:224] Handling call
0709 13:02:20.707923 (+1593887us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54000cd0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:20.708128 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289652096}
I20260709 13:02:20.709734  5451 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.710258  5451 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:20.712491  5645 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:20.702576  5648 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:20.713382  1599 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:20.713908  5648 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.714257  5559 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.715077  1599 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.714920  5648 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.688980  5557 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.715970  5349 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:20.716578  5349 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.717603  5349 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.701333  5578 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:20.721010  5578 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:20.721374  5578 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.722425  5559 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.722244  5578 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.723443  5578 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:20.724294  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.725024  2297 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:20.725672  5644 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:20.726107  5644 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.726205  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.726487  5644 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.726822  2290 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:20.727859  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.635581  5652 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:20.731564  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:20.731925  2293 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:20.733952  5645 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:20.737579  1675 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.125s (SENT)
I20260709 13:02:20.738132  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:20.703467  5428 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.571s	user 0.014s	sys 0.000s
I20260709 13:02:20.756862  5326 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.641568  5590 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:20.780305  1675 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.142s (SENT)
I20260709 13:02:20.824536  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:20.834184  1674 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.893s (SENT)
I20260709 13:02:20.834761  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:20.835390  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 144996us
I20260709 13:02:20.834676  5428 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:20.837252  5640 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.852298  5345 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:20.865655  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 85372.9us
W20260709 13:02:20.867123  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.196s (SENT)
W20260709 13:02:20.869609  1675 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.212s (SENT)
I20260709 13:02:20.876829  5661 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:02:20.854501  5652 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.881009  5590 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:20.883839  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:20.885937  5303 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:20.895742  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 582) took 1505 ms. Trace:
W20260709 13:02:20.897356  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.32996e+06us
I20260709 13:02:20.898350  5640 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:20.898860  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 584) took 1507 ms. Trace:
W20260709 13:02:20.899176  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.899896  5557 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:20.902233  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 585) took 1510 ms. Trace:
W20260709 13:02:20.904165  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.899313  1557 rpcz_store.cc:276] 0709 13:02:19.391119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.400901 (+  9782us) service_pool.cc:224] Handling call
0709 13:02:19.415046 (+ 14145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.452325 (+ 37279us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:19.498023 (+ 45698us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.898498 (+1400475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.898556 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:20.898602 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.898671 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.898802 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1279155,"prepare.queue_time_us":16155,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2594,"spinlock_wait_cycles":1955072,"thread_start_us":81972,"threads_started":1,"wal-append.queue_time_us":102940,"wal-append.run_cpu_time_us":1979,"wal-append.run_wall_time_us":1002001}
I20260709 13:02:20.902657  1559 rpcz_store.cc:276] 0709 13:02:19.391381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.760567 (+369186us) service_pool.cc:224] Handling call
0709 13:02:19.760817 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.761022 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:19.761315 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.901883 (+1140568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.901938 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:20.901990 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.902057 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.902177 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1117029,"prepare.queue_time_us":1161,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"thread_start_us":23111,"threads_started":1,"wal-append.queue_time_us":23437,"wal-append.run_cpu_time_us":959,"wal-append.run_wall_time_us":1000967}
I20260709 13:02:20.909292  5303 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:20.911988  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.228s (SENT)
I20260709 13:02:20.904626   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:20.912452  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.896292  1556 rpcz_store.cc:276] 0709 13:02:19.390513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.392853 (+  2340us) service_pool.cc:224] Handling call
0709 13:02:19.408759 (+ 15906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.408945 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:19.443291 (+ 34346us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.546596 (+103305us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.550650 (+  4054us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.026247 (+475597us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b180048e440. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:20.895203 (+868956us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b180048e440. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:20.895379 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.895435 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:20.895487 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.895552 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.895670 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6615,"prepare.queue_time_us":51307,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":1833,"raft.queue_time_us":6420,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":7087,"spinlock_wait_cycles":16993024,"thread_start_us":63171,"threads_started":1,"wal-append.queue_time_us":149724}
W20260709 13:02:20.875734  5598 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.006s
W20260709 13:02:20.911827  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 939763us
I20260709 13:02:20.917126  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.917953  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.927088  2293 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:20.930727  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:20.931878  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.931672  5303 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.932034  5663 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:20.932497  5640 consensus_queue.cc:1243] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:20.932865  5663 consensus_queue.cc:1243] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:20.933248  5391 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:20.934269  5391 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.934633  5391 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.934973  5391 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.935194  5559 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:20.936014  5559 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:20.936378  5559 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.937182  5429 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:20.937655  5428 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:20.938117  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:20.938629  5647 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:20.938998  5647 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:20.939816  5647 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.943181  5345 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:20.939550  5019 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.943704  5345 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:20.944913  5345 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:20.946524  5647 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.937233  5559 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.948470  5559 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:20.950706  2288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 905) took 1646 ms (client timeout 1846 ms). Trace:
W20260709 13:02:20.951193  2288 rpcz_store.cc:269] 0709 13:02:19.304275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.328051 (+ 23776us) service_pool.cc:224] Handling call
0709 13:02:20.950469 (+1622418us) spinlock_profiling.cc:243] Waited 99.2 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:20.950633 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228269440}
W20260709 13:02:20.952499  1674 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.949s (SENT)
I20260709 13:02:20.956570  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:20.957909  5019 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:20.959700  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:20.961277  5647 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:20.963162  1458 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 1 to 4, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 4 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:20.964885  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.847676  5223 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.955147  1551 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:20.952379  5391 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:20.962826  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:20.968842  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2315) took 1436 ms. Trace:
I20260709 13:02:20.971122  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:20.973378  1551 rpcz_store.cc:276] 0709 13:02:19.532623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.416242 (+883619us) service_pool.cc:224] Handling call
0709 13:02:20.954608 (+538366us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:20.954902 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.968448 (+ 13546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.968742 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14966016}
I20260709 13:02:20.972429  2288 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:20.978026  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:20.985170  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:20.987622  5367 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.989591  5367 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.989979  5367 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.981137  5019 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:20.984584  5640 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.974014  5598 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:21.012032  1454 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 289) took 29342 ms (client timeout 15000 ms). Trace:
I20260709 13:02:20.983848  5644 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.013564  5591 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:21.026221  1458 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 6, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:21.036124  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110231us
I20260709 13:02:21.038010  1458 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 2 to 3, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 3 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:21.046958  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161558us
I20260709 13:02:21.018478  5019 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:21.049209  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:21.052196  1707 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:21.052642  1707 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:21.040971  5443 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.053978  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.057052  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.057646  5223 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:21.058689  5644 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:21.053082  5590 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:21.059298  5590 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.059707  5590 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.061198  1702 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:21.059576  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:21.049640  5232 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.052778  5665 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:02:21.065291  1707 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.040658  5230 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.079226  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:21.091818  2294 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.080476  1707 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.083848  5391 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:21.085577  5590 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.087738  5367 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.095714  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.098330  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.098412  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.100085  1708 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.100159  1702 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:21.069689  5232 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:21.103206  5232 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:21.104197  5345 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.106899  5345 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:21.113852  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.057678  5598 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:21.118055  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.114166  1702 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:21.116567  5591 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.066450  5443 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.148124  1708 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:21.158196  5367 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:21.142593  5652 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:02:21.170079  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.174547  5451 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:21.014541  1454 rpcz_store.cc:269] 0709 13:01:51.656280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.656469 (+   189us) service_pool.cc:224] Handling call
0709 13:02:20.994599 (+29338130us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:02.409267 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:02.839798 (+430531us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:02.848629 (+  8831us) write_op.cc:435] Acquired schema lock
0709 13:02:02.850014 (+  1385us) tablet.cc:662] Decoding operations
0709 13:02:03.531881 (+681867us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:03.610666 (+ 78785us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:03.626364 (+ 15698us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:02:04.128377 (+502013us) tablet.cc:701] Row locks acquired
0709 13:02:04.128447 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:02:04.128699 (+   252us) write_op.cc:270] Start()
0709 13:02:04.128896 (+   197us) write_op.cc:276] Timestamp: P: 1783602124128612 usec, L: 0
0709 13:02:04.128928 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:02:04.129879 (+   951us) log.cc:844] Serialized 4392 byte log entry
0709 13:02:05.335795 (+1205916us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b180009da80. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:06.022804 (+687009us) op_driver.cc:464] REPLICATION: finished
0709 13:02:06.149048 (+126244us) write_op.cc:301] APPLY: starting
0709 13:02:06.149174 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:06.149549 (+   375us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:06.149583 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:06.151388 (+  1805us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:06.151715 (+   327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:02:06.151800 (+    85us) write_op.cc:312] APPLY: finished
0709 13:02:06.152218 (+   418us) log.cc:844] Serialized 83 byte log entry
0709 13:02:06.152421 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.152605 (+   184us) write_op.cc:454] Released schema lock
0709 13:02:06.152650 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:06.153085 (+   435us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.153116 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4890122,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":118328,"apply.run_cpu_time_us":3932,"apply.run_wall_time_us":4026,"num_ops":11,"prepare.queue_time_us":350511,"prepare.run_cpu_time_us":50384,"prepare.run_wall_time_us":1968411,"raft.queue_time_us":472587,"raft.run_cpu_time_us":15593,"raft.run_wall_time_us":288341,"replication_time_us":1782730,"spinlock_wait_cycles":11992448,"thread_start_us":656357,"threads_started":4,"wal-append.queue_time_us":266036,"wal-append.run_cpu_time_us":25178,"wal-append.run_wall_time_us":1929923}]]}
I20260709 13:02:21.178591  5230 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:21.181456  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.187485  1707 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:21.148408  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0b60 after lost signal to thread 434
W20260709 13:02:21.193135  2318 debug-util.cc:398] Leaking SignalData structure 0x7b0800322180 after lost signal to thread 1492
I20260709 13:02:21.205086  5443 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:21.206782  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.208164  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.209008  1626 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:21.209837  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.210568  1627 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:21.210867  1635 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:21.210959  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.215255  1626 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:21.212160  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.215839  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.216423  1632 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:21.212607  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.217808  1631 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.213794  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.219319  1634 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:21.211822  1627 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:21.220039  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.211819  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.222775  1633 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:21.229058  5640 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:21.229727  5640 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:21.231639  1628 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:21.236025  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0e80 after lost signal to thread 1584
W20260709 13:02:21.237411  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08003991c0 after lost signal to thread 1659
W20260709 13:02:21.238809  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08002a1d60 after lost signal to thread 1733
W20260709 13:02:21.239713  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.240070  1674 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.196s (SENT)
I20260709 13:02:21.243559  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.240891  1633 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:21.242038  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 996) took 3766 ms. Trace:
I20260709 13:02:21.245031  2291 rpcz_store.cc:276] 0709 13:02:17.476140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.007649 (+531509us) service_pool.cc:224] Handling call
0709 13:02:20.140270 (+2132621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.141272 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.241770 (+1100498us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54007bb1d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:21.241954 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4773146880}
I20260709 13:02:21.246346  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.249760  1633 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.251036  2294 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
W20260709 13:02:21.253369  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134241us
I20260709 13:02:21.254719  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.254384  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:21.256767  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.259193  1628 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:21.230994  5640 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:21.260696  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08000ddca0 after lost signal to thread 2321
W20260709 13:02:21.261039  2318 debug-util.cc:398] Leaking SignalData structure 0x7b080030e700 after lost signal to thread 2322
I20260709 13:02:21.263454  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.264294  1707 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:21.269034  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.270866  1707 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:21.271685  1707 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:21.271347  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:21.272732  1707 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:21.274788  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:21.275418  1708 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:21.277652  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:21.264040  5443 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:02:21.279480  1707 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.280787  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.282949  1627 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:21.283439  1633 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:21.257473  5232 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:21.264120  5557 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:21.288807  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.293409  1628 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:21.264225  5663 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:21.300047  1707 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:21.291548  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.300607  1707 pending_rounds.cc:85] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:02:21.300899  1707 pending_rounds.cc:107] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:02:21.301128  1707 raft_consensus.cc:2889] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.301189  1634 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:21.290072  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.290287  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:21.307982  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.309499  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.294792  5640 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.292034  5591 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:21.313988  5591 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.314409  5591 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.315300  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:21.316442  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:21.310411  1627 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:21.313416  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.318622  1634 raft_consensus.cc:2376] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10. Re-sending same reply.
I20260709 13:02:21.303992  5557 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:21.324317  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.325392  1627 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:21.325296  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.326532  1633 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:21.325095  1634 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.305514  5663 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.324609  5559 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:21.339424  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.344080  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:21.349907  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.354441  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:21.359959  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 989) took 5827 ms. Trace:
I20260709 13:02:21.360430  2296 rpcz_store.cc:276] 0709 13:02:15.532191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.261260 (+729069us) service_pool.cc:224] Handling call
0709 13:02:16.261574 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.402355 (+140781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.359506 (+4957151us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540031fbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:21.359779 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322222848}
I20260709 13:02:21.362416  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.377054  5590 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:21.383293  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.383370  2294 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:21.384188  5378 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:21.384599  5378 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:21.382272  2296 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:21.384217  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2378) took 3450 ms (client timeout 1760 ms). Trace:
W20260709 13:02:21.385924  1701 rpcz_store.cc:269] 0709 13:02:17.934295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.384057 (+3449762us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:21.384160 (+   103us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:21.388550  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.391134  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:21.392514  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.20586e+06us
I20260709 13:02:21.399695  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:21.392578  5465 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:21.393574  5338 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:21.395620  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 594) took 1849 ms. Trace:
I20260709 13:02:21.349414  5559 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:21.427168  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:21.432868  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.410036  1630 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.411481  1626 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:21.397541  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1719) took 1437 ms. Trace:
I20260709 13:02:21.421311  1554 rpcz_store.cc:276] 0709 13:02:19.546608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.454568 (+907960us) service_pool.cc:224] Handling call
0709 13:02:20.454954 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.455220 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:20.485380 (+ 30160us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.395118 (+909738us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.395292 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.395333 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:21.395381 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.395445 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.395562 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":865964,"prepare.queue_time_us":181930,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":868496,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":96,"thread_start_us":43238,"threads_started":1,"wal-append.queue_time_us":98237}
I20260709 13:02:21.438476  1552 rpcz_store.cc:276] 0709 13:02:19.959918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.617377 (+657459us) service_pool.cc:224] Handling call
0709 13:02:20.622631 (+  5254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.622847 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:20.755712 (+132865us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.397190 (+641478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.397246 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:21.397301 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.397372 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.397484 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":641195,"prepare.queue_time_us":13121,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":656434,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":7430,"thread_start_us":114781,"threads_started":1}
I20260709 13:02:21.425036  5253 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:21.452579  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:21.447620  5326 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:02:21.420912  5230 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:02:21.460180  1701 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:21.451352  5647 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:21.453893  1630 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:21.467700  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:21.465021  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 142374us
I20260709 13:02:21.425226  5557 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.475191  1708 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:21.475420  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:21.456754  5253 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:21.479395  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:21.519176  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.520864  1630 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:21.502385  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.515056  1626 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:21.523550  1674 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.628s (SENT)
I20260709 13:02:21.452418  5542 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 13:02:21.526057  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46776.3us
I20260709 13:02:21.529680  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:21.532229  1704 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:02:21.535597  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.535045  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.537325  1630 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:21.539179  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.539896  5465 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:21.544371  5253 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:21.546252  5663 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:21.478820  5649 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:21.548691  5691 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:21.509796  5652 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:02:21.470053  5559 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.567431  5663 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:21.579751  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:21.580272  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57243.9us
I20260709 13:02:21.588065  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:21.564716  5590 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:02:21.588778  5691 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:21.606439  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76869.7us
I20260709 13:02:21.608745  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:21.600016  5663 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:21.619727  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1718) took 1660 ms. Trace:
I20260709 13:02:21.619871  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:21.620272  1627 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:21.621091  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221538us
I20260709 13:02:21.621914  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:21.622874  5238 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:21.623256  5238 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:21.623716  5238 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.623924  5560 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.623258  5674 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:21.624599  5674 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:21.624465  5560 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.626034  5577 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:21.626852  5560 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:21.626771  5577 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:21.627635  5577 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:21.627979  5577 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:21.633345  2293 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:21.640092  2293 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:21.614651  5661 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.644891  2291 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:21.641400  5424 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 13:02:21.650367  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 724588us
I20260709 13:02:21.686450  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:21.688723  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.692123  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.692552  1551 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:21.664067  5345 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:21.684554  5367 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:21.701359  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.702525  1552 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:21.692960  1554 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:21.690048  1557 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.661530  5433 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.701786  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45212.3us
I20260709 13:02:21.712116  2291 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:21.727952  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.729163  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.729207  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.732633  1557 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:21.688455  5595 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.737843  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.729310  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.738626  1554 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:02:21.740249  1556 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:21.625576  5674 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.652006  5661 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.621456  5691 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.742470  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.744046  5661 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:21.744154  1557 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:21.745291  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.746444  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:21.750357  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.758651  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.770361  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.770843  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.771047  1558 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:21.772710  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.773453  1554 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:21.742632  5674 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.777122  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.716269  5228 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:02:21.781992  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1716) took 1822 ms. Trace:
W20260709 13:02:21.780589  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52766.2us
I20260709 13:02:21.664369  5577 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:21.766937  5338 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:21.723291  5230 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:02:21.717657  5663 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:21.776232  5701 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:02:21.759886  5595 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:21.802949  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.844520  1554 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:21.803593  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.805912  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.808481  5414 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:21.727413  5433 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.786099  1555 rpcz_store.cc:276] 0709 13:02:19.959145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.504961 (+545816us) service_pool.cc:224] Handling call
0709 13:02:20.505325 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.394034 (+888709us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:21.394352 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.630975 (+236623us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.631431 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.631573 (+   142us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:21.631605 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.631677 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.781733 (+150056us) raft_consensus.cc:1685] finished
0709 13:02:21.781754 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.781922 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1080993,"prepare.queue_time_us":281,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":1069823,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":11825,"thread_start_us":59518,"threads_started":1,"wal-append.queue_time_us":173356}
W20260709 13:02:21.838104  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.802146  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:21.852520  5699 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:21.867712  5595 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:21.870570  5433 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:21.872179  5422 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:21.872845  5422 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:21.873977  5422 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:21.875252  5591 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.003s
W20260709 13:02:21.875852  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 87414.7us
I20260709 13:02:21.876156  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.901398  1554 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:21.876384  5433 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:21.876834  5577 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:21.878115  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:21.885985  5696 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:02:21.780550  5378 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.890379  1552 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.879392  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.894898  1558 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:21.901515  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:21.911582  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 141856us
I20260709 13:02:21.911630  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.913743  1557 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:21.918608  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.922566  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:21.917209  5253 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.911952  5433 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:21.911928  5230 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:21.921595  5422 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:21.929363  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.930588  5710 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:02:21.928298  5338 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:21.935232  5433 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.928913  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:21.936595  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57291.9us
I20260709 13:02:21.950595  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:21.921475  5580 raft_consensus.cc:2764] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:21.922214  5595 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:21.954113  1558 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:21.913100  5591 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.938161  1552 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:02:21.956557  1552 pending_rounds.cc:85] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:02:21.939183  5230 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:21.935837  5253 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:21.942559  5706 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
W20260709 13:02:21.948007  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.952154   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:21.951807  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:21.953123  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:21.955971  1558 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:21.934255  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:21.957398  1554 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:21.957118  1552 pending_rounds.cc:107] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:02:21.961928  1552 raft_consensus.cc:2889] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.620239  1560 rpcz_store.cc:276] 0709 13:02:19.959641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.526972 (+567331us) service_pool.cc:224] Handling call
0709 13:02:20.531441 (+  4469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.531686 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:20.615432 (+ 83746us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.619382 (+1003950us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.619434 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:21.619480 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.619554 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.619669 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1003702,"prepare.queue_time_us":104117,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":1004898,"thread_start_us":71832,"threads_started":1}
I20260709 13:02:21.964869  5591 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:21.966408  5228 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:21.966856  5228 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.968700  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.969934  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:21.967223  5228 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:22.045424  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 75544.3us
I20260709 13:02:22.046605  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.050010  1558 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:22.050594  1558 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:21.956710  5710 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:22.054849  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 79456.3us
W20260709 13:02:22.075157  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.076047  5647 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.076205  5230 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:22.078974  5422 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:22.080750  1556 raft_consensus.cc:2410] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:22.048936  5577 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:22.086802  1626 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:22.087774  5647 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.082543  1558 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:22.125590  5704 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.312s	user 0.000s	sys 0.007s
I20260709 13:02:22.126299  5580 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:22.131757  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.131457  5367 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.132531  5367 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.135697  5367 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:22.145177  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52921.3us
I20260709 13:02:22.149989  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.081722  5709 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.157298  1558 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.091506  5577 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:22.164212  5577 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:22.102423  5292 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:22.177451  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 224367us
I20260709 13:02:22.185045  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:22.177268  5349 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:22.170495  5232 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:22.156328  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 255403us
I20260709 13:02:22.198128  1558 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:22.192585  5292 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:22.195117  5232 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.194819  5338 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:22.190491  5422 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:02:22.184520  5709 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:02:22.204186  5292 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:22.195065  5349 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:22.215878  5238 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.003s
W20260709 13:02:22.219581  5712 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:02:22.221534  5292 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:22.219228  5253 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:22.231735  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 175358us
I20260709 13:02:22.233130  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:22.231688  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 164957us
W20260709 13:02:22.223155  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 176656us
W20260709 13:02:22.234691  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.235242  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.239441  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:22.220889  5338 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.251631  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.227192  5238 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.255746  5349 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:22.256517  1554 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:22.232913  5720 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:22.262573  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.274160  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.275271  2293 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:22.286034  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.287839  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.289340  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:22.261510  5722 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:22.294963  5338 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:22.297513  1525 leader_election.cc:400] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:02:22.312114  1525 leader_election.cc:403] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:22.262451  5719 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:22.256533  5367 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:02:22.291529  5238 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:22.271982  5422 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:02:22.292238  5723 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:22.312525  5640 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
W20260709 13:02:22.347668  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102044us
W20260709 13:02:22.339143  5718 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:02:22.272265  5720 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:22.355851  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66580.1us
I20260709 13:02:22.359922  5577 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.361049  5720 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:22.344122  5722 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:22.366035  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.354584  5238 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
W20260709 13:02:22.363217  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.354130  5367 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:22.360124  5292 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.342347  5433 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.377132  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.372380  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.370312  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:22.371680  5723 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:02:22.372159  5367 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:22.392552  5724 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:22.393344  5292 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:22.398861  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:22.399650  1634 raft_consensus.cc:2376] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
I20260709 13:02:22.373227  5451 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:22.382889  5232 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:22.408404  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.414464  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.373118  5731 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:22.416426  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.414674  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.395812  5422 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:22.426828  1634 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:22.429257  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59010us
I20260709 13:02:22.430172  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.428159  5451 raft_consensus.cc:2764] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:22.454028  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.454759  1626 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:22.455499  1552 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:22.456341  1553 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:22.457345  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.457710  1559 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:22.457881  1553 raft_consensus.cc:2376] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9. Re-sending same reply.
I20260709 13:02:22.458752  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.458807  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:22.459494  1553 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:22.459704  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:22.460304  1558 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:22.461621  1560 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:22.467967  1555 raft_consensus.cc:1813] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Handling vote request from an unknown peer 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.468472  1555 raft_consensus.cc:2368] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11.
I20260709 13:02:22.463534  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:22.389995  5228 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.469210  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.467489  1559 raft_consensus.cc:2376] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10. Re-sending same reply.
I20260709 13:02:22.430629  5577 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:22.409475  5734 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 13:02:22.473604  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 209109us
I20260709 13:02:22.477772  5451 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:22.463977  5232 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:22.478547  5726 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:02:22.473997  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.478435  5451 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:22.478327  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:22.471575  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:22.481997  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:22.472153  5725 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
I20260709 13:02:22.473783  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:22.497500  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:22.483326  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:22.484014  5731 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:22.480225  5451 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.485358  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:22.485984  1552 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 8: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:22.486775  5728 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:22.507694  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.509378  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:22.510001  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.492820  5345 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:22.495395  5595 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.003s
I20260709 13:02:22.502140  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:22.531319  5731 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:22.543805  1632 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:02:22.546738  5732 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:22.551846  5223 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:22.554541  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.556282  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.556999  1628 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
I20260709 13:02:22.559947  1629 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:22.503371  5559 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:22.561129  5559 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:22.561761  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:22.562101  5559 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.563349  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.563869  1631 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:22.564157  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:22.564953  5433 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:22.501245  5232 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:22.520570  5343 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:22.503517  5230 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:22.520295  5577 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.534278  5424 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:22.566468  1629 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:22.567229  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.567363  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:22.568251  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:22.569360  2263 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.600s (SENT) [suppressed 91 similar messages]
I20260709 13:02:22.570577  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.571678  2294 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:22.532217  5731 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:22.578445  2295 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:22.595405  2293 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:22.521333  5345 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:22.586637  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.534746  5595 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:22.598765  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.603531  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:22.585670  5232 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:22.592057  5719 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:22.627470  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.630373  1557 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:22.600629  5230 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:22.603124  5433 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.637042  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:22.600185  5738 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:22.638444  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:22.594107  5736 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:22.607632  5424 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:22.622877  5232 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:22.635849  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 131298us
W20260709 13:02:22.646351  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 161039us
I20260709 13:02:22.622386  5542 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.656366  5542 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.657502  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.660146  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:22.661106  2291 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:22.629482  5719 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.662668  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:22.632198  5591 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:22.606398  5731 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.692560  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.693801  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.694903  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.695463  1551 raft_consensus.cc:2376] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10. Re-sending same reply.
I20260709 13:02:22.696533  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.696596  1635 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:22.698427  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:22.689811  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.658806  5542 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:22.665398  5663 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:22.702340  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65409.4us
I20260709 13:02:22.702337  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.699754  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:22.642302  5738 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.713284  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:22.715109  1705 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:22.717378  1705 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:22.717696  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:22.717763  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:22.718415  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:22.669853  5451 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:22.721223  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet c4b89801fe624f179701cbd4b98f1490 with delete_type TABLET_DATA_TOMBSTONED (TS 8b2c3934407d422495728f84374c6b53 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56080
I20260709 13:02:22.634913  5557 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:02:22.660343  5723 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:22.667574  5719 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:22.667900  5644 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.642711  5673 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:02:22.667482  5560 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:22.660745  5734 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:02:22.760443  1551 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:22.760743  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.706310  5542 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:22.648167  5595 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:02:22.776460  1632 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:02:22.774421  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 291148us
I20260709 13:02:22.778101  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:22.762883  5706 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:22.723315  5738 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:22.729396  5723 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.790427  1551 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:22.797736  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 125719us
I20260709 13:02:22.664153  5736 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.715294  5223 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:02:22.773049  5428 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.781198  5253 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.801872  1551 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:22.771447  5560 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:22.808121  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 595) took 1110 ms. Trace:
I20260709 13:02:22.844924  1556 rpcz_store.cc:276] 0709 13:02:21.697738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.851137 (+153399us) service_pool.cc:224] Handling call
0709 13:02:22.807917 (+956780us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54003fa054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:22.808059 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371853184}
I20260709 13:02:22.816931  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 601) took 1048 ms. Trace:
I20260709 13:02:22.846418  1554 rpcz_store.cc:276] 0709 13:02:21.768409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.961337 (+192928us) service_pool.cc:224] Handling call
0709 13:02:21.961621 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.256998 (+295377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.816671 (+559673us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000e9f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:22.816866 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677767040}
I20260709 13:02:22.819404  5228 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:22.825403  1558 raft_consensus.cc:2376] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 6. Re-sending same reply.
I20260709 13:02:22.815457  1552 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:22.856343  1552 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:22.777575  5542 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.834333  1551 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:22.791599  5323 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.722659  5345 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.800979  5723 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.773355  5644 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.867702  1552 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.882467  1552 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:22.868606  5323 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:22.888649  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.914906  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:22.915670  1701 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:02:22.805305  5685 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
W20260709 13:02:22.930570  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.861980  5542 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:22.938895  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 160905us
I20260709 13:02:22.947834  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:22.888438  5743 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:22.845966  5428 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.785655  5230 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.791992  5424 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.867499  5451 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:22.921124  5345 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.858042  5253 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:22.928166  5557 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:22.960376  5747 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:22.943091  5228 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:22.986466  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:22.983130  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:22.990154  1552 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:22.992242  1552 pending_rounds.cc:85] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 6
I20260709 13:02:22.992895  1552 pending_rounds.cc:107] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:02:22.993494  1552 raft_consensus.cc:2889] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:23.015347  5755 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:02:23.015653  5750 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:23.016428  5367 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.017649  5744 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:23.020354  5763 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:23.022850  1557 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:23.024488  5736 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.025007  5731 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.025216  5741 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:23.025883  5753 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:22.958192  5560 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.026403  5758 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:23.028860  5428 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.029372  5451 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:22.998999  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 200507us
I20260709 13:02:23.046530  5763 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:23.050351  5754 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:02:23.050935  5719 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.976743  5323 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.047546  5595 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:22.963424  5743 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.026309  5756 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:23.056666  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.056908  1634 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:23.036530  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 88848.5us
I20260709 13:02:23.058261  1627 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:02:23.017645  5735 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:22.985929  5557 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:22.996933  5710 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:02:22.995532  5228 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.044247  5640 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.035964  5757 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.077036  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.131142  2288 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:02:23.133845  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62776.1us
I20260709 13:02:23.137670  1627 pending_rounds.cc:85] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:02:23.137929  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:23.132014  5750 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:23.148490  1627 pending_rounds.cc:107] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:02:23.150086  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:23.151983  1627 raft_consensus.cc:2889] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.152458  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.067266  5433 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:02:23.139569  5367 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.151433  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:23.131661  5763 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.140216  5735 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:23.175590  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.064939  5753 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.144201  5770 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.151363  5744 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:23.073536  5758 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.162840  5736 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:23.159970  5762 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:23.199275  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.210888  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:23.213811  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.007s	sys 0.011s
I20260709 13:02:23.181584  5644 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.187337  5757 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:23.214267  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.007s	sys 0.011s
I20260709 13:02:23.144731  5756 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:23.225409  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.015s	sys 0.010s
I20260709 13:02:23.197479  5736 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:23.220525  2289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.012s
W20260709 13:02:23.297276  2289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.008s	sys 0.012s
W20260709 13:02:23.226056  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.015s	sys 0.011s
I20260709 13:02:23.149057  5640 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:23.218568  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 559818us
I20260709 13:02:23.304482  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.201767  5734 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.200577  5772 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:23.302189  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.217900  5744 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:23.302771  5736 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:23.296638  5731 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:23.306236  5742 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.303722  5253 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:23.372594  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221225us
I20260709 13:02:23.220242  5769 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 13:02:23.297760  5230 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:02:23.383834  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:23.322963  5710 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.374956  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.211867  5762 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.373389  5742 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:23.398277  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 93875.2us
I20260709 13:02:23.351222  5647 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.193805  5770 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.327925  5765 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:02:23.331893  5465 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:02:23.386338  5559 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:23.405900  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:23.437723  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 271319us
W20260709 13:02:23.423454  5767 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 13:02:23.404385  5591 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.005s
I20260709 13:02:23.383247  5772 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.387681  5673 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.444518  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.328286  5734 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.359853  5731 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.455853  5673 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.456511  5753 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.456905  5559 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.457144  5753 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.457396  5559 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.457576  5753 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.458279  5673 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.460513  5734 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.463932  5779 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:23.464610  5758 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.464543  5779 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.466754  5779 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.467236  5782 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:23.467890  5782 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:23.468150  5232 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.771s	user 0.010s	sys 0.002s
I20260709 13:02:23.468371  5782 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:23.468623  5232 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.469794  1705 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:23.477351  5557 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:23.477727  5738 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.000s
I20260709 13:02:23.451107  5378 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.478129  5738 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.479245  1704 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:23.479735  5774 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:02:23.480844  5764 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.457s	user 0.004s	sys 0.001s
I20260709 13:02:23.481231  5756 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.445924  5647 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:23.303758  5735 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.459443  5559 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.428253  5719 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.403465  5246 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.225s
W20260709 13:02:23.490195  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84405.8us
I20260709 13:02:23.448194  5644 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.501880  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.503876  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.504858  1707 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:23.505028  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.509749  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.455308  5591 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:23.507946  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 339130us
I20260709 13:02:23.490689  5772 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:23.498848  5345 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.006s
W20260709 13:02:23.525599  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.451428  5743 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.537055  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.546109  1708 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:23.537976  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.503144  5769 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.498833  5647 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:23.404183  5710 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.459470  5753 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.549043  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.514810  5557 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:23.513069  5673 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:23.539927  5776 tablet_replica.cc:333] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:02:23.557282  1601 leader_election.cc:400] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11.
I20260709 13:02:23.527262  5735 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.559252  1627 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:23.527033  5228 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.539635  5738 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:23.570677  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 138539us
I20260709 13:02:23.564738  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:23.510203  5736 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:23.578419  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 149552us
I20260709 13:02:23.515417  5644 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.598825  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:23.618078  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.618745  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.574213  5744 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:23.597842  5542 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.001s
I20260709 13:02:23.556123  5769 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.638720  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:23.624446  5772 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.572888  5591 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:02:23.640386  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:23.569849  5786 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:02:23.650985  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.615844  5785 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:23.651031  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.651714  1708 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.629016  5780 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:23.647593  1709 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:23.664043  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.627291  5738 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:23.664466  1708 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:23.669251  1708 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:23.683180  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1733) took 1225 ms. Trace:
I20260709 13:02:23.683749  1559 rpcz_store.cc:276] 0709 13:02:22.457403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.458529 (+  1126us) service_pool.cc:224] Handling call
0709 13:02:23.682905 (+1224376us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b540030a7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:23.683071 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16044416}
I20260709 13:02:23.652666  5797 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:23.642539  5542 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.686793  1700 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:02:23.657269  5753 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:23.654719  5778 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:23.686609  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.693943  1700 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:02:23.668570  5465 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:23.696632  1700 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:23.702698  1700 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.700203  1709 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:02:23.705651  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45506us
I20260709 13:02:23.706405  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:23.693517  5367 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.001s
I20260709 13:02:23.718837  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.705443  5780 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.720981  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.677884  5246 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:23.719570  2291 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:23.722934  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:23.730886  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.740615  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.747699  5780 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.748072  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.748096  5223 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:23.748773  5223 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:23.749219  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.744517  5803 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:23.721555  5731 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.750499  5685 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:23.751376  5428 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:23.742062  5710 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.751902  5758 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.720710  5783 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:02:23.752444  5758 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.753432  5790 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.753921  5790 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.754519  5758 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.741418  5736 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.749615  5223 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.754338  2296 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:23.756117  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.758792  1702 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.681928  5763 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.749840  1709 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 7.
W20260709 13:02:23.756798  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0581312199999502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 13:02:23.711687  5753 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:23.768635  5465 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.795951  5465 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.780951  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.797638  1703 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:23.798758  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.801115  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.801987  2289 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.803421  1701 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:23.756592  5428 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:23.806185  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.758317  5803 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:23.776610  5756 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.812266  5762 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.749856  5780 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.823943  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.706244  5778 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.829540  1702 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:23.832175  5749 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.832870  5749 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.834146  5232 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:23.713977  5744 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.814198  5786 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.846128  1702 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:23.848232  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.846864  1708 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:02:23.832218  5795 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:02:23.850657  5769 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:23.852922  5723 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.936s	user 0.006s	sys 0.006s
W20260709 13:02:23.853510  5422 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:02:23.863927  1702 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:23.861910  5778 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.823240  5644 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.872125  1707 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:23.873302  2289 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:23.865093  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.868386  1708 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:23.877043  1706 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:23.880265  1601 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.649s (SENT)
I20260709 13:02:23.880784  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:23.881647  5542 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.866910  5723 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.884262  5723 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.849627  5749 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:23.890265  5542 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:23.862083  5428 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.822571  5756 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.894157  5663 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:23.894590  5763 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.898831  5428 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.866014  5786 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:23.908113  5796 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
I20260709 13:02:23.897558  5663 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:23.868968  5753 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.909089  5663 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.910173  5663 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:23.910223  5742 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.907778  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:23.895946  1702 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:23.916419  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.917073  1707 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:23.880802  5223 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:23.922358  1708 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:23.923151  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.923945  2288 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:23.925979  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:23.929564  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:23.907928  5794 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.150s	user 0.000s	sys 0.002s
I20260709 13:02:23.948225  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:23.952292  1523 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.760s (SENT)
I20260709 13:02:23.932116  5742 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:23.963109  5422 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:23.968374  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.968942  5422 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.971380  5422 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:23.973366  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1724) took 1206 ms. Trace:
I20260709 13:02:23.975687  1704 rpcz_store.cc:276] 0709 13:02:22.767559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.770121 (+  2562us) service_pool.cc:224] Handling call
0709 13:02:22.770516 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.479676 (+709160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.973102 (+493426us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540016c454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:23.973300 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629415680}
I20260709 13:02:23.946782  5768 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:23.973915  5323 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:23.964193  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:23.915082  5803 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:23.956197  5738 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:23.982901  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:23.983705  1704 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.986738  5577 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.937965  5763 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:02:23.980624  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:23.965286  5742 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:23.939126  5223 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:23.980233  5323 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:23.982817  5738 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.001492  5746 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.225s	user 0.001s	sys 0.007s
W20260709 13:02:24.003890  5791 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:02:23.960983  5782 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:24.005237  5752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.007s	sys 0.018s
I20260709 13:02:23.994257  5742 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:24.020471  5756 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.005801  5591 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:24.018801  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.024612  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.992125  5768 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.030515  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.028199  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.032150  1704 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:24.033074  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:24.037314  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:23.987569  5577 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.025326  5738 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:24.038313  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.044003  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:24.045383  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.038347  5723 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:24.038230  5591 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.046557  5775 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.652s	user 0.003s	sys 0.000s
I20260709 13:02:24.047591  5640 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:24.047065  5323 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.058337  5323 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:24.032508  5752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.007s	sys 0.019s
I20260709 13:02:24.051662  1706 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:24.056856  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:24.039037  5422 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:24.051739  5644 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:24.049803  5595 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:24.070499  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:24.035929  5778 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:24.054572  5343 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.074076  5424 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:02:24.119052  5424 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:24.119587  5424 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:24.120729  5424 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.121878  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.052745  5749 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:24.119180  5230 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:24.122354  5424 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:24.076701  5223 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.051684  5673 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:02:24.080194  5761 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:24.081744  5685 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:24.082976  5782 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.068135  5710 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:02:24.084235  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:24.084374  5422 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:24.088418  5591 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.090487  5790 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.073678  5644 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.096604  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.096922  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.071583  5433 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:24.102468  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:24.112584  5735 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
W20260709 13:02:24.160569  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.057251  5723 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.172940  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.186926  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.125375  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.126394  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 632) took 1458 ms. Trace:
I20260709 13:02:24.188248  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:24.155045  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70894.9us
I20260709 13:02:24.175182  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.191666  1627 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:24.192144  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.192504  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.116528  5768 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:24.076620  5577 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.155818  5422 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:24.127980  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.202015  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:24.128345  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:24.139714  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.173554  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.166083  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1726) took 1339 ms. Trace:
I20260709 13:02:24.144371  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1017) took 1890 ms. Trace:
W20260709 13:02:24.174696  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.156827  5814 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.003s
I20260709 13:02:24.159917  5782 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:24.195474  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.130164  5770 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.195669  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.129298  5685 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.161654  5752 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.264299  5735 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.264585  5752 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.264851  5735 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.266808  5752 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.160027  5765 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.267196  5422 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.267707  5422 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.160331  5803 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.268482  5771 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:24.268996  5771 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:24.268680  5803 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.269436  5771 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:24.269972  5422 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.190609  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:24.125735  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:24.270970  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 80433.5us
I20260709 13:02:24.196470  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.249928  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.271656  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:24.271975  5789 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:24.272311  5465 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.272888  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:24.273469  2288 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:24.252470  1634 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:24.275952  5577 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:24.276360  5577 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.276697  5577 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:24.276793  5782 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.276644  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.279291  1634 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:24.156585  5644 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:24.217514  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.126554  5778 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:24.132714  5560 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:24.253067  1705 rpcz_store.cc:276] 0709 13:02:22.826617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.845151 (+ 18534us) service_pool.cc:224] Handling call
0709 13:02:22.845571 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.470164 (+624593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.165814 (+695650us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003dc554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:24.166011 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435089152}
I20260709 13:02:24.188781  1635 rpcz_store.cc:276] 0709 13:02:22.668467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.706999 (+ 38532us) service_pool.cc:224] Handling call
0709 13:02:22.730941 (+ 23942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.731255 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.731299 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.764564 (+ 33265us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.191513 (+426949us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:23.191554 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.191620 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.045882 (+854262us) spinlock_profiling.cc:243] Waited 9.11 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:24.126060 (+ 80178us) raft_consensus.cc:1685] finished
0709 13:02:24.126103 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.126333 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177173,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":30904,"spinlock_wait_cycles":20961024,"thread_start_us":391722,"threads_started":1,"wal-append.queue_time_us":547485}
I20260709 13:02:24.256033  2296 rpcz_store.cc:276] 0709 13:02:22.253603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.253743 (+   140us) service_pool.cc:224] Handling call
0709 13:02:22.254047 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.758354 (+1504307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.143977 (+385623us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400350cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:24.144271 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3449589504}
I20260709 13:02:24.294629  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.295804  1552 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:24.167398  5756 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:24.124850  5230 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:24.124533  5749 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:24.124652  5595 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:24.252285  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.258862  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.306156  1630 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:24.306350  1633 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:24.259351  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:24.323935  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64703.6us
I20260709 13:02:24.325048  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.280879  5685 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:24.259459  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57553.7us
W20260709 13:02:24.125512  5810 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:24.247882  5790 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:24.123706  5742 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.263796  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.331740  1629 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:24.267652  5765 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.271013  5803 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.253052  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.280318  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.298915  5778 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:24.276681  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:24.368558  1709 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:24.245800  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.252682  1702 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:24.283566  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:24.286960  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.250841  1628 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:24.287110  1559 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:24.289742  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.154505  5710 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:24.202936  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.300518  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.300719  5773 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:24.301177  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.302767  5723 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.245211  5776 raft_consensus.cc:2243] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.304697  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.266767  5735 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.183849  5433 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:24.324153  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:24.326632  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.325438  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:24.327749  1556 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.280357  5782 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.259972  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.330888  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.291121  5343 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.202927  5414 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 13:02:24.281196  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:24.333796  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:24.293846  5323 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.360695  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.369382  1632 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:24.322088  5559 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:24.376417  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.478035  1627 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:24.490232  1553 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:24.489763  5591 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:02:24.291570  5422 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:02:24.330505  5731 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:24.483718  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.486479  1551 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:24.588069  5780 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:24.656352  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:24.657310  1551 raft_consensus.cc:2376] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11. Re-sending same reply.
I20260709 13:02:24.658696  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:24.488212  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.489096  1630 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:24.305255  5768 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.171s
I20260709 13:02:24.325969  5595 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.129032  5761 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.321727  5378 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 13:02:24.360569  5789 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.798257  5789 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.800114  5595 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.476879  5710 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:24.325742  5749 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.326944  5756 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.361565  5790 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:24.597298  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.619618  1554 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:24.500098  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:24.805496  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.006s	sys 0.007s
I20260709 13:02:24.528972  1633 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:24.528751  1705 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:24.502251  1560 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.547482  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.807583  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.514570  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:24.495154  2288 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:24.808409  1554 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.622402  1552 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.546192  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.622983  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.516125  1702 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 5. (term mismatch)
I20260709 13:02:24.810227  1702 pending_rounds.cc:85] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 4
I20260709 13:02:24.623911  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:24.811527  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.001s	sys 0.013s
I20260709 13:02:24.525496  5230 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.550817  1709 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:24.479251  2292 raft_consensus.cc:2410] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:24.528550  5640 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.818419  5640 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.819303  5749 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:24.536079  1446 thread.cc:641] dir /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.012s	sys 0.016s
I20260709 13:02:24.590755  5785 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.705578  5577 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.524744  5223 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.618198  5735 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.499002  5786 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.517942  5773 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:24.714339  1632 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:24.519606  5803 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.190s
I20260709 13:02:24.708156  5776 pending_rounds.cc:70] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Trying to abort 1 pending ops.
I20260709 13:02:24.547124  5433 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.643563  5323 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.722560  5738 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.000s
I20260709 13:02:24.745416  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:24.746207  2290 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.020s	sys 0.054s
W20260709 13:02:24.870186  2290 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.020s	sys 0.054s
W20260709 13:02:24.870568  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 125288us
I20260709 13:02:24.497536  5771 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.876464  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 595276us
I20260709 13:02:24.637586  5559 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.638772  5343 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.762059  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.768342  5819 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:02:24.752560  5817 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:02:24.590670  5778 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.790705  5742 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.898499  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:24.769209  5424 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.791785  5765 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.794337  1629 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:24.759719  5414 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.796851  1628 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
W20260709 13:02:24.805932  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.006s	sys 0.007s
W20260709 13:02:24.615613  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 290231us
W20260709 13:02:24.881713  5685 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.005s
I20260709 13:02:24.910404  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:24.809435  1556 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:24.914218  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.914770  1556 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:24.915477  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:24.812418  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.916896  1554 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:24.917677  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.918141  1554 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:24.918658  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.919173  1554 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.805694  2296 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:24.810786  2291 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:24.578826  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:24.812995  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.001s	sys 0.013s
I20260709 13:02:24.813128  1702 pending_rounds.cc:107] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:02:24.921459  1702 raft_consensus.cc:2889] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.920781  1559 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:24.820962  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:24.922502  1551 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:24.805176  5673 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.011s	sys 0.000s
W20260709 13:02:24.931730  5673 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.012s	sys 0.000s
I20260709 13:02:24.936254  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.936882  5756 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.938210  1551 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:24.939237  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.803165  5557 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:24.361583  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:24.940132  5557 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:24.940807  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 579260us
I20260709 13:02:24.942056  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:24.942286  5557 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.853359  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.857793  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:24.943209  1560 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:02:24.943323  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 85646.3us
W20260709 13:02:24.844424  1446 thread.cc:608] dir /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.017s	sys 0.019s
I20260709 13:02:24.944361  5761 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.862284  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.825165  5789 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:24.945624  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:24.896482  1557 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.004s	sys 0.016s
I20260709 13:02:24.945896  2297 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:24.946215  1557 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.004s	sys 0.016s
I20260709 13:02:24.865533  5776 pending_rounds.cc:77] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Aborting op as it isn't in flight: id { term: 3 index: 5 } timestamp: 7305634150532644864 op_type: NO_OP noop_request { }
I20260709 13:02:24.947077  5776 raft_consensus.cc:2889] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:02:24.947265  5323 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.947535  5776 raft_consensus.cc:2272] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.947868  5323 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:24.948506  5323 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:24.950589  5776 ts_tablet_manager.cc:1916] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:24.951406  1551 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:24.855966  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:24.955751  1552 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:24.907939  5577 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:24.864168  5223 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.864415  5710 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.860958  5785 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.923890  1556 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:24.881156  5595 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:24.919649  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.980824  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.880672  5738 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.929144  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:24.895135  5723 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.944190  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.877923  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:24.986511  1560 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
W20260709 13:02:24.986649  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108786us
I20260709 13:02:24.904690  5343 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.864006  5735 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.907783  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:25.004904  5785 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.878472  5773 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.005367  5735 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.005872  5773 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.982023  5778 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.008766  1553 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:25.008865  5786 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:25.009377  5786 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:25.010229  5786 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.011492  1553 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:24.992053  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:24.969527  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:25.013486  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44070.6us
I20260709 13:02:24.971724  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.972586  1709 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:24.974745  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:24.927198  5757 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:24.985347  2293 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
W20260709 13:02:24.942394  5644 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 13:02:24.967096  5323 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:24.989984  5723 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.990576  1560 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 5. (index mismatch)
W20260709 13:02:24.992513  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.030735  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:25.031447  1703 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.031649  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:25.032770  1633 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:24.995568  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.034526  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.011402  5786 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:24.998741  1707 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:02:25.040486  5223 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.041687  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.042214  5790 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:25.042917  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.043515  1707 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:25.044575  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.048698  5771 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:24.963125  5685 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.049517  5723 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.050410  2295 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:25.051182  5685 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.051267  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:25.021067  5428 consensus_queue.cc:1175] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 14 status { last_received { term: 14 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 14 index: 8 } }
I20260709 13:02:25.051728  5663 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:25.052244  5663 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:25.053095  5663 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.054569  5663 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:25.055157  5230 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:25.011842  5595 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:25.056999  5595 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:25.057963  5595 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.977238  5560 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:25.026537  5785 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:25.061240  5785 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.061632  5785 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:25.062620  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.063323  5710 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.026244  5644 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.065264  5738 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.066022  5644 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.066632  2289 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:25.052011  1635 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:24.967733  5640 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:24.910616  5424 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:24.980365  5756 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:24.978135  5765 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:25.025296  5735 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.080492  5765 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.080921  5765 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.080956  5735 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.082406  5433 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.025090  5768 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:25.085072  1559 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:25.023537  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.024487  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.094908  1702 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:02:25.096113  1555 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.004s	sys 0.032s
W20260709 13:02:25.096561  1555 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.004s	sys 0.032s
W20260709 13:02:25.096936  1704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.019s	sys 0.017s
W20260709 13:02:25.097312  1704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.019s	sys 0.017s
I20260709 13:02:25.099014  1702 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [6.7-8.8]
W20260709 13:02:25.101143  1701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.004s	sys 0.016s
I20260709 13:02:25.102536  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.085099  5756 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:25.024644  1552 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:25.025791  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.111265  1709 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:24.959427  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:24.925719  5414 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:24.989924  1551 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:25.059533  1705 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:25.077409  1560 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:25.079195  5559 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:25.080108  5822 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:02:25.116930  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 157384us
I20260709 13:02:25.106933  5735 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.124636  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:25.115727  1552 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:25.154289  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.155822  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:25.156474  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.157585  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:25.158442  2288 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:25.164177  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:25.130539  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.132836  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:25.166113  1705 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:25.166790  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.167233  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.135620  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:25.118798  1700 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:02:25.146888  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:25.149948  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:25.170946  2293 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:25.168329  1705 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:25.174100  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:25.116093  5833 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.182649  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.184662  1705 raft_consensus.cc:2410] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:25.177170  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.177675  2289 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:25.179095  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:25.120246  5428 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:25.182577  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:25.174310  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.181772  5433 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:02:25.206967  1709 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:25.140074  5757 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.194290  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.214428  5414 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.217589  5424 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.220501  5433 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:25.221014  5433 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:25.232249  1703 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:25.234225  2295 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:25.173102  5559 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:25.181973  5789 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:25.182793  5815 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:25.182525  5323 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:25.210402  5253 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:25.252933  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 945) took 1128 ms. Trace:
I20260709 13:02:25.256135  2294 rpcz_store.cc:276] 0709 13:02:24.124058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.125822 (+  1764us) service_pool.cc:224] Handling call
0709 13:02:24.126117 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.126505 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.126534 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.127024 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.529622 (+402598us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:24.529671 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.529746 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.252639 (+722893us) raft_consensus.cc:1685] finished
0709 13:02:25.252668 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.252856 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":905,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"thread_start_us":402246,"threads_started":1,"wal-append.queue_time_us":784012}
W20260709 13:02:25.253904  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59712us
I20260709 13:02:25.179087  5465 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:02:25.263871  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.003s	sys 0.022s
W20260709 13:02:25.264287  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.003s	sys 0.022s
I20260709 13:02:25.238056  5559 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:25.267112  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.012s	sys 0.016s
W20260709 13:02:25.270256  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.012s	sys 0.016s
W20260709 13:02:25.270632  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 978209us
W20260709 13:02:25.277686  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.251621  5789 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:25.266546  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:25.254307  5323 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.269053  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.259902  5779 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:25.348208  5779 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:25.349489  5779 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.255947  5253 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:25.356104  5253 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:25.207710  5833 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.361897  5833 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:25.301834  5790 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:02:25.307802  5789 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.308269  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:25.368636  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60449us
I20260709 13:02:25.371098  5644 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:25.371557  5644 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.371942  5644 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.299661  5424 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:02:25.376571  5738 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.310969  5753 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.311021  5414 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:25.313783  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.389890  1703 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:25.233721  5756 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.235653  5757 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.299708  5640 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:25.300201  5559 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.320115  1455 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:25.322712  5731 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:25.265754  5433 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:25.379567  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.388142  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:25.393864  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 83957.9us
I20260709 13:02:25.299716  5768 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.377508  5323 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.386775  5738 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:25.411243  2290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.015s	sys 0.023s
I20260709 13:02:25.434966  1455 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:25.437081  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49034us
I20260709 13:02:25.466954  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:25.468792  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:25.437368  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:25.417273  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.474053  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:25.435910  1675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.018s	sys 0.015s
W20260709 13:02:25.457870  2296 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.024s
W20260709 13:02:25.476234  1675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.018s	sys 0.015s
W20260709 13:02:25.476347  2296 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.007s	sys 0.024s
W20260709 13:02:25.476668  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 603892us
W20260709 13:02:25.478119  2290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.015s	sys 0.024s
I20260709 13:02:25.479333  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 947) took 1350 ms. Trace:
I20260709 13:02:25.489377  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:25.493839  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.494772  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
W20260709 13:02:25.446060  5673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.012s	sys 0.021s
I20260709 13:02:25.497764  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:25.500010  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.505720  1634 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:25.435026  5731 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.490653  1455 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 8, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.510102  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:25.406311  5424 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 13:02:25.504765  1708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.019s
W20260709 13:02:25.502863  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.511284  5673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.013s	sys 0.022s
I20260709 13:02:25.503505  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:25.513520  1708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.010s	sys 0.019s
I20260709 13:02:25.515797  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:25.516579  1706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.020s
W20260709 13:02:25.516968  1706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.011s	sys 0.020s
I20260709 13:02:25.458559  5414 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:25.520936  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.517028  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:25.407696  5753 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.522557  1633 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.486307  5710 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:25.517879  5761 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.016s
I20260709 13:02:25.508394  5723 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:25.490228  5735 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:25.531872  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 298) took 36338 ms (client timeout 15000 ms). Trace:
I20260709 13:02:25.532470  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.504539  5433 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:25.533728  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.541529  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.519413  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:25.413058  5785 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.549706  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.552209  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.553113  1629 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:25.519399  5773 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:25.457857  5768 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.525929  1455 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 4 to 8, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:25.537467  1557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.009s	sys 0.013s
I20260709 13:02:25.555019  1629 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:02:25.513063  5731 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.555598  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.561872  1631 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:25.523360  5428 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:25.508260  5465 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:02:25.530511  5230 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:25.562912  1557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.009s	sys 0.013s
W20260709 13:02:25.533705  5761 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.013s	sys 0.016s
I20260709 13:02:25.573421  1628 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:25.563529  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.025s	sys 0.013s
W20260709 13:02:25.574064  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.025s	sys 0.013s
W20260709 13:02:25.574759  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 664677us
W20260709 13:02:25.565585  1554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.013s	sys 0.031s
I20260709 13:02:25.529927  5831 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.560117  1627 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:25.532698  5560 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:25.556424  5735 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:25.563213  1626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.018s	sys 0.019s
W20260709 13:02:25.559106  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.016s	sys 0.019s
I20260709 13:02:25.536834  5414 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:25.576159  1554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.013s	sys 0.031s
I20260709 13:02:25.557082  5723 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.590526  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1745) took 1305 ms. Trace:
I20260709 13:02:25.563692  5773 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:25.584772  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.016s	sys 0.019s
I20260709 13:02:25.580129  1455 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:25.583758  5560 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:25.594559  5422 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:25.595031  1455 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 9 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:25.596876  1556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.001s	sys 0.019s
W20260709 13:02:25.537878  1458 rpcz_store.cc:269] 0709 13:01:49.202010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.202270 (+   260us) service_pool.cc:224] Handling call
0709 13:02:25.523992 (+36321722us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:21.256951 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:21.278598 (+ 21647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:21.278659 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:02:21.278677 (+    18us) tablet.cc:662] Decoding operations
0709 13:02:21.378907 (+100230us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:21.379009 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:21.379037 (+    28us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:02:21.379983 (+   946us) tablet.cc:701] Row locks acquired
0709 13:02:21.380035 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:02:21.380196 (+   161us) write_op.cc:270] Start()
0709 13:02:21.380336 (+   140us) write_op.cc:276] Timestamp: P: 1783602141380133 usec, L: 0
0709 13:02:21.380360 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:02:21.381038 (+   678us) log.cc:844] Serialized 4009 byte log entry
0709 13:02:21.693918 (+312880us) op_driver.cc:464] REPLICATION: finished
0709 13:02:21.921221 (+227303us) write_op.cc:301] APPLY: starting
0709 13:02:21.926049 (+  4828us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:21.933263 (+  7214us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:21.933379 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:22.071424 (+138045us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:22.096326 (+ 24902us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:02:22.096398 (+    72us) write_op.cc:312] APPLY: finished
0709 13:02:22.096841 (+   443us) log.cc:844] Serialized 77 byte log entry
0709 13:02:22.097085 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:22.097246 (+   161us) write_op.cc:454] Released schema lock
0709 13:02:22.097316 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:22.097813 (+   497us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:22.097839 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11878010,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":218898,"apply.run_cpu_time_us":5345,"apply.run_wall_time_us":176577,"num_ops":10,"prepare.queue_time_us":148333,"prepare.run_cpu_time_us":44963,"prepare.run_wall_time_us":299538,"raft.queue_time_us":79131,"raft.run_cpu_time_us":60290,"raft.run_wall_time_us":127435,"replication_time_us":313367,"thread_start_us":296483,"threads_started":4,"wal-append.queue_time_us":164334,"wal-append.run_cpu_time_us":14050,"wal-append.run_wall_time_us":1586274}]]}
I20260709 13:02:25.590437  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:25.584543  1626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.018s	sys 0.019s
W20260709 13:02:25.566430  5577 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.012s	sys 0.019s
I20260709 13:02:25.602469  1559 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:02:25.573122  5230 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:25.587816  5735 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:25.592350  5723 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:25.601042  5577 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.019s
I20260709 13:02:25.598105  1552 raft_consensus.cc:2410] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:25.591022  1557 rpcz_store.cc:276] 0709 13:02:24.285189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.285298 (+   109us) service_pool.cc:224] Handling call
0709 13:02:24.285566 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.285796 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:24.946588 (+660792us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.590170 (+643582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.590225 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:25.590272 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.590337 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.590458 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16730,"prepare.queue_time_us":823530,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":19809,"thread_start_us":1286992,"threads_started":2}
I20260709 13:02:25.618547  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.596622  5773 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.620081  1455 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.616389  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.610280  5230 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:25.638897  1559 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:25.612602  5765 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 13:02:25.644526  1455 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.646950  5765 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.588685  5831 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:25.650449  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.022s	sys 0.022s
W20260709 13:02:25.654601  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.022s	sys 0.023s
W20260709 13:02:25.634020  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.010s	sys 0.007s
W20260709 13:02:25.655892  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.011s	sys 0.007s
I20260709 13:02:25.601912  5560 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:25.674975  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 495956us
I20260709 13:02:25.641518  5465 raft_consensus.cc:2764] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:02:25.660080  1702 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.014s	sys 0.023s
W20260709 13:02:25.687103  1702 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.015s	sys 0.023s
W20260709 13:02:25.660544  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 485125us
W20260709 13:02:25.597249  1556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.001s	sys 0.019s
I20260709 13:02:25.658638  5803 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:25.658187  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.012s	sys 0.029s
I20260709 13:02:25.652022  5757 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.678210  1559 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:25.647830  5414 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:25.674226  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 460863us
W20260709 13:02:25.676440  1701 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.004s	sys 0.012s
I20260709 13:02:25.672014  5752 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:25.699363  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.738552  1557 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:25.699555  1455 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 7, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 7 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:25.702576  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.012s	sys 0.029s
I20260709 13:02:25.703066  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.699252  5560 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:25.723094  5738 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:25.729816  5753 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.753566  1455 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 3 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 5 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:25.754539  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.693s (SENT)
I20260709 13:02:25.758541  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.758735  1633 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
W20260709 13:02:25.759868  1709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.016s
W20260709 13:02:25.760259  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 295328us
I20260709 13:02:25.734189  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:25.732620  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1739) took 1560 ms. Trace:
W20260709 13:02:25.760586  1709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.000s	sys 0.016s
W20260709 13:02:25.732358  1701 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.004s	sys 0.012s
I20260709 13:02:25.764832  1708 rpcz_store.cc:276] 0709 13:02:24.172473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.197097 (+ 24624us) service_pool.cc:224] Handling call
0709 13:02:24.197500 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.197942 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.197985 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.198594 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.910401 (+711807us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:25.514183 (+603782us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.730942 (+216759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.732359 (+  1417us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.732406 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:25.732418 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.732547 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14128,"prepare.queue_time_us":761153,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2921,"raft.queue_time_us":81560,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":7797,"thread_start_us":1461216,"threads_started":3,"wal-append.queue_time_us":738735}
W20260709 13:02:25.767805  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.600s (SENT)
I20260709 13:02:25.769663  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.770988  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:25.777287  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:25.769629  1675 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.022s (SENT)
I20260709 13:02:25.768795  1455 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 4 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 5 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.768723  5640 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:25.782162  1455 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 8, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.780591  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:25.783198  5640 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:25.752789  5768 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:25.798092  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 239370us
W20260709 13:02:25.804932  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 110192us
I20260709 13:02:25.805667  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:25.808429  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.768919  5738 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.806152  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:25.853977  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47977.6us
W20260709 13:02:25.838032  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 143947us
W20260709 13:02:25.852296  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46724.7us
I20260709 13:02:25.855950  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:25.856287  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.856794  2294 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:25.839042  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2522) took 1589 ms. Trace:
I20260709 13:02:25.790995  5591 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:25.802273  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:25.852171  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 802) took 1704 ms. Trace:
I20260709 13:02:25.852316  1559 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:25.852916  1557 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:02:25.864705  1557 pending_rounds.cc:85] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 7
I20260709 13:02:25.865000  1557 pending_rounds.cc:107] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:02:25.865267  1557 raft_consensus.cc:2889] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.779070  5753 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:02:25.857509  1626 rpcz_store.cc:276] 0709 13:02:24.249514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.249964 (+   450us) service_pool.cc:224] Handling call
0709 13:02:24.250418 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.250799 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.250829 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.251292 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.921481 (+670189us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:25.598002 (+676521us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.838683 (+240681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.838770 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.838823 (+    53us) raft_consensus.cc:1685] finished
0709 13:02:25.838835 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.838976 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":655089,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":196552,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":43436,"spinlock_wait_cycles":721280,"thread_start_us":1542613,"threads_started":3,"wal-append.queue_time_us":776953}
I20260709 13:02:25.790692  5253 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.809926  5673 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.841935  5824 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:25.859743  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:25.842962  5771 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:25.871070  2294 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:25.868455  2296 rpcz_store.cc:276] 0709 13:02:24.148180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.937358 (+789178us) service_pool.cc:224] Handling call
0709 13:02:24.937729 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.938008 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:25.476898 (+538890us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.849449 (+372551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.849532 (+    83us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:25.849603 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.851987 (+  2384us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.852109 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272496,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":910567,"threads_started":2}
W20260709 13:02:25.863808  5837 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:02:25.885365  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.888578  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:25.869975  5786 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.000s
I20260709 13:02:25.863369  5815 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:25.901450  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2526) took 1575 ms. Trace:
I20260709 13:02:25.902647  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 650) took 1772 ms. Trace:
W20260709 13:02:25.904265  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.901939  1630 rpcz_store.cc:276] 0709 13:02:24.326047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.798218 (+472171us) service_pool.cc:224] Handling call
0709 13:02:24.798573 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.798926 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.798956 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.799461 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.656909 (+857448us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:25.656966 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.657060 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.901223 (+244163us) raft_consensus.cc:1685] finished
0709 13:02:25.901244 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.901388 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1083,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":857094,"threads_started":1,"wal-append.queue_time_us":1043123}
I20260709 13:02:25.903110  1701 rpcz_store.cc:276] 0709 13:02:24.130158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.130277 (+   119us) service_pool.cc:224] Handling call
0709 13:02:24.130604 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.130953 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:24.131276 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.765651 (+1634375us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.771819 (+  6168us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.780247 (+  8428us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.780425 (+   178us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:25.780467 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.780544 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.902395 (+121851us) raft_consensus.cc:1685] finished
0709 13:02:25.902425 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.902581 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12080,"prepare.queue_time_us":742,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":143749,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":14173,"thread_start_us":1627195,"threads_started":3,"wal-append.queue_time_us":769459}
I20260709 13:02:25.908533  2294 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:02:25.890616  5840 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:02:25.883728  5765 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:25.893040  5644 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:02:25.944664  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:25.867228  5738 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.949357  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:25.885039  5868 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:25.901067  5831 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:25.956053  5854 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:25.928608  5465 raft_consensus.cc:2764] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:25.884613  5673 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.957609  5253 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:25.970460  5773 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.979351  5422 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.990545  5752 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:25.937983  5866 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.940809  5644 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:25.992153  5885 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:25.908346  5815 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:25.934122  5786 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:26.002601  5883 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.999563  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:26.001554  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114411us
W20260709 13:02:25.960799  5789 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.000s
I20260709 13:02:26.030213  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:26.010149  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 207972us
I20260709 13:02:25.997490  5831 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:26.019572  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 131080us
I20260709 13:02:26.036211  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:26.010080  5843 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:02:25.893720  5756 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:02:26.028112  1557 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:26.028949  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:26.018630  5752 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.014735  5422 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.017594  5465 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:26.046162  5465 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.046599  5465 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.047894  5753 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:26.024255  5866 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:26.055070  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.056344  1626 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:26.056346  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1755) took 1760 ms. Trace:
W20260709 13:02:26.030586  5849 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:02:26.036932  2294 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.069841   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:26.006824  5863 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:02:26.007303  5230 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.085059  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1744) took 1800 ms. Trace:
I20260709 13:02:26.085530  1555 rpcz_store.cc:276] 0709 13:02:24.284792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.284914 (+   122us) service_pool.cc:224] Handling call
0709 13:02:24.321350 (+ 36436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.321633 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:25.102193 (+780560us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.084695 (+982502us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.084749 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.084802 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.084867 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.084987 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":429649,"prepare.queue_time_us":844583,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":1322136,"threads_started":2,"wal-append.queue_time_us":691717}
I20260709 13:02:26.044445  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.024557  5786 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.481849  2290 rpcz_store.cc:276] 0709 13:02:24.128753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.128896 (+   143us) service_pool.cc:224] Handling call
0709 13:02:24.129185 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.129428 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:24.870692 (+741264us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.478873 (+608181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.478948 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:25.479016 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.479104 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.479260 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1012056,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":11018,"raft.queue_time_us":110,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":18198,"thread_start_us":1348480,"threads_started":2}
I20260709 13:02:26.019312  5735 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.036342  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:26.065096  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.063531  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.011724  5868 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:26.081264  2294 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:26.093235  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.093488  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.094343  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.095364  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.095438  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.100474  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.065214  5253 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:26.110952  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.111682  2290 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:26.113482  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.020365  5858 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:02:26.105893  1632 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:26.116995  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:26.072930  5833 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.123656  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:26.119323  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 83185.8us
I20260709 13:02:26.124333  2290 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:26.120915  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.125959  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.126149  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.127440  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.128103  1555 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:26.126570  1632 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:26.091321  5786 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.128053  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.038889  5789 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.037906  5723 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.129536  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:26.107872  5851 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:02:26.028316  5885 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.088675  5835 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.134404  1626 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:26.134961  1552 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:26.146466  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:26.127573  5884 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:26.160070  5831 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:26.161453  5323 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:02:26.161929  5323 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.163926  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.164431  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.149117  5872 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:26.150748  2290 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:26.168377  2289 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:26.169080  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2422) took 1170 ms. Trace:
I20260709 13:02:26.155849  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.169535  1707 rpcz_store.cc:276] 0709 13:02:24.998498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.045513 (+ 47015us) service_pool.cc:224] Handling call
0709 13:02:25.045842 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.046619 (+   777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.046662 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.047156 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.596676 (+549520us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:25.596725 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.596790 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.168870 (+572080us) raft_consensus.cc:1685] finished
0709 13:02:26.168893 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.169014 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":549098,"threads_started":1,"wal-append.queue_time_us":686216}
I20260709 13:02:26.156234  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.164572  1555 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:26.141744  5595 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.165068  2295 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:26.034505  5883 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.128450  5253 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:26.154537  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.174444  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.165688  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:26.183024  1700 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:26.187908  2296 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:26.204377  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48222.7us
I20260709 13:02:26.205726  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.213792  1631 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:26.225274  1555 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:26.215265  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 665) took 1081 ms. Trace:
I20260709 13:02:26.234058  1705 rpcz_store.cc:276] 0709 13:02:25.134033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.195141 (+ 61108us) service_pool.cc:224] Handling call
0709 13:02:25.195592 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:25.260549 (+ 64957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.260596 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:25.266237 (+  5641us) log.cc:844] Serialized 52 byte log entry
0709 13:02:25.659863 (+393626us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:25.659914 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.659997 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.215050 (+555053us) raft_consensus.cc:1685] finished
0709 13:02:26.215069 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.215207 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35355,"raft.queue_time_us":48296,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6806,"thread_start_us":253824,"threads_started":1,"wal-append.queue_time_us":439375}
I20260709 13:02:26.239194  1552 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:26.240687  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.241564  1552 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:26.166659  5761 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.133728  5833 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:26.248464  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 243586us
I20260709 13:02:26.250819  1705 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:26.251873  1632 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:26.252885  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.253840  1552 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:26.218976  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:26.168411  5884 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:26.133244  5663 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:02:26.244133  1629 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:26.185346  5872 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:26.259032  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.178125  5591 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:02:26.256148  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:26.288836  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:26.291824  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.266010  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.267454  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.215271  5889 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:26.287349  1555 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
W20260709 13:02:26.296157  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45907.1us
W20260709 13:02:26.249208  5859 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.309s	user 0.001s	sys 0.001s
I20260709 13:02:26.184478  5595 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.243189  5890 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.303936  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.264569  5378 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.304490  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:26.305032  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86143.3us
I20260709 13:02:26.305182  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.305867  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.260906  5773 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.309000  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.309412  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21235338303a443e8f10b8773c68e2db"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:26.312976  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.314108  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.328541  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.311507  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.329577  1702 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:26.329385  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.264956  5832 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:02:26.057077  1558 rpcz_store.cc:276] 0709 13:02:24.296142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.013121 (+716979us) service_pool.cc:224] Handling call
0709 13:02:25.037149 (+ 24028us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:25.037569 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.037602 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:25.041335 (+  3733us) log.cc:844] Serialized 52 byte log entry
0709 13:02:25.536217 (+494882us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:25.558249 (+ 22032us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.593921 (+ 35672us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.986179 (+392258us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b180044e5a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:26.055948 (+ 69769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.056025 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.056143 (+   118us) raft_consensus.cc:1685] finished
0709 13:02:26.056158 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.056283 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":945306,"prepare.queue_time_us":28387,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":32007,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":6083,"spinlock_wait_cycles":23084160,"thread_start_us":494400,"threads_started":1,"wal-append.queue_time_us":530495}
I20260709 13:02:26.311967  1708 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:26.333055  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.333963  1707 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:26.353477  1702 pending_rounds.cc:85] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
W20260709 13:02:26.312428  5857 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:02:26.356387  1707 pending_rounds.cc:85] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 4
I20260709 13:02:26.362602  1707 pending_rounds.cc:107] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:26.362920  1707 raft_consensus.cc:2889] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.356453  1702 pending_rounds.cc:107] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:02:26.364639  1702 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.306663  5560 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:26.307816  5890 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.384256  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:26.378257  5881 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:02:26.335287  5673 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:26.378685  5875 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:02:26.380910  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:26.359054  5778 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:26.383697  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.380210  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1758) took 2050 ms. Trace:
I20260709 13:02:26.356285  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:26.386926  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 221353us
I20260709 13:02:26.406560  1558 raft_consensus.cc:2376] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9. Re-sending same reply.
I20260709 13:02:26.332044  5710 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.310992  5378 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:26.386795  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:26.393182  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.299718  5830 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 13:02:26.396703  5433 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.003s
I20260709 13:02:26.387836  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2532) took 1260 ms. Trace:
W20260709 13:02:26.378696  5871 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:02:26.312093  5414 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.181097  5835 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.393088  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:26.354194  5738 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.401157  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:26.406481  1556 rpcz_store.cc:276] 0709 13:02:24.330241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.025659 (+695418us) service_pool.cc:224] Handling call
0709 13:02:25.067434 (+ 41775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.067782 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.067818 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.068250 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.690995 (+622745us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:25.726165 (+ 35170us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.105338 (+379173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.105416 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.365276 (+259860us) raft_consensus.cc:1685] finished
0709 13:02:26.365299 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.380099 (+ 14800us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":378891,"prepare.queue_time_us":38971,"prepare.run_wall_time_us":36,"raft.queue_time_us":398114,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":26581,"spinlock_wait_cycles":3127680,"thread_start_us":622355,"threads_started":1,"wal-append.queue_time_us":682626}
I20260709 13:02:26.407469  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:26.331073  5761 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:02:26.435154  1635 rpcz_store.cc:276] 0709 13:02:25.127410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.127536 (+   126us) service_pool.cc:224] Handling call
0709 13:02:25.163310 (+ 35774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.163984 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.164032 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.164660 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.659407 (+494747us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:25.659464 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.659556 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.387557 (+728001us) raft_consensus.cc:1685] finished
0709 13:02:26.387582 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.387744 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160,"raft.queue_time_us":1217,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":16759,"thread_start_us":427784,"threads_started":1,"wal-append.queue_time_us":692971}
W20260709 13:02:26.409337  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97886us
I20260709 13:02:26.381978  5735 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.392627  5773 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.386725  5230 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.387691  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:26.457118  1557 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:26.392797  5785 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.386401  5890 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.387269  5889 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.383626  5644 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:26.435801  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74274.9us
I20260709 13:02:26.437469  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.312470  5253 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:26.439564  1551 raft_consensus.cc:2410] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:26.444379  1709 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:26.396629  5560 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.452382  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.434477  5378 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:26.460035  5559 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.020s	user 0.013s	sys 0.000s
I20260709 13:02:26.260344  5868 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:26.575553  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97486.5us
I20260709 13:02:26.451321  5789 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:26.458843  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 51476.8us
I20260709 13:02:26.461901  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.467538  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.447942  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.601795  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.549716  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.433992  5898 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:26.462944  5839 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:26.508597  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115589us
I20260709 13:02:26.607111  1558 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:26.461901  5433 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:02:26.509974  5644 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:26.379685  5884 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.579124  5785 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.592658  1635 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:26.590960  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:26.601501  1556 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:26.448702  5710 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.576084  5779 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:26.512672  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:26.602859  1552 raft_consensus.cc:2376] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9. Re-sending same reply.
I20260709 13:02:26.442239  5414 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.608034  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:26.608840  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:26.462049  5773 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:26.591198  5892 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:02:26.608526  5898 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:26.592190  5428 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.019s	user 0.011s	sys 0.001s
W20260709 13:02:26.601670  5880 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.316s	user 0.001s	sys 0.001s
I20260709 13:02:26.614394  5833 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.628464  1709 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.405563  5872 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:26.634661  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43845.2us
W20260709 13:02:26.614768  5821 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:02:26.619396  5853 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:02:26.595379  5883 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.554951  5663 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:02:26.551681  5673 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:02:26.646203  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.735224  1556 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:26.523365  5559 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.577793  5858 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.661384  1701 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:02:26.666903  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 199547us
I20260709 13:02:26.508255  5889 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.636969  5644 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:26.725150  1555 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:26.726203  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:26.677099  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.727623  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2349) took 2245 ms. Trace:
I20260709 13:02:26.746729  1554 rpcz_store.cc:276] 0709 13:02:24.481799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.064046 (+582247us) service_pool.cc:224] Handling call
0709 13:02:25.064426 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.064770 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:25.586724 (+521954us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.118881 (+532157us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.119355 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.119506 (+   151us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.405155 (+285649us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.462598 (+ 57443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.462670 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.727271 (+264601us) raft_consensus.cc:1685] finished
0709 13:02:26.727306 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.727524 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":291230,"prepare.queue_time_us":332715,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":20908,"raft.queue_time_us":448896,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":40500,"thread_start_us":1100486,"threads_started":4,"wal-append.queue_time_us":447831}
I20260709 13:02:26.646261  5898 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.748872  1554 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:26.750281  1554 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:26.751400  1554 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:26.735196  5824 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:02:26.620627  5378 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.761703  1633 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:26.635924  5465 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 13:02:26.662874  5595 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.661966  5428 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:26.677543  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.742605  5640 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:26.737872  5897 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 13:02:26.728482  1709 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:26.756470  1556 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:26.615758  5839 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.705721  5885 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:26.633823  5785 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.674991  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:26.768841  5663 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:26.766078  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:26.840878  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74862.5us
I20260709 13:02:26.774187  5644 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:26.773393  1598 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:26.773459  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47343.3us
I20260709 13:02:26.830220  2296 raft_consensus.cc:2410] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:26.646108  5779 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.745340  5858 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.846010  1558 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:26.768961  5893 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:02:26.768138  5789 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:26.837355  1552 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:02:26.765333  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 113196us
W20260709 13:02:26.829551  5853 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:02:26.829891  5821 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:26.841725  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:26.753814  1554 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:02:26.769659  5776 ts_tablet_manager.cc:1929] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
W20260709 13:02:26.848223  1598 raft_consensus.cc:2686] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:02:26.852900  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 79651us
W20260709 13:02:26.853950  5904 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:02:26.856129  5835 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:26.857162  5891 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 13:02:26.857231  5378 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:26.860127  5899 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:02:26.861275  5776 log.cc:1199] T c4b89801fe624f179701cbd4b98f1490 P 8b2c3934407d422495728f84374c6b53: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-1-root/wals/c4b89801fe624f179701cbd4b98f1490
I20260709 13:02:26.861992  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 649) took 2737 ms. Trace:
I20260709 13:02:26.862545  1704 rpcz_store.cc:276] 0709 13:02:24.124327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.124478 (+   151us) service_pool.cc:224] Handling call
0709 13:02:24.124821 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.125284 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:24.125609 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.740071 (+1614462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.473052 (+732981us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b18001ea840. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:02:26.473325 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.473573 (+   248us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.474155 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.573567 (+ 99412us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.573647 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.573952 (+   305us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.861724 (+287772us) raft_consensus.cc:1685] finished
0709 13:02:26.861745 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.861913 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10723,"prepare.queue_time_us":627405,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":6276,"raft.queue_time_us":47910,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":42838,"spinlock_wait_cycles":100478336,"thread_start_us":1613592,"threads_started":3,"wal-append.queue_time_us":797965}
I20260709 13:02:26.855443  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:26.862041  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1757) took 2557 ms. Trace:
I20260709 13:02:26.770588  5465 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.874060  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:26.840113  5663 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:26.843567  5673 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:26.878620  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:26.879683  2294 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:26.882367  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:26.829272  5640 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.868350  1553 rpcz_store.cc:276] 0709 13:02:24.304588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.021006 (+716418us) service_pool.cc:224] Handling call
0709 13:02:25.021318 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.069530 (+ 48212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.069573 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.070099 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.560921 (+490822us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:25.569117 (+  8196us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.569370 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.982609 (+413239us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b54003faa54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:25.982652 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.053085 (+ 70433us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b18000e5cc0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70d28b858f 00007f70cb8f7571 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd
0709 13:02:26.243888 (+190803us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b18000e5cc0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:26.861768 (+617880us) raft_consensus.cc:1685] finished
0709 13:02:26.861789 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.861958 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6283,"prepare.queue_time_us":15845,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":5642,"raft.queue_time_us":9266,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":9058,"spinlock_wait_cycles":140518784,"thread_start_us":463173,"threads_started":1,"wal-append.queue_time_us":1155950}
I20260709 13:02:26.893582  5877 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:02:26.894488  5889 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:26.896332  5422 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.000s
W20260709 13:02:26.897071  5752 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.000s
I20260709 13:02:26.896775  5422 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.897512  5752 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:26.898012  5909 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:26.898952  5422 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.883987  1635 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:26.900305  5378 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.900846  5378 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.901250  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.902207  5640 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:26.902894  5378 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.904172  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:26.905978  5865 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.596s	user 0.003s	sys 0.000s
I20260709 13:02:26.833312  5912 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:02:26.899526  5752 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.862071  5560 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.912099  1634 raft_consensus.cc:606] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:26.915383  1634 raft_consensus.cc:614] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:26.877390  5731 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.297s	user 0.014s	sys 0.000s
I20260709 13:02:26.911398  5912 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:26.875006  5428 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:26.906065  5710 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.907676  5905 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:26.916486  5731 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.926589  1703 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:26.861846  5789 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.859174  1446 maintenance_manager.cc:643] P 8c0fa710867745f189aafd0e7d84fa64: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.566s	user 0.214s	sys 0.102s Metrics: {"bytes_written":30131,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":876278,"dirs.run_cpu_time_us":490,"dirs.run_wall_time_us":26804,"drs_written":1,"lbm_read_time_us":15640,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":329225,"lbm_writes_1-10_ms":3,"lbm_writes_10-100_ms":13,"lbm_writes_lt_1ms":18,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":1162447,"threads_started":3,"update_count":10,"wal-append.queue_time_us":178195}
I20260709 13:02:26.918744  5912 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:26.923405  5773 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.956698  5465 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:26.958662  5831 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.002s
I20260709 13:02:26.959323  5831 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:26.947994  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 74059us
I20260709 13:02:26.948788  5559 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.930475  5663 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.952486  5789 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:26.953433  5912 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:26.978794  5912 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.978982  5839 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.953840  5890 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:26.981513  5890 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:26.983620  5785 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:26.977919  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 461060us
I20260709 13:02:26.989105  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:26.933374  5776 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56080 (request call id 0) took 4318 ms. Trace:
I20260709 13:02:26.955199  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:26.970767  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:26.947968  1447 maintenance_manager.cc:419] P 8c0fa710867745f189aafd0e7d84fa64: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9527 bytes on disk
I20260709 13:02:26.973829  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:26.954043  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:26.995651  2293 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.991120  2288 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:26.992631  2292 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.992672  1627 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:27.012027  5868 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:27.013762  5414 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.015513  5789 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:27.018625  1628 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:27.021018  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.023276  5731 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:27.023921  5731 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:27.025535  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51815.1us
I20260709 13:02:27.021240  5761 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:27.027637  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.027747  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.028386  2290 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:27.029819  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:27.030524  1632 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:27.037533  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:27.029942  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.042074  1446 maintenance_manager.cc:643] P 8c0fa710867745f189aafd0e7d84fa64: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.047s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":460,"lbm_reads_lt_1ms":4}
I20260709 13:02:27.049929  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.054006  1634 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:27.056843  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.058359  5884 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:27.060058  5858 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:26.989785  5803 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:27.034513  5559 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.064448  5663 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.047787  5765 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:02:26.992128  5776 rpcz_store.cc:276] 0709 13:02:22.637950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.691887 (+ 53937us) service_pool.cc:224] Handling call
0709 13:02:23.439093 (+747206us) ts_tablet_manager.cc:1151] Deleting tablet c4b89801fe624f179701cbd4b98f1490
0709 13:02:23.453361 (+ 14268us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:02:25.588846 (+2135485us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:26.760356 (+1171510us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:26.927793 (+167437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":614213,"tablet-delete.queue_time_us":672239,"thread_start_us":466136,"threads_started":1}
I20260709 13:02:27.070819  2292 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:27.076978  1632 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:27.078761  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.079710  2293 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:27.080036  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.034082  5761 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:27.059660  5786 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.905s	user 0.004s	sys 0.005s
I20260709 13:02:27.083235  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.092356  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.052006  5773 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:27.070537  5789 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:27.100176  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.101758  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.103097  1632 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:27.076164  5230 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:27.070796  5868 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:27.106302  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.107120  1632 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:27.108393  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:27.109122  1632 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:27.109933  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.096388  5790 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:27.101303  5877 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:27.101737  5789 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.133324  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.105692  5872 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:27.098850  5786 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.135471  5789 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:27.133868  5856 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:27.122120  5230 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:27.140687  5872 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:27.136909  5790 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:27.160408  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.132838  5761 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.165548  5761 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:27.166983  5872 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.171980  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet c4b89801fe624f179701cbd4b98f1490 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:02:27.172686  1444 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 102994us
I20260709 13:02:27.173833  5898 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.161779  5856 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:27.185554  5230 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.207505  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.209748  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.212220  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.213003  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:27.214237  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:27.216171  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.216953  1708 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:27.225055  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.225894  1705 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:27.231276  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:27.249979  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.254966  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.236299  5882 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.237890  5905 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.238940  5378 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:27.279044  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.277733  5883 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:27.237707  5673 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:02:27.286183  5378 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:27.233112  5790 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.292404  5559 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:27.303689  5786 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.308490  5890 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.307576  5883 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.315804  5378 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:27.317252  5924 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:02:27.316879  5559 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.403290  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 115808us
I20260709 13:02:27.404537  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 643) took 1320 ms. Trace:
W20260709 13:02:27.414455  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 646) took 1294 ms (client timeout 1630 ms). Trace:
W20260709 13:02:27.432873  1675 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.649s (SENT)
I20260709 13:02:27.341003  5768 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:27.342833  5779 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.524552  5835 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.524741  5768 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:27.525641  5835 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.526973  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1770) took 1466 ms. Trace:
I20260709 13:02:27.345678  5815 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.334060  5866 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:27.322182  5883 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.324573  5890 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.553201  1708 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.5-13.5]
I20260709 13:02:27.343711  5898 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.558142  1704 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:27.558653  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:27.553283  5771 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.559712  1702 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:27.560511  5771 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:27.565336  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 1.11752e+06us
I20260709 13:02:27.565938  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:27.336187  5927 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:27.476670  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 617) took 1875 ms. Trace:
I20260709 13:02:27.570896  1551 rpcz_store.cc:276] 0709 13:02:25.600552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.615987 (+ 15435us) service_pool.cc:224] Handling call
0709 13:02:27.475946 (+1859959us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54002f9754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.476293 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872274816}
I20260709 13:02:27.484344  2288 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:27.487327  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 668) took 2108 ms. Trace:
I20260709 13:02:27.489555  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2437) took 2021 ms. Trace:
I20260709 13:02:27.497414  2292 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:27.513893  5422 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:27.517176  5577 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.516531  5433 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:27.517846  5595 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.518126  5710 raft_consensus.cc:2764] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:27.516611  5560 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:27.518697  5860 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.524683  5559 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:27.283167  5856 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.590726  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.527350  5768 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.527539  1557 rpcz_store.cc:276] 0709 13:02:26.060357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.100155 (+ 39798us) service_pool.cc:224] Handling call
0709 13:02:27.526500 (+1426345us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54000771d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.526719 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818174720}
I20260709 13:02:27.592648  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.594237  5595 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.596808  5577 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.598630  5560 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.547309  5815 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:27.523733  1633 rpcz_store.cc:269] 0709 13:02:26.120341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.120600 (+   259us) service_pool.cc:224] Handling call
0709 13:02:27.413809 (+1293209us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.414160 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463093632}
I20260709 13:02:27.574332  5833 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.575098  5882 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.606668  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.575426  5912 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:27.573113  1700 rpcz_store.cc:276] 0709 13:02:25.378364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.382298 (+  3934us) service_pool.cc:224] Handling call
0709 13:02:25.382608 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.215676 (+833068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.486671 (+1270995us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b540010d1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:27.487079 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820875520}
I20260709 13:02:27.576921  5422 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.586488  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:27.586763  5253 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:27.602839  5723 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.573185  1703 rpcz_store.cc:276] 0709 13:02:25.468174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.468440 (+   266us) service_pool.cc:224] Handling call
0709 13:02:27.488965 (+2020525us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540016ba54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.489272 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3349620096}
I20260709 13:02:27.599103  5860 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.578976  5771 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.624540  1701 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:27.626619  5866 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.619174  5422 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:27.523319  1630 rpcz_store.cc:276] 0709 13:02:26.084399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.085928 (+  1529us) service_pool.cc:224] Handling call
0709 13:02:26.086281 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.109254 (+ 22973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.109283 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.109721 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.972219 (+862498us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b18002defe0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:27.100678 (+128459us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:27.100731 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.100823 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.403404 (+302581us) raft_consensus.cc:1685] finished
0709 13:02:27.403459 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.404255 (+   796us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22583,"raft.queue_time_us":23012,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":24100,"spinlock_wait_cycles":12442496,"thread_start_us":128241,"threads_started":1,"wal-append.queue_time_us":1027647}
I20260709 13:02:27.646827  5898 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.647935  5898 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.648476  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:27.650803  5898 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.655236  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.655587  5882 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:27.658275  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.486s (SENT) [suppressed 25 similar messages]
I20260709 13:02:27.659940  5831 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.667639  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.627434  5866 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.668494  1705 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:27.669378  5882 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:27.664201  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 231703us
W20260709 13:02:27.665293  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64615.9us
I20260709 13:02:27.654644  5771 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.670140  5866 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.641247  5253 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.659992  5735 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.684434  5710 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.707312  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 140867us
I20260709 13:02:27.700067  5866 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.679580  5890 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.710235  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.717134  5951 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.717809  5951 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:27.719587  5577 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.720248  5835 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.721787  5771 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.722381  5253 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.723937  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 616) took 2112 ms. Trace:
I20260709 13:02:27.724407  5771 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.725430  5945 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.671950  5912 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.736272  5428 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.730091  5866 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.736445  5710 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.724351  1560 rpcz_store.cc:276] 0709 13:02:25.597504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.598540 (+  1036us) service_pool.cc:224] Handling call
0709 13:02:25.598897 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.393060 (+794163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.393095 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.393587 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.603925 (+210338us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.616386 (+ 12461us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.695374 (+1078988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.704086 (+  8712us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.708153 (+  4067us) raft_consensus.cc:1685] finished
0709 13:02:27.708181 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.709363 (+  1182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1342242,"prepare.queue_time_us":9560,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":39,"raft.queue_time_us":1418055,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":24371,"thread_start_us":640302,"threads_started":3,"wal-append.queue_time_us":327080}
I20260709 13:02:27.708820  5858 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:27.740242  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 631) took 1724 ms. Trace:
W20260709 13:02:27.743929  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.736917  5887 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.760211  5887 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.738467  1630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:27.761792  1630 raft_consensus.cc:606] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:27.762290  1630 raft_consensus.cc:614] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:27.744531  5253 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:02:27.764016  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.765336  5595 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.766395  5866 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.768618  5860 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.753175  5768 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:27.769263  5858 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.771904  1701 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:27.773391  5560 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:27.767366  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.1584e+06us
I20260709 13:02:27.775161  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:27.747406  5428 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.787966  5428 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.789284  5710 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.790073  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.791038  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:27.792122  5866 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:27.754377  5323 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.777169  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.793418  1701 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:27.779810  5591 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:27.794348  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2543) took 1117 ms. Trace:
I20260709 13:02:27.794644  5591 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:27.795138  5591 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:27.796171  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 57132.5us
I20260709 13:02:27.797442  2293 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:27.799398  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.796263  5591 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.800899  5591 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.804330  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.804713  2292 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:27.808275  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.809000  2289 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:27.791567  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:27.811350  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2366) took 1548 ms (client timeout 1649 ms). Trace:
W20260709 13:02:27.811937  1555 rpcz_store.cc:269] 0709 13:02:26.262608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.297753 (+ 35145us) service_pool.cc:224] Handling call
0709 13:02:27.810878 (+1513125us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.811132 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731076864}
I20260709 13:02:27.815317  5640 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:27.814236  5738 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:02:27.816462  5738 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:27.816844  5738 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:27.817140  5742 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:27.817586  5742 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:27.818100  5738 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.818823  5742 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.820482  5742 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:27.820705  5738 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:27.822360  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.741607  5951 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:27.824298  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.826884  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.828346  5640 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:27.828927  5640 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:27.743886  1559 rpcz_store.cc:276] 0709 13:02:25.913423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.914202 (+   779us) service_pool.cc:224] Handling call
0709 13:02:25.914617 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:26.286724 (+372107us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:26.300214 (+ 13490us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.511250 (+211036us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.511496 (+   246us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:26.571416 (+ 59920us) log.cc:844] Serialized 52 byte log entry
0709 13:02:26.571633 (+   217us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.571937 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.423608 (+851671us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b180048e440. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:27.571610 (+148002us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b180048e440. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:27.576791 (+  5181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.579938 (+  3147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.605882 (+ 25944us) raft_consensus.cc:1685] finished
0709 13:02:27.605901 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.637412 (+ 31511us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":315443,"prepare.queue_time_us":41965,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":114,"raft.queue_time_us":1350841,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":4428,"spinlock_wait_cycles":34401408,"thread_start_us":362561,"threads_started":4,"wal-append.queue_time_us":302593}
I20260709 13:02:27.792181  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.794749  1635 rpcz_store.cc:276] 0709 13:02:26.676585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.676763 (+   178us) service_pool.cc:224] Handling call
0709 13:02:27.793970 (+1117207us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540019c154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:27.794153 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315248512}
I20260709 13:02:27.781461  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:27.726155  5945 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:27.846235  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.790073  5710 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:27.848498  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:27.849787  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:27.850226  5912 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.851317  5912 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:27.851967  5912 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.771656  5858 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.854818  5945 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.852684  5912 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:27.859231  5761 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.007s
W20260709 13:02:27.845548  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64385.7us
W20260709 13:02:27.859702  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 110635us
I20260709 13:02:27.860093  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:27.864512  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 73606.5us
I20260709 13:02:27.865023  5710 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:27.763079  5887 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:27.878880  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:27.885063  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:27.851716  5883 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:27.888415  1700 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:27.893725  5323 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.913234  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.913458  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:27.890259  5883 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:27.918191  1635 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:27.919157  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:27.929653  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 189755us
W20260709 13:02:27.959848  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45298.8us
I20260709 13:02:27.956480  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:27.964752  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46432us
I20260709 13:02:27.973403  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.972690  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:27.979724  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.925889  5858 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:27.984477  5323 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:28.009703  1700 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:02:28.011948  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.012450  5893 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:02:28.012296  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.016103  5771 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:28.016795  5771 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:28.018018  5771 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.019258  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.019433  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.022557  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.027392  1701 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:28.029531  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.031239  1702 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:02:28.043867  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.033042  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1779) took 1959 ms. Trace:
I20260709 13:02:28.045552  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.036509  1706 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:28.051453  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.050877  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:28.054550  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:27.995574  5883 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:02:28.028522  5422 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.062804  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1778) took 1995 ms. Trace:
I20260709 13:02:28.056067  5773 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:28.064289  5773 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:28.078054  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.055883  5323 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:02:28.108206  5323 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:28.108671  5323 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:28.108850  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.109817  5323 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.111337  5710 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.111917  5710 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.112069  5422 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:28.113058  5789 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.952s	user 0.019s	sys 0.000s
I20260709 13:02:28.113184  5673 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.113586  5789 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.044544  1552 rpcz_store.cc:276] 0709 13:02:26.073433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.849423 (+775990us) service_pool.cc:224] Handling call
0709 13:02:26.849734 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.568418 (+718684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.576749 (+  8331us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.657663 (+ 80914us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.734528 (+ 76865us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:27.734558 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.740603 (+  6045us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.032636 (+292033us) raft_consensus.cc:1685] finished
0709 13:02:28.032660 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.032846 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":583992,"raft.queue_time_us":678033,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":194,"thread_start_us":141338,"threads_started":2,"wal-append.queue_time_us":237444}
I20260709 13:02:28.003980  5230 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.178s
I20260709 13:02:28.121495  1709 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:28.111972  1627 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:28.125356  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.126184  1704 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.128013  5710 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:28.129757  5378 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.004s
I20260709 13:02:28.130259  5378 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.133332  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:28.134047  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 215807us
I20260709 13:02:28.134416  2293 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:28.141646  5815 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
I20260709 13:02:28.132615  1706 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:28.068236  5773 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:28.157058  1675 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.656s (SENT)
I20260709 13:02:28.164306  5422 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.164891  5422 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.165252  5422 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.165664  5414 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:28.166102  5414 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:28.167124  5414 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.168319  5414 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.169296  5866 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:28.170254  5866 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:28.170630  5866 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:28.173581  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.174414  1702 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:28.182188  5965 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.151s	user 0.002s	sys 0.002s
I20260709 13:02:28.184561  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:28.185245  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.186503  5673 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:28.187100  5673 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:28.188656  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 131610us
I20260709 13:02:28.189486  5673 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.190325  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:28.190672  5963 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:02:28.191465  5951 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:28.192059  2293 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:02:28.187450  5803 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.193511  5803 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.195948  5858 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:28.198195  5918 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:28.198731  5918 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:28.199934  5918 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.201184  1707 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:28.201916  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.201190  5378 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.203727  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:28.203976  1706 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:02:28.204890  5968 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:02:28.204855  5868 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:28.206187  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58085.1us
W20260709 13:02:28.206933  5964 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:02:28.207126  1675 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.737s (SENT)
I20260709 13:02:28.207716  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:28.208146  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.208738  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.209527  1702 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:28.206751  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.210865  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.211524  1700 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:28.207515  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.214022  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:28.216257  5559 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.004s
I20260709 13:02:28.216717  5559 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.216276  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:28.222723  5803 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.222966  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.223585  1701 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:28.225314  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.230598  1457 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 7, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.233368  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:28.235211  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.236469  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.237874  5951 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.238266  5951 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.238642  5951 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.239709  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:28.241614  5735 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.242034  5735 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.242354  5735 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.171588  5866 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.245878  5866 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.245468  5773 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.248593  5789 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.250874  5595 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:28.252020  5833 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:28.252388  5833 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:28.253093  5768 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:28.253654  5768 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.253417  5833 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.201337  5918 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:28.258270  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.260100  5833 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:28.261508  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.261868  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:28.263482  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:28.263664  5323 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.260121  5768 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.266053  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:28.268483  5834 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.268883  5834 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.269212  5834 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:28.270709  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.270453  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.271561  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:28.271699  5560 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:28.272284  5560 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.272744  5560 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:28.229962  5559 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.276753  5834 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.278913  5834 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.282745  5773 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.285590  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.286175  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.287304  5595 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:28.287873  5595 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.288331  5595 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.288816  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.290957  1701 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:28.063284  1556 rpcz_store.cc:276] 0709 13:02:26.067130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.842392 (+775262us) service_pool.cc:224] Handling call
0709 13:02:26.870623 (+ 28231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.680570 (+809947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.686466 (+  5896us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.723219 (+ 36753us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.742745 (+ 19526us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:27.888148 (+145403us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.888396 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.888507 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.062473 (+173966us) raft_consensus.cc:1685] finished
0709 13:02:28.062493 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.062628 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":572719,"prepare.queue_time_us":145072,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":9211,"raft.queue_time_us":659561,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":141,"spinlock_wait_cycles":9972480,"thread_start_us":132311,"threads_started":2,"wal-append.queue_time_us":248659}
I20260709 13:02:28.289955  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.292217  1706 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.292865  1703 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:02:28.293311  1703 pending_rounds.cc:85] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 9
I20260709 13:02:28.293627  1703 pending_rounds.cc:107] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:28.293973  1703 raft_consensus.cc:2889] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.299361  1706 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:28.304159  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.306293  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.306907  2288 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.307560  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.308435  5945 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.308802  5945 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.309209  5945 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.313220  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:28.314673  2288 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:28.314824  5877 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.330902  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.332659  5890 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.333252  5890 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.333600  1457 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 11, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.334708  5422 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.335470  5890 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:28.141726  5640 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.315872  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:28.320667  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52542.6us
I20260709 13:02:28.317351  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.327978  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:28.328213  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.326882  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:28.329073  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 95809.7us
I20260709 13:02:28.335327  1457 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:28.351935  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.355388  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.315754  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.345283  1704 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:28.359808  1708 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: [6.5-8.6]
W20260709 13:02:28.359714  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44025.3us
I20260709 13:02:28.364158  2289 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:28.378608  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.365856  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.379294  1704 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.367736  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 9, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.376636  5832 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.386327  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.387982  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:28.363117  5877 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.390996  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:28.342597  5945 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.390834  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.400796  2296 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:28.386693  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:28.383179  5773 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.410151  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.392513  5877 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.368162  5595 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:28.420343  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 213236us
I20260709 13:02:28.427911  5951 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.428231  5945 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:28.429049  1675 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.885s (SENT)
I20260709 13:02:28.429538  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:28.429919  5951 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.410712  5773 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.429852  5815 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:28.430716  5773 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.432415  5595 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:28.434352  5414 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:28.434966  5414 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.386222  5912 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.439568  5866 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:28.421497  5560 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.441220  5414 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.423470  5877 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.444089  5422 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.411080  5559 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:28.441708  5912 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.441732  5560 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.451413  1457 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 10, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.453737  1457 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.454290  5735 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.456220  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.456873  2295 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:28.430495  5753 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:28.464994  1702 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:28.425648  2294 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:28.457844  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:28.469122  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.469452  1457 catalog_manager.cc:5697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.494122  1704 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:28.498298  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.467499  5945 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:28.504170  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:28.495595  5803 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:28.495877  5753 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:28.517936  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.522701  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.524134  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 636) took 2390 ms. Trace:
I20260709 13:02:28.524695  1554 rpcz_store.cc:276] 0709 13:02:26.085401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.852289 (+766888us) service_pool.cc:224] Handling call
0709 13:02:27.532313 (+680024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.533198 (+   885us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:27.573660 (+ 40462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.912145 (+338485us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.914508 (+  2363us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.914776 (+   268us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.976666 (+ 61890us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.976856 (+   190us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.976897 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.982618 (+  5721us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.181425 (+198807us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:28.193550 (+ 12125us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b1800139120. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:28.201958 (+  8408us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b1800139120. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:28.225030 (+ 23072us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b1800139120. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:28.378107 (+153077us) raft_consensus.cc:1685] finished
0709 13:02:28.378127 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.467705 (+ 89578us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54000771d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:28.475616 (+  7911us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3606,"prepare.queue_time_us":396964,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":17137,"raft.queue_time_us":4981,"raft.run_cpu_time_us":490,"raft.run_wall_time_us":6652,"spinlock_wait_cycles":1625717888,"thread_start_us":320345,"threads_started":2,"wal-append.queue_time_us":451306}
I20260709 13:02:28.532162  1635 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.523641  1556 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:28.514459  5428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.028s	sys 0.051s
I20260709 13:02:28.510545  5230 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:28.500614  5559 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:28.509951  5803 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:28.515993  5753 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.546274  5428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.028s	sys 0.052s
I20260709 13:02:28.553476  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.562310  1635 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:28.559275  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.547181  5735 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.529613  5945 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:28.554126  5465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.017s	sys 0.021s
I20260709 13:02:28.578594  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:28.556566  5858 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:28.561801  5753 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.581849  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:28.577651  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:28.583447  2263 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.718s (SENT)
I20260709 13:02:28.588976  1555 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14.
I20260709 13:02:28.591126  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.591161  1554 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.568969  5591 raft_consensus.cc:2764] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:28.550427  5877 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.600340  1630 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:28.603346  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.594877  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 8 to 11, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 11 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:28.608414  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.612583  1556 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:28.617491  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:28.615085  1457 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.624264  1557 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:02:28.624775  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50672.8us
W20260709 13:02:28.625576  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 209) took 21991 ms (client timeout 15000 ms). Trace:
I20260709 13:02:28.598124  5803 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.627920  5884 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:28.592927  5858 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:28.583132  5465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.018s	sys 0.022s
W20260709 13:02:28.630515  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.005s	sys 0.024s
I20260709 13:02:28.641564  1552 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:28.636075  5803 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.650242  1457 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 11, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:28.654462  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.660899  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:28.664855  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:28.661188  5428 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.691805  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.014s	sys 0.016s
W20260709 13:02:28.694664  1601 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.551s (SENT)
I20260709 13:02:28.705443  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.719518  1557 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:28.709586  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.006s	sys 0.024s
I20260709 13:02:28.717809  1555 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:28.724175  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64633.1us
I20260709 13:02:28.732764  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.728778  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:28.625993  1455 rpcz_store.cc:269] 0709 13:02:06.633996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.650196 (+ 16200us) service_pool.cc:224] Handling call
0709 13:02:28.625309 (+21975113us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:26.594738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:26.595285 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:26.595325 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:02:26.595342 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:26.596642 (+  1300us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:26.596708 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:26.596739 (+    31us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:02:26.597826 (+  1087us) tablet.cc:701] Row locks acquired
0709 13:02:26.597843 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:02:26.598011 (+   168us) write_op.cc:270] Start()
0709 13:02:26.598136 (+   125us) write_op.cc:276] Timestamp: P: 1783602146597961 usec, L: 0
0709 13:02:26.598156 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:02:26.598852 (+   696us) log.cc:844] Serialized 4813 byte log entry
0709 13:02:26.955429 (+356577us) op_driver.cc:464] REPLICATION: finished
0709 13:02:27.101757 (+146328us) write_op.cc:301] APPLY: starting
0709 13:02:27.104723 (+  2966us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:27.337932 (+233209us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:27.338063 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:27.526983 (+188920us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:27.527595 (+   612us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:02:27.527675 (+    80us) write_op.cc:312] APPLY: finished
0709 13:02:27.689192 (+161517us) log.cc:844] Serialized 113 byte log entry
0709 13:02:27.692392 (+  3200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:27.692785 (+   393us) write_op.cc:454] Released schema lock
0709 13:02:27.692890 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:27.746449 (+ 53559us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:27.805693 (+ 59244us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":466639,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":133940,"apply.run_cpu_time_us":28138,"apply.run_wall_time_us":615170,"cfile_cache_hit":46,"cfile_cache_hit_bytes":48116,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":2,"lbm_read_time_us":59902,"lbm_reads_1-10_ms":3,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":7,"num_ops":12,"prepare.queue_time_us":635922,"prepare.run_cpu_time_us":4501,"prepare.run_wall_time_us":18798,"replication_time_us":357089,"thread_start_us":516627,"threads_started":2}]]}
I20260709 13:02:28.749197  1557 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:28.751652  5640 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.756315  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.761142  5912 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.761945  5912 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:28.729563  5710 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.001s
I20260709 13:02:28.775274  1556 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:28.730587  5323 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.771065  5673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.048s	sys 0.072s
I20260709 13:02:28.792987  2295 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:28.722456  1457 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.737275  5945 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:28.800632  5673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.048s	sys 0.072s
I20260709 13:02:28.773862  5877 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.707302  5858 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.809959  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:28.770531  5428 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:28.798836  2293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.013s	sys 0.035s
I20260709 13:02:28.814833  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:28.796226  5834 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.820823  1557 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:28.795413  5771 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:28.821741  2263 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.925s (SENT)
I20260709 13:02:28.819432  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:28.820034  2293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.013s	sys 0.035s
I20260709 13:02:28.799413  5773 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.822371  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:28.822888  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.821163  5834 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.796640  5710 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:28.846578  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 151989us
I20260709 13:02:28.848399  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:28.846258  1556 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:28.858655  1630 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:28.859057  1633 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:28.860216  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.861203  1557 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:28.861570  5428 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.863410  1556 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:02:28.861313  2263 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.958s (SENT)
I20260709 13:02:28.864578  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.865101  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.865801  1557 raft_consensus.cc:2368] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11.
I20260709 13:02:28.865847  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.866621  1556 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:28.867074  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.867727  2295 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:28.868413  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.869148  2294 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:28.869148  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:28.870098  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.866676  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.828009  5866 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:28.877384  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.878108  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:28.877909  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 447184us
I20260709 13:02:28.812403  5323 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.885918  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:28.886749  2290 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:28.881170  1457 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 6 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 9 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:28.844079  5591 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:28.902261  1552 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:28.902964  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:28.900709  1524 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:28.846195  5831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.030s	sys 0.030s
I20260709 13:02:28.902048  1556 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:28.906167  5945 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.910367  5560 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:28.910753  5945 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:28.911856  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:28.910427  1457 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 8 to 10, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.914933  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.915009  1457 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 8 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:28.916144  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.916934  1553 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:28.917060  1457 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:28.917835  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:28.913033  5945 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.925026  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.913553  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:28.935230  5378 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.935593  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 495196us
I20260709 13:02:28.935858  5378 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.936370  2263 leader_election.cc:400] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14.
I20260709 13:02:28.936447  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:28.940801  5834 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:28.941079  5378 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.944962  1558 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:02:28.945998  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.958s (SENT)
I20260709 13:02:28.946401  1556 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:02:28.946739  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:28.948544  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.019s (SENT)
W20260709 13:02:28.949958  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.082s (SENT)
I20260709 13:02:28.950498  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:28.960495  5912 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:28.909533  5831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.031s	sys 0.030s
I20260709 13:02:28.964184  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 660) took 1910 ms. Trace:
I20260709 13:02:28.964709  1631 rpcz_store.cc:276] 0709 13:02:27.053937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.055276 (+  1339us) service_pool.cc:224] Handling call
0709 13:02:27.057042 (+  1766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.095411 (+ 38369us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:27.796592 (+701181us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800405dc0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:02:27.962426 (+165834us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.785144 (+822718us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.797371 (+ 12227us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.801460 (+  4089us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.839670 (+ 38210us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.839887 (+   217us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:28.839936 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.936888 (+ 96952us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b54002972d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:28.936945 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.963501 (+ 26556us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b1800405dc0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:28.963874 (+   373us) raft_consensus.cc:1685] finished
0709 13:02:28.963911 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.964110 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75710,"prepare.queue_time_us":389920,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":2743,"raft.queue_time_us":56512,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":14047,"spinlock_wait_cycles":55281664,"thread_start_us":866024,"threads_started":2,"wal-append.queue_time_us":767441}
I20260709 13:02:28.972020  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:28.973206  5890 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:28.973459  5323 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.974368  5890 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.975652  5710 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:28.976162  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:28.977005  5832 raft_consensus.cc:2764] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:02:28.977591  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.977727  5832 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:28.978130  5832 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:28.978479  5832 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:28.982323  5428 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:28.983665  5877 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:28.984922  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:28.985787  5890 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:28.986232  5890 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:28.986572  5773 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:28.988346  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:28.901254  5866 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:28.989807  5866 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:28.939584  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.001922  1560 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:28.933888  1457 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:29.004658  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65157.9us
I20260709 13:02:29.005781  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.009473  5378 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:29.010126  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.011616  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.012492  1704 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:29.013283  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.013911  1702 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:02:29.009088  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 97322us
I20260709 13:02:29.017550  1633 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:28.986610  5890 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.945043  5640 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:29.018859  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:29.007577  5753 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.019291  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.019985  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:29.030158  1701 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:29.010697  5868 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:29.021026  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.011703  2290 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:29.022449  5832 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:29.040337  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.014982  5591 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:29.018812  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.044287  1703 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:29.045493  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:29.048065  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.012s	sys 0.055s
W20260709 13:02:29.048614  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.012s	sys 0.055s
W20260709 13:02:29.048919  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 531060us
I20260709 13:02:29.024119  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.018368  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.026757  1708 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:29.016307  5378 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:29.031602  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.046331  2296 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:29.038671  5945 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:29.068564  1708 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:29.032562  1702 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:29.036460  1706 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:29.021618  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:29.016551  1631 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:29.039918  1457 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 6 to 9, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.028446  5230 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:29.053949  1630 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:29.056221  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 106302us
I20260709 13:02:29.057353  1554 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:29.057593  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.069109  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:29.068006  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:29.073388  1627 raft_consensus.cc:606] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Received request to transfer leadership to TS 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.073756  1707 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:29.073853  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.091s (SENT)
I20260709 13:02:29.074383  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.035894  5560 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:29.076961  5868 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:29.077180  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.198s (SENT)
I20260709 13:02:29.077801  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.042119  5831 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:02:29.080009  5868 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:29.080654  5868 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.081163  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:29.081064  5868 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.081740  1628 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:29.082176  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.083146  5868 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.083346  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:29.083623  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:29.084208  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.084295  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.084614  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.085017  1634 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:29.085630  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.075443  1457 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.090247  1457 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:29.093837  1457 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:29.096805  5753 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:29.096343  1457 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 3 to 4, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.098630  1457 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:29.091662  5979 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:29.105459  5979 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.105578  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.105919  5979 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:29.106185  1704 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:02:29.106750  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.092473  5868 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:29.092304  5378 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.087386  5230 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.092324  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.077235  5832 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.103031  1706 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:29.091063  1628 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:29.108855  1701 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:29.041363  5710 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.110148  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.123409  1707 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:29.068707  5974 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.133s	user 0.003s	sys 0.000s
W20260709 13:02:29.116570  1601 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.763s (SENT)
I20260709 13:02:29.124678  1457 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 6 to 8, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:29.128136  5560 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:29.130079  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.101351  5890 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.144344  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.117447  5803 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:29.148640  1704 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:29.150179  1457 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 8, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.151007  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:29.152269  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 135425us
I20260709 13:02:29.152506  1631 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:29.156623  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64378.6us
I20260709 13:02:29.156080  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.159219  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.160490  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 682) took 1418 ms. Trace:
I20260709 13:02:29.135249  5831 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:29.153192  5710 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.165117  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.149379  5735 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:29.167186  1628 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:29.168534  5323 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.169368  5323 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:29.171306  5803 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.178774  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.179527  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:29.162358  5887 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.179738  1633 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:29.180377  5887 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.182652  5887 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.183561  1632 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:29.185606  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.187809  5868 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:29.188202  5868 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.188539  5868 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:29.189854  5710 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:29.155912  5832 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:29.160908  1705 rpcz_store.cc:276] 0709 13:02:27.738801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.741769 (+  2968us) service_pool.cc:224] Handling call
0709 13:02:27.829435 (+ 87666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.829879 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:27.878773 (+ 48894us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.151815 (+1273042us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.153227 (+  1412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.153287 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:29.153345 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.153407 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.153518 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131814,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":9399,"raft.queue_time_us":920,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7070,"thread_start_us":1269638,"threads_started":3,"wal-append.queue_time_us":963371}
I20260709 13:02:29.210397  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:29.211119  2294 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:29.213726  5868 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.217585  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:29.182694  5735 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:29.208756  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:29.293187  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84529.3us
I20260709 13:02:29.206653  5832 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:29.254056  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 181070us
I20260709 13:02:29.233258  5803 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.235481  5979 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:29.267120  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 255291us
I20260709 13:02:29.273051  1457 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:29.282428  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64906.6us
I20260709 13:02:29.318596  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:29.325173  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.327576  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:29.316938  5832 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:29.316411  5773 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:29.365247  1703 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:29.316846  5877 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:29.368113  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:29.371165  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 346216us
W20260709 13:02:29.375478  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 56948.5us
I20260709 13:02:29.365347  5761 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:29.371958  5912 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:29.372157  5877 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.323508  5735 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.410887  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.372665  5815 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:29.420889  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:29.403576  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:29.379611  5945 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:29.426160  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:29.385807  5981 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:29.440699  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:29.443732  2290 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:29.444804  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.446524  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.447713  1554 raft_consensus.cc:2410] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:29.423836  5877 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:29.430267  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:29.453188  1706 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:29.435935  1632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:29.454495  1632 raft_consensus.cc:606] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:29.454977  1632 raft_consensus.cc:614] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:29.436753  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:29.455942  1559 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.418236  5710 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.416478  5560 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.429754  1701 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:02:29.456233  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 297119us
I20260709 13:02:29.458127  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:29.462975  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.381942  5773 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:29.433462  5988 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:29.428998  5912 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:29.469803  1560 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:29.433336  5890 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.468962  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.448623  5981 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.499536  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:29.503525  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.504418  1554 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:29.510437  5981 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:29.510653  5993 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:29.511444  1556 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:29.505383  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:29.517860  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 48951.3us
I20260709 13:02:29.518195  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:29.518627  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:29.519043  1556 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:29.478423  5858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.062s
I20260709 13:02:29.459003  5323 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:29.434474  5883 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:02:29.441072  5831 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:02:29.521811  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200638us
W20260709 13:02:29.522827  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.519431  5768 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.476795  5803 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.506734  5735 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.500497  5866 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:29.525813  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.542676  1632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:29.586249  1632 raft_consensus.cc:606] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
W20260709 13:02:29.542263  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 93515us
I20260709 13:02:29.562901  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:29.526821  5323 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.524523  5753 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:02:29.567898  5831 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:29.569299  5883 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:29.567256  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.570559  5912 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.576637  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:29.464098  5815 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.579517  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.525503  5378 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:29.541581  5890 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:29.610956  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 92398.9us
W20260709 13:02:29.524026  5858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.006s	sys 0.065s
I20260709 13:02:29.523413  5560 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.632912  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.631487  2296 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:29.635336  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:29.637686  1706 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:29.636164  1551 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:29.506176  5710 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.640560  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:29.640105  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69412.5us
I20260709 13:02:29.647500  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:29.648337  1551 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:29.600423  5323 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:29.639642  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:29.601001  5831 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:29.636549  5975 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:29.653831  1558 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:02:29.654899  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104818us
I20260709 13:02:29.572166  5803 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.676590  5868 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.679061  5868 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.660326  5831 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:29.681079  1551 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:29.636576  5378 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.604272  5995 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:29.574219  5768 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.658107  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:29.662873  5971 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:29.653235  5858 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:02:29.629391  5979 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:29.643069  5773 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.665305  5640 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:29.670112  5761 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:29.626014  5883 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:29.683597  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.660259  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:29.693794  1601 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.667s (SENT)
I20260709 13:02:29.581341  5945 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.687683  5378 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:29.742017  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 96314.9us
I20260709 13:02:29.687606  5771 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:29.743013  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 88850.7us
I20260709 13:02:29.679559  5868 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.719858  5640 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:29.778139  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84398.2us
W20260709 13:02:29.749426  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 89230.4us
I20260709 13:02:29.747433  1706 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:29.718721  5803 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:29.654374  5890 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.774797  5771 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.800652  5761 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:29.801057  5761 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:29.801872  5771 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.817163  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:29.817948  2294 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:29.869951  6014 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:29.896770  6019 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.942196  5323 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.924996  5815 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.945032  5323 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:29.947949  5710 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:29.801934  5761 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:29.950950  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 318138us
I20260709 13:02:29.870060  5883 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:29.904842  5995 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:29.964295  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 181530us
I20260709 13:02:29.965008  1709 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:29.965507  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87697.3us
I20260709 13:02:29.922405  5735 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:29.934868  6014 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:29.980625  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.974694  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:29.984824  1706 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:29.986400  6014 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:29.973233  5323 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:29.988291  5323 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:29.989814  6011 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:29.972898  5735 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.992506  5735 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:29.988864  6013 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:29.972433  5761 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:30.010017  5768 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:30.006980  5591 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:30.011404  5883 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:30.024739  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50153.3us
I20260709 13:02:30.025751  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:30.014207  5981 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.000s
I20260709 13:02:29.994812  6024 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:30.061506  6024 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:30.029984  5877 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
W20260709 13:02:30.018138  1601 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.864s (SENT)
I20260709 13:02:30.008320  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:30.059937  5868 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:30.001799  5560 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:30.106640  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101082us
W20260709 13:02:30.128621  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.108541  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.808s (SENT)
I20260709 13:02:30.140713  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:30.170215  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 113111us
I20260709 13:02:30.170867  5868 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.171458  6024 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:30.178134  5832 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.000s
I20260709 13:02:30.065888  5591 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:29.958763  6019 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.184798  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:30.188464  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 170336us
I20260709 13:02:30.001747  6016 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:02:30.200426  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.850s (SENT)
I20260709 13:02:30.205330  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:30.206020  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:30.209324  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.064378  5877 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.228128  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.230873  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:30.237864  1551 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:30.231112  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:30.109617  5981 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:30.238443  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.147s (SENT)
I20260709 13:02:30.209197  5832 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:30.241531  1601 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.691s (SENT)
I20260709 13:02:30.247723  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:30.242555  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:30.242952  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:30.239670  6027 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.251915  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:30.256930  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:30.259064  1700 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:30.259866  6030 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:30.260150  1554 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:30.259666  5868 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:30.260747  6001 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.509s	user 0.000s	sys 0.004s
W20260709 13:02:30.261945  6002 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.525s	user 0.001s	sys 0.002s
I20260709 13:02:30.256480  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.264866  1551 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:30.265878  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:30.267015  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.267611  1551 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:30.268546  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.269615  1551 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:30.270780  1551 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:30.227377  5378 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.239486  6020 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:30.240971  6023 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:30.252935  5831 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 13:02:30.239596  6028 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.276369  1554 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:30.239642  6022 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:30.294165  2295 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:30.301527  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:30.293226  6023 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:30.295166  5378 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:30.291906  5773 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:30.241204  5995 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:30.302179  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60687.2us
I20260709 13:02:30.302335  1557 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
W20260709 13:02:30.322425  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84003.5us
I20260709 13:02:30.324294  6019 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:30.260499  6030 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.327530  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:30.306318  6023 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:30.301120  5831 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:30.301546  6022 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:30.333684  5988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.004s	sys 0.024s
I20260709 13:02:30.332422  5773 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.299993  6028 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:30.339794  5735 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:30.346160  5988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.004s	sys 0.025s
I20260709 13:02:30.339380  5979 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:30.361588  5981 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:30.362994  5988 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:30.374619  6028 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:30.375965  6022 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:30.378161  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.007s	sys 0.024s
W20260709 13:02:30.378528  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.008s	sys 0.025s
I20260709 13:02:30.378012  6014 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.381449  6030 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:30.382861  6014 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:30.558780  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.719990  6025 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.720784  6025 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.723610  6025 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.481207  6025 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:31.482128  6025 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:31.373840  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08002981c0 after lost signal to thread 431
W20260709 13:02:31.485304  1581 debug-util.cc:398] Leaking SignalData structure 0x7b080033dea0 after lost signal to thread 434
I20260709 13:02:31.487257  6025 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:31.488309  6025 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:31.489066  6025 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:31.491822  6025 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.492656  6025 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:31.516774  6025 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:31.523504  6025 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.574895  6025 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.577368  6039 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:02:31.575634  6025 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:30.387301  6023 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:31.626176  6025 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.626454  6023 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.634972  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:31.635807  1558 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
W20260709 13:02:31.636387  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 656) took 1583 ms (client timeout 2083 ms). Trace:
W20260709 13:02:31.637609  1558 rpcz_store.cc:269] 0709 13:02:30.052732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.634635 (+1581903us) service_pool.cc:224] Handling call
0709 13:02:31.636309 (+  1674us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.685379  6025 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:31.686441  6025 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:31.690024  6025 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:31.805461  6025 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.806178  6025 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:31.808732  6025 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:31.829772  6025 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:31.830506  6025 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:31.833511  6025 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:32.055693  6025 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:32.056454  6025 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:32.059118  6025 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:32.433947  1559 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 2.965s	user 0.017s	sys 0.014s
I20260709 13:02:32.436857  1559 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:32.438339  6012 raft_consensus.cc:2764] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:02:32.442793  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1780) took 3022 ms (client timeout 1911 ms). Trace:
W20260709 13:02:32.445019  1559 rpcz_store.cc:269] 0709 13:02:29.420190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.420543 (+   353us) service_pool.cc:224] Handling call
0709 13:02:32.442622 (+3022079us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.898475  1556 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:32.925076  5773 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:32.927685  5773 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:32.929947  5992 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 3.265s	user 0.004s	sys 0.024s
I20260709 13:02:30.347577  5591 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:32.938364  1524 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.755s (SENT) [suppressed 24 similar messages]
W20260709 13:02:32.991645  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1056) took 2770 ms (client timeout 2755 ms). Trace:
W20260709 13:02:32.992183  2295 rpcz_store.cc:269] 0709 13:02:30.221591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.222170 (+   579us) service_pool.cc:224] Handling call
0709 13:02:32.991386 (+2769216us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:32.991544 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125673344}
W20260709 13:02:33.047006  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 2.71931e+06us
I20260709 13:02:33.049863  5559 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:33.050302  5559 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:33.051007  5559 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.051964  5559 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:33.058622  5988 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:33.067679  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:33.071998  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1787) took 3546 ms (client timeout 2660 ms). Trace:
W20260709 13:02:33.072485  1556 rpcz_store.cc:269] 0709 13:02:29.525416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.525567 (+   151us) service_pool.cc:224] Handling call
0709 13:02:33.071740 (+3546173us) spinlock_profiling.cc:243] Waited 3.31 s on lock 0x7b5400128ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:33.071919 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7617337216}
I20260709 13:02:33.075461  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 662) took 2893 ms. Trace:
I20260709 13:02:33.075847  1553 rpcz_store.cc:276] 0709 13:02:30.182224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.074625 (+2892401us) service_pool.cc:224] Handling call
0709 13:02:33.074991 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.075170 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.075196 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.075241 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:33.075281 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.075333 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.075416 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.078887  1557 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:02:33.080642  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2401) took 2876 ms. Trace:
I20260709 13:02:33.080906  1557 rpcz_store.cc:276] 0709 13:02:30.204578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.323791 (+119213us) service_pool.cc:224] Handling call
0709 13:02:30.324162 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.079277 (+2755115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.080457 (+  1180us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b54002ba7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:33.080599 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6334642432}
W20260709 13:02:33.091048  5997 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 3.450s	user 0.019s	sys 0.022s
W20260709 13:02:33.093856  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.445s	user 0.014s	sys 0.053s
W20260709 13:02:33.094399  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.449s	user 0.014s	sys 0.053s
I20260709 13:02:33.100049  6016 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.100710  6016 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.103006  6016 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.103185  5323 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.104830  5323 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.105412  5323 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.107673  5323 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:33.111416  5883 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 2.972s	user 0.009s	sys 0.027s
W20260709 13:02:33.104466  1523 connection.cc:256] RPC call timeout handler was delayed by -1.9427213640000218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:02:33.113709  5323 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:33.115736  5591 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.116349  6026 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.116334  5591 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.117106  6026 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.117933  5988 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.118448  5988 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.121112  5591 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.121903  5591 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.122371  5591 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:33.123870  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75382.4us
I20260709 13:02:33.125072  5591 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.125747  5591 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.126223  5591 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.126613  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.128016  6028 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:33.128796  5591 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.129361  5591 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.129544  6028 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:33.129937  6028 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.129889  5591 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.130422  6028 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:33.132077  5591 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.132578  5890 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.133999  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.135350  5979 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:33.135738  5979 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.136178  5979 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:33.137159  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.800s	user 0.011s	sys 0.011s
W20260709 13:02:33.137633  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.801s	user 0.011s	sys 0.011s
W20260709 13:02:33.138178  1601 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.503s (SENT)
I20260709 13:02:33.141655  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:33.142264  1601 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.788s (SENT)
I20260709 13:02:33.142939  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:33.143286  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.911s (SENT)
I20260709 13:02:33.143817  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:33.144603  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:33.144961  1601 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.486s (SENT)
I20260709 13:02:33.145228  1559 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:33.146358  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.146862  5884 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:33.147183  1601 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.535s (SENT)
I20260709 13:02:33.148348  5884 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:33.148898  5884 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.150877  5884 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.151067  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 853) took 2931 ms. Trace:
I20260709 13:02:33.135872  5868 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:33.153476  5868 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.154208  5868 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.143574  1553 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:33.146917  1558 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:33.142427  1557 raft_consensus.cc:2368] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11.
W20260709 13:02:33.158293  5994 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 3.573s	user 0.003s	sys 0.036s
I20260709 13:02:33.144119  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.160234  1556 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:33.164100  6005 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.164765  1558 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:02:33.166980  6007 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 3.372s	user 0.010s	sys 0.020s
I20260709 13:02:33.164773  6005 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.170334  6005 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.170894  6005 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.172564  1557 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:33.173659  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.174670  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.175855  1555 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:33.176342  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 659) took 3122 ms (client timeout 3345 ms). Trace:
W20260709 13:02:33.176659  1555 rpcz_store.cc:269] 0709 13:02:30.053630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.174348 (+3120718us) service_pool.cc:224] Handling call
0709 13:02:33.176283 (+  1935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.177542  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.178026  1555 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:33.181350  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2392) took 4362 ms (client timeout 2416 ms). Trace:
W20260709 13:02:33.181919  1635 rpcz_store.cc:269] 0709 13:02:28.818916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.819157 (+   241us) service_pool.cc:224] Handling call
0709 13:02:33.181053 (+4361896us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b540019c154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:33.181230 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582956928}
I20260709 13:02:33.171423  6005 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.183421  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.184248  1559 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:33.184638  6005 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.174824  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.185024  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.185866  1559 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.186800  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.188036  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.188766  1553 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:33.189556  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.190030  1553 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:33.192073  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.192863  1553 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.193717  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.194430  1553 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.147684  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:33.247274  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 100109us
I20260709 13:02:33.171346  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.249226  1558 raft_consensus.cc:2410] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:33.250258  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.252952  5832 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.253319  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.151453  2290 rpcz_store.cc:276] 0709 13:02:30.219748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.219940 (+   192us) service_pool.cc:224] Handling call
0709 13:02:30.220309 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.148737 (+2928428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.148766 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.148808 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:33.148857 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.150809 (+  1952us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b5400720954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:33.150852 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.150973 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6734232960}
I20260709 13:02:33.254467  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 660) took 3114 ms. Trace:
I20260709 13:02:33.141700  5890 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:02:33.217710  5771 raft_consensus.cc:993] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: : Instructing follower 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:02:33.255404  5771 raft_consensus.cc:1081] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Signalling peer 2579bae7bf1543bf9b10d68e93a6ebbb to start an election
I20260709 13:02:33.254942  1552 rpcz_store.cc:276] 0709 13:02:30.140286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.243588 (+3103302us) service_pool.cc:224] Handling call
0709 13:02:33.253901 (+ 10313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.254100 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.254127 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.254177 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.254217 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.254278 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.254391 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.221341  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1817) took 3587 ms. Trace:
W20260709 13:02:33.233959  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.257725  1525 connection.cc:256] RPC call timeout handler was delayed by -3.5706103480000593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 13:02:33.258057  1525 connection.cc:256] RPC call timeout handler was delayed by -3.5616103480000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 13:02:33.258345  1525 connection.cc:256] RPC call timeout handler was delayed by -3.5058103480000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:02:33.258651  1525 connection.cc:256] RPC call timeout handler was delayed by -3.4608103480000523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 13:02:33.258936  1525 connection.cc:256] RPC call timeout handler was delayed by -3.3393103480000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260709 13:02:33.259217  1525 connection.cc:256] RPC call timeout handler was delayed by -3.2538103480000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260709 13:02:33.259505  1525 connection.cc:256] RPC call timeout handler was delayed by -3.236710348000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:02:33.259914  5710 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.260468  5710 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.263577  5323 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.257155  1700 rpcz_store.cc:276] 0709 13:02:29.634467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.634641 (+   174us) service_pool.cc:224] Handling call
0709 13:02:29.634976 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.259591 (+624615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.221034 (+2961443us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:33.221262 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434786688}
I20260709 13:02:33.248132  5559 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.265003  6016 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.265571  6016 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.202279  5868 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.264204  5323 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.268580  5323 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.268747  5710 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.245641  5831 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:33.251114  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.912s	user 0.021s	sys 0.029s
W20260709 13:02:33.270094  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.932s	user 0.021s	sys 0.029s
W20260709 13:02:33.270428  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 2.93483e+06us
W20260709 13:02:33.270642  6000 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 3.598s	user 0.018s	sys 0.013s
W20260709 13:02:33.270990  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.272352  1525 connection.cc:256] RPC call timeout handler was delayed by -3.2313103480000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 13:02:33.272780  1525 connection.cc:256] RPC call timeout handler was delayed by -3.2043103480000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 13:02:33.273063  1525 connection.cc:256] RPC call timeout handler was delayed by -3.1485103480000589s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
W20260709 13:02:33.273376  1525 connection.cc:256] RPC call timeout handler was delayed by -3.0018103480000491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
I20260709 13:02:33.273628  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.273821  6016 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.274650  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.276119  1559 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:33.273669  1525 connection.cc:256] RPC call timeout handler was delayed by -2.9748103480000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260709 13:02:33.277066  5735 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:33.277357  1525 connection.cc:256] RPC call timeout handler was delayed by -2.8209103480000408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2416s for a response
W20260709 13:02:33.277730  1525 connection.cc:256] RPC call timeout handler was delayed by -2.4996103480000329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27730000000000005s for a response
W20260709 13:02:33.278017  1525 connection.cc:256] RPC call timeout handler was delayed by -2.3691103480000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2918s for a response
W20260709 13:02:33.279165  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.279858  5834 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.280038  5640 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.280406  5834 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.281299  5640 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:33.282588  5834 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.283131  5834 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.284278  5883 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.284850  5883 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.286649  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1810) took 4181 ms. Trace:
I20260709 13:02:33.287288  5883 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.283605  5834 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.289342  5378 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.290870  5378 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.293061  5378 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.287057  1702 rpcz_store.cc:276] 0709 13:02:29.104945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.119227 (+ 14282us) service_pool.cc:224] Handling call
0709 13:02:29.119910 (+   683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.123868 (+  3958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.123912 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.124459 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.367350 (+242891us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:29.367389 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.367455 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.090502 (+3723047us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:33.286324 (+195822us) raft_consensus.cc:1685] finished
0709 13:02:33.286349 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.286581 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3525,"raft.queue_time_us":59035,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"spinlock_wait_cycles":105384832,"thread_start_us":214038,"threads_started":1,"wal-append.queue_time_us":400252}
I20260709 13:02:33.295126  5834 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.297358  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:33.289399  2263 connection.cc:256] RPC call timeout handler was delayed by -2.5376685410000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260709 13:02:33.289965  5591 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.299814  5591 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:33.304188  2263 connection.cc:256] RPC call timeout handler was delayed by -2.3230910369999833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:02:33.304627  2263 connection.cc:256] RPC call timeout handler was delayed by -1.2565910370000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3345s for a response
W20260709 13:02:33.309916  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.311317  5832 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.311548  6004 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.312371  5831 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.312240  5832 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.312994  5831 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.312238  6004 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.323248  6004 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.324913  5832 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.326416  5591 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.327431  5735 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:33.328989  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.329877  5735 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:33.331172  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.332106   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:33.347747  6012 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:33.324968  5831 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.332010  6004 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.351504  6004 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.353469  1556 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
W20260709 13:02:33.340801  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.355050  5982 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.355820  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.355546  5982 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.356547  1630 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.357905  5982 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.358385  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.359030  1632 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:33.359282  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.360088  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.360364  5773 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:33.346356  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08001cdb40 after lost signal to thread 1492
I20260709 13:02:33.361099  5773 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:33.362093  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9520 after lost signal to thread 1584
W20260709 13:02:33.362388  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbda0 after lost signal to thread 1585
I20260709 13:02:33.363034  5773 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.363519  5773 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.363826  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.364566  1635 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:33.345052  5988 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.359318  5887 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.363988  5773 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.365847  5887 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.367376  5773 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.359418  5883 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.368028  5887 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.368304  5883 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.369863  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.371096  5883 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.371573  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.372450  5673 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.372635  5884 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.372962  5673 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.373229  5988 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:33.373183  5884 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.374342  5988 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:33.375435  5884 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.375957  5884 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.376202  5761 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:33.376637  5761 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:33.376425  5884 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.377559  5761 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.378129  6026 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.378779  5761 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:33.379505  5559 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:33.379850  2262 connection.cc:256] RPC call timeout handler was delayed by -1.777077290999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:02:33.380220  1581 debug-util.cc:398] Leaking SignalData structure 0x7b080040da80 after lost signal to thread 1659
I20260709 13:02:33.380069  5559 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:33.380347  2262 connection.cc:256] RPC call timeout handler was delayed by -1.4429746120000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:02:33.380673  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08003fc6a0 after lost signal to thread 1660
W20260709 13:02:33.380837  2262 connection.cc:256] RPC call timeout handler was delayed by -1.4130034309999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 13:02:33.381215  2262 connection.cc:256] RPC call timeout handler was delayed by -1.0818456669999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 13:02:33.382040  5559 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.375432  5673 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.383450  5673 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.383960  5673 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.384579  5877 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.385774  6006 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.385982  5673 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:33.386761  1598 connection.cc:256] RPC call timeout handler was delayed by -2.3649228900000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 13:02:33.386349  6006 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.372447  1630 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.389122  6006 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.389523  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:33.389765  5982 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.390297  2295 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:33.390302  5982 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.392518  5982 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.392678  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:33.393292  2295 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:33.393677  1598 connection.cc:256] RPC call timeout handler was delayed by -2.1759228899999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:02:33.394047  1598 connection.cc:256] RPC call timeout handler was delayed by -1.4361228899999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.286s for a response
I20260709 13:02:33.395313  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.396524  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.396605  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.397367  1633 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.397435  1634 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.398319  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.398391  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.399008  1633 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:33.353632  6004 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.400242  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.401242  1629 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:33.402364  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.378562  5884 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.399631  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.400360  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.393919  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:33.391594  1581 debug-util.cc:398] Leaking SignalData structure 0x7b0800164860 after lost signal to thread 1733
I20260709 13:02:33.372902  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.403734  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.411311  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.412030   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:33.412377  1559 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.412779  1629 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.404902  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.414515  1626 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:33.415395  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:33.417284  5465 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.067s	user 0.012s	sys 0.041s
I20260709 13:02:33.417163  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.405241  1627 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:33.418474  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.419124  1626 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:33.419049  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.420028  1553 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:33.420622  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.421038  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.421792  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.422850  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.423369  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:33.405176  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.424988  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.425925  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.425814  5979 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.426919  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.405539  1632 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:33.429913  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.430735  1632 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.410185  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.436375  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.437681  1633 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:33.409766  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.404752  1634 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.413621  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.405531  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.465000  1628 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:33.444321  1626 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:33.444945  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.445300  1631 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:33.421805  1553 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:33.482642  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.483597  1556 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 8.
I20260709 13:02:33.449900  1629 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:33.405890  1581 debug-util.cc:398] Leaking SignalData structure 0x7b080034e8a0 after lost signal to thread 1734
I20260709 13:02:33.459662  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:33.408846  1635 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:33.461671  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:33.438606  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.439752  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:33.479094  1674 connection.cc:256] RPC call timeout handler was delayed by -2.169128134999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
I20260709 13:02:33.440135  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:33.513370  1674 connection.cc:256] RPC call timeout handler was delayed by -1.7502078430000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:02:33.488319  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:33.514075  1674 connection.cc:256] RPC call timeout handler was delayed by -1.5819078430000673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
I20260709 13:02:33.514195  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.514464  2296 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:33.514879  1555 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:33.513103  2290 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:33.516060  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76382.2us
I20260709 13:02:33.254509  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:33.517838  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0720 after lost signal to thread 2321
W20260709 13:02:33.517650  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 178122us
I20260709 13:02:33.518939  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:33.520140  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 189034us
I20260709 13:02:33.484794  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.522552  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:33.518783  1673 connection.cc:256] RPC call timeout handler was delayed by -2.1874228959999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
W20260709 13:02:33.523825  1673 connection.cc:256] RPC call timeout handler was delayed by -1.7254230899999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
W20260709 13:02:33.524170  1673 connection.cc:256] RPC call timeout handler was delayed by -1.7019136880000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
W20260709 13:02:33.524427  1673 connection.cc:256] RPC call timeout handler was delayed by -1.3198228960000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2927s for a response
I20260709 13:02:33.523315  1633 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:33.427695  1630 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:02:33.527309  1558 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.496542  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2397) took 3293 ms. Trace:
I20260709 13:02:33.530462  2295 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:33.531208  2292 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:33.532091  1581 debug-util.cc:398] Leaking SignalData structure 0x7b08003968c0 after lost signal to thread 2322
W20260709 13:02:33.532927  1581 debug-util.cc:398] Leaking SignalData structure 0x7b080040dc60 after lost signal to thread 5856
W20260709 13:02:33.409706  5993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.780s	user 0.006s	sys 0.037s
I20260709 13:02:33.536417  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.531620  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.537762  1552 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:33.523020  1556 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.538758  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.492794  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.449304  5979 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:33.539705  1559 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.540284  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.545111  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:33.545050  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.546253  1631 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:33.540299  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.548180  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.548704  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:33.549122  1556 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:33.552172  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.459690  5866 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.559644  2290 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:33.450121  5832 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.451988  6004 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.451651  6064 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.482515  2294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
 from {username='slave'} at 127.0.0.1:57934
I20260709 13:02:33.449994  5890 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:33.459751  5990 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.459318  5868 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:33.594712  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54483.3us
I20260709 13:02:33.614373  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.494974  6005 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.554255  5465 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:33.572748  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.561144  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.619240  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.619989  1552 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.598528  5761 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:33.642292  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.596580  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.643608  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.644035  2295 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:33.528116  1551 rpcz_store.cc:276] 0709 13:02:30.203541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.291887 (+ 88346us) service_pool.cc:224] Handling call
0709 13:02:30.306653 (+ 14766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.320782 (+ 14129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.320830 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.338840 (+3018010us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b2c00000f48. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:02:33.339287 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.409916 (+ 70629us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.486814 (+ 76898us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.496036 (+  9222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.496116 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.496160 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:33.496286 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.496457 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":77419,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":13921,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":3864,"spinlock_wait_cycles":3712384,"thread_start_us":146754,"threads_started":2,"wal-append.queue_time_us":68639}
I20260709 13:02:33.598588  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.443921  6028 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.563158  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:33.543921  5993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.942s	user 0.006s	sys 0.042s
W20260709 13:02:33.598632  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 77527.2us
I20260709 13:02:33.599280  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.653452  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:33.650046  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.612696  5378 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:33.603744  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.666730  5378 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.667069  2293 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:33.667394  5378 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.603896  2294 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:33.668193  2294 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:33.478514  5831 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.671442  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.607736  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.617270  5979 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.595086  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.677404  2296 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:33.624732  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.636866  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.678624  2290 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:33.625381  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.588943  1556 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:33.681393  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.536223  5771 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:33.597137  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:33.643306  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 387125us
I20260709 13:02:33.644347  1705 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:33.682658  1601 connection.cc:256] RPC call timeout handler was delayed by -2.5852536500000269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:02:33.646219  1631 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:33.646078  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.655113  1703 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:33.598994  6050 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.654058  1635 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:33.604969  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:33.663969  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.664505  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.692586  1559 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:33.693850  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.694862  1559 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:33.664016  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.608712  5640 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.602860  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.672608  1558 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.544693  5591 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:33.673995  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.674085  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.675544  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.553133  1553 raft_consensus.cc:2376] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9. Re-sending same reply.
W20260709 13:02:33.683461  1601 connection.cc:256] RPC call timeout handler was delayed by -2.5632423620000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
I20260709 13:02:33.684264  1707 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:33.686908  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.620523  6006 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:33.687852  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.604045  5866 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.710359  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.710742  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.609555  6004 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.714733  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.608359  5832 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.696130  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.730278  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.705731  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.619144  5868 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.708698  1708 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:33.706513  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.652616  5993 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:33.614300  6064 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:33.698146  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 173223us
I20260709 13:02:33.643097  5761 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.616689  5890 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.775935  5761 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:33.710207  6065 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:33.705161  1601 connection.cc:256] RPC call timeout handler was delayed by -2.5258812170000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3022s for a response
I20260709 13:02:33.658268  5465 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:33.721499  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:33.783982  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 62598.1us
I20260709 13:02:33.784626  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.714907  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.785310  1556 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:33.619612  5990 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.663949  6055 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:02:33.729635  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 657) took 3667 ms. Trace:
I20260709 13:02:33.706297  1552 raft_consensus.cc:2410] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:33.731701  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.645228  6005 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.706722  1555 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:33.698769  5912 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:33.730753  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.733106  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.733944  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.729453  5735 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.701628  5979 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.736302  5832 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.736970  1456 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 6 to 8, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:33.739393  5866 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.719936  5560 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.690745  5831 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.739845  2291 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:02:33.740367  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 328) took 13600 ms (client timeout 15000 ms). Trace:
I20260709 13:02:33.741268  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.709528  6006 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.744170  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.805347  5887 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:33.745718  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.746413  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.749150  1551 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:33.750633  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.750809  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.739857  5591 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:33.754952  1631 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:33.709961  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.754997  2294 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.881673  2288 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:33.881832  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.766317  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.882647  1559 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:33.701524  5884 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:33.700335  2292 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:33.778939  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.780176  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet c4b89801fe624f179701cbd4b98f1490 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:56080
W20260709 13:02:33.884186  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 105343us
I20260709 13:02:33.780408  6066 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:33.780926  5890 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.781553  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:33.787707  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 101354us
I20260709 13:02:33.789356  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:33.885275  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 97893.8us
I20260709 13:02:33.887970  1628 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:33.790056  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:33.791018  1635 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:33.792630  1560 rpcz_store.cc:276] 0709 13:02:30.053032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.358047 (+3305015us) service_pool.cc:224] Handling call
0709 13:02:33.358377 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.358575 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:33.605697 (+247122us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.720128 (+114431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.720183 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.720238 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.720310 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.720447 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13092,"prepare.queue_time_us":128282,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":15947,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4023,"thread_start_us":327568,"threads_started":2}
I20260709 13:02:33.803644  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:33.808193  1601 connection.cc:256] RPC call timeout handler was delayed by -2.2333536500000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2268s for a response
I20260709 13:02:33.808849  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.889401  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.889081  1708 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:33.811581  1702 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:02:33.810418  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78801.5us
I20260709 13:02:33.891139  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.807521  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.812767  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.892516  1552 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:33.816697  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.816558  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.894155  1555 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:33.810386  5912 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:33.745000  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:33.878611  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.899379  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.900055  1556 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:33.806154  6012 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:33.718389  6050 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.774607  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:33.884245  6051 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:33.880728  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:33.885402  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:33.909565  2290 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:33.876479  1457 rpcz_store.cc:269] 0709 13:02:20.139922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.340602 (+200680us) service_pool.cc:224] Handling call
0709 13:02:33.740280 (+13399678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:32.894846 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:32.895511 (+   665us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:32.895566 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:02:32.895581 (+    15us) tablet.cc:662] Decoding operations
0709 13:02:32.897737 (+  2156us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:32.897829 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:32.897848 (+    19us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:02:32.899199 (+  1351us) tablet.cc:701] Row locks acquired
0709 13:02:32.899240 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:02:32.899388 (+   148us) write_op.cc:270] Start()
0709 13:02:32.899500 (+   112us) write_op.cc:276] Timestamp: P: 1783602152899325 usec, L: 0
0709 13:02:32.899513 (+    13us) op_driver.cc:348] REPLICATION: starting
0709 13:02:32.900386 (+   873us) log.cc:844] Serialized 9796 byte log entry
0709 13:02:32.907631 (+  7245us) op_driver.cc:464] REPLICATION: finished
0709 13:02:32.917869 (+ 10238us) write_op.cc:301] APPLY: starting
0709 13:02:32.917982 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:32.929277 (+ 11295us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:32.929416 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:32.943752 (+ 14336us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:32.944197 (+   445us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:02:32.944250 (+    53us) write_op.cc:312] APPLY: finished
0709 13:02:32.944752 (+   502us) log.cc:844] Serialized 220 byte log entry
0709 13:02:32.945690 (+   938us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:32.945911 (+   221us) write_op.cc:454] Released schema lock
0709 13:02:32.945978 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:32.949408 (+  3430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:32.949440 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":976338,"reported_tablets":49,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":9911,"apply.run_cpu_time_us":27538,"apply.run_wall_time_us":31521,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108727,"num_ops":25,"prepare.queue_time_us":1412,"prepare.run_cpu_time_us":6408,"prepare.run_wall_time_us":7107,"raft.queue_time_us":979,"raft.run_cpu_time_us":7017,"raft.run_wall_time_us":13770,"replication_time_us":7484,"thread_start_us":14942,"threads_started":4,"wal-append.queue_time_us":2126}]]}
I20260709 13:02:33.905355  1456 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:33.886706  6069 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.886878  2294 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:33.918108  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:33.913659  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 110116us
I20260709 13:02:33.919368  1559 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:33.889942  2295 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:33.720444  5640 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:33.922725  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.923436  1555 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:33.890305  2292 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:33.800227  5883 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:33.879076  6006 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:33.895706  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:33.889678  1601 connection.cc:256] RPC call timeout handler was delayed by -2.1865858500000286s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 13:02:33.925215  1601 connection.cc:256] RPC call timeout handler was delayed by -2.1339423620000275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 13:02:33.925540  1601 connection.cc:256] RPC call timeout handler was delayed by -1.9975858500000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:02:33.925819  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8250368420000314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 13:02:33.896487  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:33.897684  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 90242.3us
I20260709 13:02:33.886966  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:33.883989  5591 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.903000  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.886700  5884 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.897158  5834 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:33.926728  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:33.886185  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69572.5us
I20260709 13:02:33.805018  6065 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.887089  5866 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.949030  1456 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:33.933203  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:33.934041  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.953312  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.954532  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.906980  6026 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:33.907617  5735 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:33.937971  2288 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:33.938988  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:33.926080  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7566368420000344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
I20260709 13:02:33.951354  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:33.940897  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49837.5us
I20260709 13:02:33.913522  5887 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:33.923794  5983 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:33.912292  6073 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.919639  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.928864  5834 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.950124  6072 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:33.955722  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.933470  5591 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:33.957825  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.949414  5768 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:33.928958  5884 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:33.958314  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.983338  2290 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:33.963901  1555 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
W20260709 13:02:33.964243  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7525216820000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 13:02:33.989683  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5820368420000364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 13:02:33.990211  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5073368420000293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 13:02:33.990609  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3106335429999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 13:02:33.990996  1601 connection.cc:256] RPC call timeout handler was delayed by -1.2782335429999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:02:33.962416  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:33.970896  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:33.970934  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:33.967608  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:33.975066  1553 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:02:33.964802  1559 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:33.978849  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 59299.9us
I20260709 13:02:33.971699  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:33.994508  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:33.992887  1553 pending_rounds.cc:85] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:02:33.995704  1553 pending_rounds.cc:107] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:02:33.995947  1553 raft_consensus.cc:2889] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.998318  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:34.004540  1558 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:34.005080  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:33.967427  6063 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:02:34.016113  1551 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:34.009465  1601 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.532s (SENT)
W20260709 13:02:34.017805  1601 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.684s (SENT)
I20260709 13:02:34.018427  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:34.021096  1632 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.004s
I20260709 13:02:34.021683  5735 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:34.022274  1552 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:02:34.023078  6068 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:02:33.912827  5993 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:33.932888  6069 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.986897  6052 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:33.986824  5768 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:33.979926  5834 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:33.956310  5883 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:33.976967  5832 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:02:34.029186  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:33.969831  6028 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.961632  6026 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:34.018007  1456 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 4 to 6, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 6 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.033243  1558 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:34.033749  1632 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:33.974834  5378 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.974007  5983 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:33.981163  6073 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.038386  6004 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:33.912467  5465 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.017215  5673 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:34.027904  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104521us
I20260709 13:02:33.962703  5866 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.099409  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.035519  1559 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:34.060973  1555 pending_rounds.cc:85] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:02:34.057258  5771 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:34.068230  6070 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.037977  6052 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.075451  5983 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:34.078814  1551 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:02:34.091849  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 161081us
I20260709 13:02:34.077157  6005 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.057178  6026 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:34.040266  1553 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:34.123450  6066 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:34.121313  1555 pending_rounds.cc:107] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:02:34.026569  5912 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:33.999818  6072 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.127419  2293 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:34.072844  5560 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.045311  5768 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.129925  1559 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:34.128595  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:34.138119  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:34.056485  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:34.141490  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123723us
I20260709 13:02:34.117007  5673 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.117156  5832 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:34.145573  1555 raft_consensus.cc:2889] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:34.069097  5868 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.122728  6004 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:34.118309  5979 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.127813  5771 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.122087  5831 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.153643  6064 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.158579  5560 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.180441  5560 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.133363  6052 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.187139  6006 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.188437  2297 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:34.130115  6070 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.190040  6006 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:34.190341  6070 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.190759  6006 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:34.163295  5673 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:34.160190  1601 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.720s (SENT)
I20260709 13:02:34.161228  1456 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 6 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:34.192523  1601 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.766s (SENT)
I20260709 13:02:34.192951  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:34.193126  1456 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 9 to 10, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:02:34.194885  1601 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.902s (SENT)
I20260709 13:02:34.194599  1456 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 7 to 11, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 11 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.164366  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:34.135249  5990 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.146714  6012 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:02:34.163496  5832 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.197220  6012 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:34.073212  5887 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.169761  5378 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:34.198194  6088 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:34.198598  5735 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.197500  5832 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:34.200822  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.096s (SENT)
W20260709 13:02:34.203097  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 204834us
I20260709 13:02:34.139518  6079 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.204933  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:34.211699  2290 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:34.213150  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.214332  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50061.6us
I20260709 13:02:34.216429  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:34.164039  6055 raft_consensus.cc:2764] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:34.164597  6005 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.179112  5761 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.231335  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:34.157327  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:34.202457  5771 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.167557  5640 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.202234  6004 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:34.241428  1601 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.179s (SENT)
I20260709 13:02:34.203213  6086 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:34.214781  6079 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.227577  5990 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:34.245623  6073 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.271979  1444 catalog_manager.cc:5028] TS 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): tablet c4b89801fe624f179701cbd4b98f1490 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:02:34.274314  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.275168  1706 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:34.194921  5834 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.276796  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.277392  5834 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:34.245803  5890 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:34.246047  6077 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:02:34.277390  1708 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:34.278004  5890 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:34.237807  6064 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:34.256340  6079 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.278921  6064 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:34.206332  6071 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:34.253163  6028 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:34.253008  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.280369  6028 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:34.280577  1700 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:34.281090  6064 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.281316  6028 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.280499  6071 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:34.284228  6071 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:34.285060  6071 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.253898  6069 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.285878  6052 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.259824  6072 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.262166  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.286448  6071 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:34.262708  6086 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.256253  5990 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.288115  5990 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:34.245390  6078 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:02:34.268179  1601 leader_election.cc:336] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.660s (SENT)
I20260709 13:02:34.278970  5890 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.242156  6065 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.233234  6005 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:34.295377  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 134916us
W20260709 13:02:34.296733  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.006s	sys 0.003s
W20260709 13:02:34.297102  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.006s	sys 0.003s
I20260709 13:02:34.205202  5883 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:34.241714  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 86536.9us
I20260709 13:02:34.297520  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:34.303216  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.304409  1708 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:34.308482  1703 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:34.328635  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 272190us
I20260709 13:02:34.321543  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.331267  1700 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 8.
I20260709 13:02:34.332520  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:34.336920  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.337139  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.294845  6005 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.350131  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.354048  1702 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:34.355217  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.357450  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.368973  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.360212  1703 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:34.370241  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.286223  6026 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.374140  1705 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:34.375010  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1799) took 1212 ms. Trace:
I20260709 13:02:34.386032  1557 rpcz_store.cc:276] 0709 13:02:33.162650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.174592 (+ 11942us) service_pool.cc:224] Handling call
0709 13:02:34.368712 (+1194120us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003098d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:34.374933 (+  6221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677235456}
I20260709 13:02:34.356818  1706 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:34.355931  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.286932  6070 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.305058  6065 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:34.397588  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.402594  1700 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:34.348611  5735 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:34.300530  5591 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:34.399447  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 510622us
I20260709 13:02:34.355108  6055 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:34.418848  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.419287  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.372475  6052 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.436373  5761 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:34.471185  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52355.8us
I20260709 13:02:34.477439  1707 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:34.306722  6050 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:34.477041  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 505369us
I20260709 13:02:34.538121  1675 leader_election.cc:400] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11.
W20260709 13:02:34.480927  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 62172.3us
W20260709 13:02:34.539840  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.029s	sys 0.043s
I20260709 13:02:34.498906  5771 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.503217  5591 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.518232  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1818) took 1338 ms. Trace:
I20260709 13:02:34.519615  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.520383  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.520682  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.521070  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.553966  1709 raft_consensus.cc:2410] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:34.522358  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.321336  6004 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.527479  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.481123  5887 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:02:34.528402  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.528503  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.528421  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.536489  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:34.480450  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:34.540182  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.029s	sys 0.043s
W20260709 13:02:34.537744  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58136.6us
I20260709 13:02:34.546669  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.547621  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.552807  1556 rpcz_store.cc:276] 0709 13:02:33.179536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.016020 (+836484us) service_pool.cc:224] Handling call
0709 13:02:34.039423 (+ 23403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.044409 (+  4986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.044455 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.068157 (+ 23702us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.330177 (+262020us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:34.381789 (+ 51612us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.382052 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.382160 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.517999 (+135839us) raft_consensus.cc:1685] finished
0709 13:02:34.518019 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.518168 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52076,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":110039,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":9769,"thread_start_us":201002,"threads_started":1,"wal-append.queue_time_us":416925}
I20260709 13:02:34.472427  6026 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.549665  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.476300  5866 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.531215  5673 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.559670  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.487223  6055 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.581315  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:34.593207  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:34.593976  2290 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.580107  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.571878  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.574709  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.584419  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:34.597117  1626 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:34.579077  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:34.565716  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 598375us
I20260709 13:02:34.598315  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.598719  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.598863  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.599195  2295 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:34.599961  2263 leader_election.cc:400] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:34.600271  2263 leader_election.cc:403] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:34.566027  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.418329  6065 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:34.586323   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:34.593947  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:34.570865  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.562266  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 308577us
I20260709 13:02:34.604862  1702 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:34.606231  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.607959  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.609084  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:34.610970  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44673us
I20260709 13:02:34.551756  6094 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.478685  6099 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:34.612115  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.611196  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.604599  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.620010  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.613595  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.627400  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.617007  1702 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:34.628417  1705 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:34.636158  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.636389  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.636158  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.640223  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.640483  2292 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:34.642477  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.644490  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.643352  2294 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:34.649148  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:34.654759  2292 raft_consensus.cc:2368] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12.
I20260709 13:02:34.656630  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.656952  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.657415  1703 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:34.659795  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46281.1us
I20260709 13:02:34.663615  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:34.664038  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:34.665457  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 310313us
I20260709 13:02:34.664705  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.666973  1705 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:34.668008  1704 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:02:34.666136  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 57125.9us
I20260709 13:02:34.668838  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.670776  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 675) took 1051 ms. Trace:
I20260709 13:02:34.604496  5591 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:34.672917  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61805.3us
I20260709 13:02:34.610342  6050 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.612716  5761 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.674011  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:34.674453  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:34.671196  1554 rpcz_store.cc:276] 0709 13:02:33.619157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.055795 (+436638us) service_pool.cc:224] Handling call
0709 13:02:34.085906 (+ 30111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.086184 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:34.131372 (+ 45188us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.163529 (+ 32157us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.252322 (+ 88793us) spinlock_profiling.cc:243] Waited 9.71 ms on lock 0x7b18003d9480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:34.253164 (+   842us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18003d9480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:34.293787 (+ 40623us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b18003d9480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:34.670427 (+376640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.670482 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:34.670542 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.670613 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.670721 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88254,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":8040,"raft.queue_time_us":35584,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":48492416,"thread_start_us":406060,"threads_started":2,"wal-append.queue_time_us":250488}
I20260709 13:02:34.611992  5640 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:34.676565  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.627018  5912 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.637727  6073 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:34.682302  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:34.627619  5834 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.561200  5560 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.680696  2290 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:34.636827  5993 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.642366  6083 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:34.636466  6052 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:34.663029  6004 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:34.659789  6095 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.678467  5912 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:34.706558  1709 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:34.686237  6089 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:34.709319  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.707269  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:34.711109  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 111256us
I20260709 13:02:34.712548  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.636267  5735 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:34.740252  6052 consensus_queue.cc:1243] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:34.679489  6028 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:34.756740  6028 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:34.756614  5735 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.757220  6028 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:34.760317  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47836.4us
I20260709 13:02:34.760759  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.679950  6026 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.762020  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.762745  1628 raft_consensus.cc:2368] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11.
I20260709 13:02:34.719309  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:34.713868  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:34.676527  5591 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:34.679663  5640 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.678658  5761 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.539018  5983 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:34.769004  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55204.1us
I20260709 13:02:34.643115  5768 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.735706  1558 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:34.739838  6087 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:02:34.740489  5912 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.780421  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.740203  6088 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:34.740769  6098 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:02:34.742340  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.743615  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.715310  6064 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:34.769443  5761 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.768580  6095 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:34.749349  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.751111  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:34.793049  5884 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.000s
W20260709 13:02:34.768458  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49227.9us
I20260709 13:02:34.751748  5890 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.770414  1705 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
W20260709 13:02:34.770865  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 104644us
I20260709 13:02:34.726078  6012 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.686861  6055 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:02:34.724272  6050 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:34.716759  6083 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.735401  6073 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:34.708837  6005 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.642755  6094 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.768133  6006 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:34.772609  5591 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:34.769271  5640 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.744882  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.775301  6069 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.805500  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.770153  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.807417  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:34.808059  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 141052us
W20260709 13:02:35.025889  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 255786us
I20260709 13:02:34.810313  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:35.028513  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 218262us
I20260709 13:02:35.029450  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.812227  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:35.032289  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.032568  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.033567  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.033754  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.034166  1559 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:34.813438  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:34.793942  6104 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:34.814043  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:34.805812  6071 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
W20260709 13:02:34.806051  6079 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 13:02:34.803737  6070 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.804255  6064 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:34.833878  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:35.078588  1705 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:34.861922  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:34.868738  5771 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.081176  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.081965  1556 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:35.083156  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.083935  1554 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.814529  5884 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.086232  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:35.086377  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:34.876055  6102 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 13:02:34.891300  6101 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:02:34.892768  6116 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:34.902025  5673 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:34.903640  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2411) took 1096 ms. Trace:
I20260709 13:02:34.831285  6088 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:02:34.905278  5912 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.095757  5912 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:35.036424  6069 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.096477  6069 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.908150  5983 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.908501  5560 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:34.912814  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 676) took 1273 ms. Trace:
W20260709 13:02:35.109469  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 347253us
I20260709 13:02:34.815893  6012 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:35.115350  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 341640us
I20260709 13:02:34.917896  6100 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:34.921332  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.922974  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:34.924314  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:35.121846  1627 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:35.121752  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200536us
I20260709 13:02:34.934003  6086 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:35.123449  6086 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.124110  6066 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:34.934504  6083 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:34.936328  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 682) took 1183 ms. Trace:
W20260709 13:02:34.938115  5832 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.000s
I20260709 13:02:34.939256  5465 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:34.947046  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:35.126833  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:34.867403  6115 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.128974  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.037597  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.039525  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:35.042934  1557 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:35.131232  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.044657  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:34.996191  6117 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:35.132359  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 92903.4us
I20260709 13:02:35.074673  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:35.132970  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.133669  1704 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:35.133409  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:35.075423  1552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.026s	sys 0.046s
I20260709 13:02:35.077466  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:35.134373  1552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.027s	sys 0.046s
I20260709 13:02:35.087155  1702 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
W20260709 13:02:35.134531  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 389058us
I20260709 13:02:35.086073  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.088013  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.092947  6119 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.141140  1709 raft_consensus.cc:2368] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12.
I20260709 13:02:35.093369  5884 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.093937  5735 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:34.906800  6113 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.042711  6050 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.088999  1560 rpcz_store.cc:276] 0709 13:02:33.807675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.071110 (+263435us) service_pool.cc:224] Handling call
0709 13:02:34.100440 (+ 29330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:34.129572 (+ 29132us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:34.158281 (+ 28709us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.603641 (+445360us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:34.626802 (+ 23161us) log.cc:844] Serialized 52 byte log entry
0709 13:02:34.632002 (+  5200us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:34.632043 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.632112 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.903386 (+271274us) raft_consensus.cc:1685] finished
0709 13:02:34.903407 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.903567 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30367,"prepare.queue_time_us":28646,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":54055,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":17715,"thread_start_us":350888,"threads_started":1,"wal-append.queue_time_us":587989}
I20260709 13:02:35.120975  6100 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.076941  5993 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.110651  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.109786  1555 rpcz_store.cc:276] 0709 13:02:33.639541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.060297 (+420756us) service_pool.cc:224] Handling call
0709 13:02:34.060731 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.195898 (+135167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.195933 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.196375 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.301247 (+104872us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:34.301284 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.301350 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.912574 (+611224us) raft_consensus.cc:1685] finished
0709 13:02:34.912599 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.912747 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":104520,"threads_started":1,"wal-append.queue_time_us":274020}
I20260709 13:02:35.088105  6120 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:02:35.130529  5465 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.080957  6064 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.124372  6086 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.125358  1558 rpcz_store.cc:276] 0709 13:02:33.752967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.070961 (+317994us) service_pool.cc:224] Handling call
0709 13:02:34.071225 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.775602 (+704377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.936015 (+160413us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b540036f6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:34.936248 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513174912}
I20260709 13:02:35.081250  6104 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:35.126456  1633 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:35.126554  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.194211  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:34.893116  6080 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:35.006078  1703 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:35.132653  1559 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:35.201545  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.076647  1706 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:02:35.135320  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 428130us
W20260709 13:02:35.136535  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99028.2us
I20260709 13:02:35.161235  6070 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:35.205427  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:35.134176  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.138489  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:35.134580  1553 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:35.206549  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 72438.2us
I20260709 13:02:35.132372  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.146843  1554 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:35.077172  6079 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:34.861887  6106 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.351s
I20260709 13:02:35.166643  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:35.168134  1551 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:35.174122  1555 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:35.171433  6026 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.121557  6012 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.142968  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:35.172528  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62000.1us
I20260709 13:02:35.188977  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:35.191176  1601 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.643s (SENT)
W20260709 13:02:35.225610  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59108.9us
I20260709 13:02:35.194794  1558 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.141335  6099 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.208179  1557 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:35.242246  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.209676  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.210592  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.245534  1553 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:35.211738  6071 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.131734  5832 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.113168  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.215758  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.257869  1556 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:35.214778  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 559113us
I20260709 13:02:35.217990  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.259735  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.127460  5912 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:02:35.229593  1635 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:35.200843  6066 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.231792  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.231608  6100 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.233736  6079 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.234328  6086 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.184003  5884 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:35.210484  5735 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:02:35.236127  6081 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:35.279443  5990 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.991s	user 0.010s	sys 0.003s
I20260709 13:02:35.208573  6070 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.135855  6103 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
W20260709 13:02:35.286446  6028 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.002s
I20260709 13:02:35.281067  1551 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.212831  6095 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.283475  1560 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:35.163276  6119 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.140494  6073 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.248926  5771 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:35.291718  1599 leader_election.cc:400] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12.
I20260709 13:02:35.311259  1599 leader_election.cc:403] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:02:35.293072  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:35.300252  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 157365us
I20260709 13:02:35.268313  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.212155  6123 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:35.263655  5560 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.299386  6070 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.306023  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:35.309634  1555 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:35.319185  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.343499  2297 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.319855  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.286548  6106 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:35.339083  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1819) took 2159 ms. Trace:
I20260709 13:02:35.340823  6123 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:35.350745  6123 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:35.351402  5761 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.538s	user 0.005s	sys 0.001s
I20260709 13:02:35.350330  1552 rpcz_store.cc:276] 0709 13:02:33.185516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.023067 (+837551us) service_pool.cc:224] Handling call
0709 13:02:34.023406 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:34.023818 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:34.024131 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.198253 (+174122us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18004d0200. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:35.134859 (+936606us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:35.141407 (+  6548us) log.cc:844] Serialized 52 byte log entry
0709 13:02:35.141668 (+   261us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:35.285783 (+144115us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.310923 (+ 25140us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.312152 (+  1229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.338680 (+ 26528us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.338759 (+    79us) raft_consensus.cc:1685] finished
0709 13:02:35.338775 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.338979 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3553,"prepare.queue_time_us":220656,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":123,"raft.queue_time_us":79676,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":9228,"spinlock_wait_cycles":27516160,"thread_start_us":1207095,"threads_started":4,"wal-append.queue_time_us":658025}
I20260709 13:02:35.318590  1556 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:35.311208  5771 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.356524  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:35.356963  6004 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 13:02:35.357120  5771 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.358330  5832 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:35.355876  6100 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:35.360847  5890 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:02:35.343972  6113 raft_consensus.cc:2764] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:35.356572  1558 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:35.305599  5735 consensus_queue.cc:1243] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:35.290292  6087 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:02:35.356071  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.369072  1701 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:35.296375  5887 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:35.369864  1551 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:35.370214  1708 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:02:35.370877  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64964.3us
I20260709 13:02:35.383322  1553 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:35.384784  5771 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.387012  5771 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:35.392727  5560 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.262669  5465 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:35.317459  6104 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.314389  6081 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:35.345064  6079 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.303025  5990 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.382589  5761 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.316437  5988 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:35.305845  6028 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.419387  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:35.419108  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 152204us
I20260709 13:02:35.423957  1553 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:35.424642  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.381330  6120 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.313781  6006 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:02:35.426542  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.437685  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:35.412242  6081 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.435282  5735 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.417387  6105 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.442868  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:35.445749  1702 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:35.417534  5887 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:35.446599  2288 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
I20260709 13:02:35.449422  2294 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:35.450553  2296 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:35.455687  1553 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:35.457229  1598 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
I20260709 13:02:35.461186  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:35.453493  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.446841  6081 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.417287  6130 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
W20260709 13:02:35.465988  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:35.479279  6139 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:35.461197  5591 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.504793  6139 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:35.505251  6139 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.418767  6123 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.514448  5771 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:35.457763  6105 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.482697  6124 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.201s
I20260709 13:02:35.450506  5988 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:35.489385  6129 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:35.493148  6126 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:02:35.496021  6145 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:35.450693  6141 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:35.498153  6028 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:35.499933  5990 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:35.500624  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.504096  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.569140  1551 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.455530  5887 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.516343  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1836) took 1123 ms. Trace:
W20260709 13:02:35.575135  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114038us
I20260709 13:02:35.515035  5771 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:35.531749  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68001us
I20260709 13:02:35.532394  5883 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:35.531816  6121 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:35.575730  5735 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:35.577881  5768 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:35.578699  6052 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:35.600262  6052 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.601208  6052 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.602133  6050 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.579584  5591 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:35.536257  6145 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.515575  5640 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:35.631932  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44936.5us
I20260709 13:02:35.631613  6145 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.632912  5640 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.463920  5560 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.633225  6145 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.581998  5988 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.634006  1700 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:35.516949  6143 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.633216  1598 leader_election.cc:400] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12.
I20260709 13:02:35.586232  1706 rpcz_store.cc:276] 0709 13:02:34.392871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.397013 (+  4142us) service_pool.cc:224] Handling call
0709 13:02:35.516142 (+1119129us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54003e1f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:35.516277 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509775104}
I20260709 13:02:35.639585  1551 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:02:35.588241  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 380245us
I20260709 13:02:35.645285  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:35.649889  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.514914  6123 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:35.462852  6106 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 13:02:35.592625  5887 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.673741  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.720605  1704 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.593951  5771 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.596191  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.723272  2293 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:35.596589  5465 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.597012  5735 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.602681  5761 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:35.603030  6052 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.567822  6095 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:35.604410  6119 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:35.605051  6069 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.006s
I20260709 13:02:35.605857  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:35.606258  6105 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.606563  6065 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:35.744516  2294 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:35.608265  1628 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:35.748425  1628 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.611287  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1833) took 1257 ms. Trace:
I20260709 13:02:35.611779  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1830) took 1263 ms. Trace:
I20260709 13:02:35.612798  6122 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:35.620576  2292 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.624195  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:35.629733  2297 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:35.607446  6070 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.653864  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.757097  1551 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:35.656319  2262 leader_election.cc:400] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11.
I20260709 13:02:35.758268  2262 leader_election.cc:403] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:35.759248  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 103126us
I20260709 13:02:35.761889  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.658223  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.764156  1706 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.766399  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:35.672960  5640 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:35.750885  1628 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.781343  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.782582  1555 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:35.783703  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.777240  1628 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:35.785073  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.744302  6146 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:35.660915  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.796236  1700 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:35.800936  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.721515  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.722347  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.674978  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.730103  1709 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:02:35.654944  6050 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:35.736590  5993 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.014s
W20260709 13:02:35.740540  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.737288  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.808353  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:35.737181  5735 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.749377  1701 rpcz_store.cc:276] 0709 13:02:34.354414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.355073 (+   659us) service_pool.cc:224] Handling call
0709 13:02:35.611097 (+1256024us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b540010e0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:35.611228 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291197824}
I20260709 13:02:35.749944  1708 rpcz_store.cc:276] 0709 13:02:34.348891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.349891 (+  1000us) service_pool.cc:224] Handling call
0709 13:02:35.611603 (+1261712us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540010d6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:35.611703 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2324509952}
I20260709 13:02:35.778482  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:35.588585  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.668960  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.745698  6005 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.672194  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:35.754099  6122 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:35.670420  6081 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.776517  5591 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.847312  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.847961  1552 raft_consensus.cc:2376] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16. Re-sending same reply.
I20260709 13:02:35.839184  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.729393  6088 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:02:35.841421  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 217316us
I20260709 13:02:35.851404  1599 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
I20260709 13:02:35.851691  1599 leader_election.cc:403] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:02:35.841665  1703 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
I20260709 13:02:35.809731  1707 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.776923  5771 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.842746  1706 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:02:35.847213  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.757129  5887 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.784065  6055 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.778501  6104 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:35.795631  6146 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:35.854805  1628 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:35.907379  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 706) took 1166 ms. Trace:
I20260709 13:02:35.842584  5993 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:35.814713  1704 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:35.752136  6139 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.786351  6141 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.858204  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 79800.1us
I20260709 13:02:35.921646  6079 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12.
W20260709 13:02:35.859555  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 190721us
I20260709 13:02:35.863137  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:35.863441  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:35.975463  1627 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.752034  6069 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:35.850270  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.990144  1556 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:35.848969  6083 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.006s
I20260709 13:02:35.867383  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:35.995070  1631 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:35.881551  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.881361  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:35.997043  1629 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:35.854995  1706 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:35.868829  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.999621  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:36.000144  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:36.000474  1552 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 10: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:36.001299  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:36.001586  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.770630  6123 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11.
W20260709 13:02:35.903139  1585 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37587 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37587 timed out after 15.000s (SENT)
I20260709 13:02:35.916385  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:35.917459  1632 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:02:36.006089  1635 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:35.815490  6050 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:35.908383  5673 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:35.921864  1703 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:35.907744  1628 rpcz_store.cc:276] 0709 13:02:34.741182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.799269 (+ 58087us) service_pool.cc:224] Handling call
0709 13:02:35.608140 (+808871us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540021d2a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e7e18 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:35.907322 (+299182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457617664}
I20260709 13:02:35.922124  6070 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:35.922556  6149 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:35.884581  5912 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:36.004359  6064 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.781s	user 0.007s	sys 0.008s
I20260709 13:02:35.934587  1707 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:02:35.946235  6117 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.002s
I20260709 13:02:35.866979  6104 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.905462  6005 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:35.958130  5983 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:02:35.973871  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:35.975595  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.992607  1555 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:36.002334  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:35.910288  1709 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:36.006660  1631 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:35.969512  5640 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.028749  1556 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:02:36.029141  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53633.9us
I20260709 13:02:36.029903  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:36.033263  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:35.881455  1634 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:36.008406  6050 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.751870  6095 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.015558  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:35.905406  6055 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:35.973096  6105 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:02:36.044247  1627 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:35.963869  6141 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.049144  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.047140  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:36.099234  1556 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:36.003572  6083 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.085041  6153 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:02:36.068022  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:36.019853  5912 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.034523  6117 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:36.121443  1555 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:36.035867  6070 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:36.085037  1635 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:35.974298  6122 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.050880  6005 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:36.094116  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 242824us
I20260709 13:02:36.140141  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:36.140376  1551 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:36.141088  2288 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:02:36.139118  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:36.141439  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:36.075466  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:36.099370  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50354.6us
W20260709 13:02:36.142496  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.881892  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:36.101809  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:36.145311  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.145965  1556 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:36.146905  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.147722  1626 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.148795  1635 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:36.022482  6104 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:36.022953  6064 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.040836  5640 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.113850  1552 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:36.172720  2297 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:36.173448  1552 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:36.116003  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 185347us
I20260709 13:02:36.174393  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.116900  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.176172  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.176867  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.111012  6094 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.086336  6149 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.173147  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:36.150808  6139 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.017966  6146 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.156838  6055 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:36.158639  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 83300.2us
I20260709 13:02:36.141528  1551 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:36.183364  1707 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:36.183719  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:36.184906  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:36.185526  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:36.185941  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:36.159036  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.181687  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.176993  1634 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:36.187147  1632 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.160010  1631 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:36.160333  6117 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:36.162359  6012 consensus_meta.cc:287] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.907s	user 0.008s	sys 0.006s
I20260709 13:02:36.163069  2289 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:02:36.113384  6088 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.164450  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.196635  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.172720  5761 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:36.178339  6115 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.017s	user 0.008s	sys 0.009s
I20260709 13:02:36.178519  1708 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:36.204150  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.186471  6005 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.181479  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.181342  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:36.151804  6159 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:36.163894  6123 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:02:36.187875  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.189345  1703 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:02:36.172309  6080 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.960s	user 0.014s	sys 0.001s
I20260709 13:02:36.190410  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.179162  5673 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:36.191741  1552 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:36.198477  6055 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:36.167248  6087 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:36.193585  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.200127  6012 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.186514  1630 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.109786  6076 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:36.198140  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:36.217690  6139 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.172217  6150 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.181792  6122 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.204221  1559 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:36.204767  1556 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:36.266709  1551 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:36.181385  5834 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:36.261291  6087 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.204946  1708 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:36.122951  1555 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:36.179729  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62923.5us
I20260709 13:02:36.182583  2293 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:36.275152  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:36.206373  1553 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:02:36.275434  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.191547  5560 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:36.222782  1598 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 9 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
W20260709 13:02:36.276399  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53749.9us
I20260709 13:02:36.191124  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.192041  1706 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:02:36.277750  1706 pending_rounds.cc:85] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 11
I20260709 13:02:36.277980  1706 pending_rounds.cc:107] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:02:36.278226  1706 raft_consensus.cc:2889] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.234660  6080 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:36.205476  6052 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:02:36.200923  6006 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:36.236394  1703 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:36.201987  6069 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:36.290306  2290 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:36.290871  1701 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:36.193748  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.292498  1703 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 13:02:36.244508  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:36.166021  6081 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:36.206712  6115 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:36.298565  2290 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:02:36.257222  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 449811us
I20260709 13:02:36.264269  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.300171  1629 raft_consensus.cc:2368] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
I20260709 13:02:36.205587  5988 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:36.279399  1632 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:36.205564  6151 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:36.280575  1708 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:36.280789  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:36.282655  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.308495  1558 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:36.309551  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:36.217620  6113 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:36.283645  2296 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:36.262008  6160 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:02:36.285861  1554 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:36.279145  5560 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.237833  6117 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:02:36.293687  1628 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:36.265904  6139 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.302911  1703 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:36.274408  6150 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.279076  6076 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:36.311899  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.312258  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:36.299049  6145 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.665s	user 0.019s	sys 0.000s
I20260709 13:02:36.332069  6147 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:36.312561  6069 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.320948  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:36.274042  5834 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.315953  1626 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:36.267375  6055 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:36.340270  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.300572  6006 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:36.432569  1632 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:36.413682  1630 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:36.343753  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.343465  1631 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:02:36.361785  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49951.3us
I20260709 13:02:36.436785  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.438391  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:36.446309  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.434345  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:36.432641  1705 raft_consensus.cc:2368] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
I20260709 13:02:36.434345  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:36.442989  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 122179us
I20260709 13:02:36.452325  5771 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:36.452747  1626 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:36.452983  6094 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:36.454255  5735 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:36.454763  5735 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:36.455735  5735 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.457737  1633 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:36.310976  5988 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:36.314496  6125 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:36.481694  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.482111  1626 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:36.462800  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:36.337561  5560 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.351066  5890 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.371609  6070 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:36.343327  5465 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:36.374312  6146 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:36.447865  6086 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.154s	user 0.014s	sys 0.004s
I20260709 13:02:36.351298  6083 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.302029  6080 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.179s
I20260709 13:02:36.376724  6095 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.283753  6149 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.339838  6143 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.148s
I20260709 13:02:36.344274  6106 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 13:02:36.355698  6087 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.552994  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.553861  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.449244  6115 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:36.558132  1525 leader_election.cc:400] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
I20260709 13:02:36.441485  6117 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.449895  6113 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:36.451299  6147 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.590610  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:36.563510  5983 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:36.540141  1701 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:02:36.487824  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 289845us
I20260709 13:02:36.785678  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.489566  1628 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:02:36.733333  2292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.005s	sys 0.031s
W20260709 13:02:36.787613  2292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.007s	sys 0.031s
I20260709 13:02:36.450150  6105 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:36.557197  2296 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:02:36.554680  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 685) took 1365 ms (client timeout 1746 ms). Trace:
I20260709 13:02:36.762630  1525 leader_election.cc:403] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:36.347959  6089 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:36.555158  1560 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:36.790828  1559 rpcz_store.cc:269] 0709 13:02:35.189462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.201276 (+ 11814us) service_pool.cc:224] Handling call
0709 13:02:36.554503 (+1353227us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54000ea454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:36.554621 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180893568}
I20260709 13:02:36.592211  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:36.761513  1705 raft_consensus.cc:2376] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12. Re-sending same reply.
I20260709 13:02:36.792582  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2440) took 1476 ms. Trace:
W20260709 13:02:36.747566  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.009s
W20260709 13:02:36.793550  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.007s	sys 0.010s
W20260709 13:02:36.761551  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.617699  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.591434  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:36.528586  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 686) took 1338 ms. Trace:
I20260709 13:02:36.795643  1632 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:36.662220  2294 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:36.795753  1554 rpcz_store.cc:276] 0709 13:02:35.189691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.245774 (+ 56083us) service_pool.cc:224] Handling call
0709 13:02:35.259176 (+ 13402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.311016 (+1051840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.528342 (+217326us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540036f1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:36.528518 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360573824}
W20260709 13:02:36.759661  1700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.021s	sys 0.017s
I20260709 13:02:36.686203  1634 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:36.548915  6005 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:36.797636  1700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.021s	sys 0.017s
I20260709 13:02:36.564147  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.448535  6071 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:36.749585  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 435763us
W20260709 13:02:36.534875  5887 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.001s
I20260709 13:02:36.783346  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:36.516021  6069 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:36.537420  5834 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:36.527066  5735 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.512984  5912 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:36.785805  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:36.563570  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.530889  6064 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.529897  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1041) took 1210 ms. Trace:
I20260709 13:02:36.822171  2290 rpcz_store.cc:276] 0709 13:02:35.319397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.319562 (+   165us) service_pool.cc:224] Handling call
0709 13:02:36.529707 (+1210145us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540040a954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:36.529816 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173983488}
W20260709 13:02:36.771916  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 227920us
W20260709 13:02:36.823442  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.793946  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 564955us
I20260709 13:02:36.482277  5832 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.327s
W20260709 13:02:36.800217  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 337506us
I20260709 13:02:36.793143  1560 rpcz_store.cc:276] 0709 13:02:35.316239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.316658 (+   419us) service_pool.cc:224] Handling call
0709 13:02:36.554971 (+1238313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.791580 (+236609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.792416 (+   836us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000ea450. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:36.792528 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704365824}
I20260709 13:02:36.836966  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.524574  6141 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.803944  1523 leader_election.cc:400] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
I20260709 13:02:36.530171  5883 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:36.533474  6050 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.546355  6139 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:36.850651  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 397890us
I20260709 13:02:36.529134  6012 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.780597  6117 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.533217  5640 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.539520  6088 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10.
I20260709 13:02:36.777877  6070 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:36.560693  6119 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.228s
I20260709 13:02:36.763520  5890 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:36.793254  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.819070  1455 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.825026  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.851186  6106 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:36.822892  6069 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:36.826537  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:36.571681  6138 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.917526  6138 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.918705  6050 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:36.919418  6166 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:36.790386  6105 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:36.838449  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:36.850805  5883 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.781775  6086 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:02:36.862550  5465 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.584024  6150 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.812698  6071 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:36.536036  6006 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.566676  6104 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.947476  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.823987  6081 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:36.512125  6145 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.789552  6115 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.819988  6170 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.799510  5983 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.545126  5988 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.812909  6005 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:36.557343  6169 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.541240  6055 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.966363  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 135466us
I20260709 13:02:36.967119  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:36.972193  6168 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:36.982081  6086 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:36.985363  5983 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.959861  6145 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:36.987903  6145 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:36.988514  6168 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:36.989465  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 431456us
I20260709 13:02:36.952147  6071 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:36.915757  6165 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:02:36.915953  6070 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.916532  5890 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.823239  6073 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:02:37.053954  5890 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.054296  6073 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.054704  6073 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:37.055557  6073 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.920445  1704 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:36.921756  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:36.825747  6052 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
W20260709 13:02:36.922866  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127861us
W20260709 13:02:36.924961  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98526.5us
I20260709 13:02:37.062196  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.062999  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:36.920065  6105 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.845137  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:36.894371  5761 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.932061  6138 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.932399  6069 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.971040  6169 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.934016  6164 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:02:36.937556  6117 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:36.935457  6139 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:37.094898  6139 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:37.095799  6139 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:36.940877  6122 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.002s
I20260709 13:02:36.926697  5834 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:02:37.059152  5883 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.941886  5465 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:36.944950  6180 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:36.905092  5640 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:36.817817  6123 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:36.921228  6106 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:36.895582  5887 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:02:36.893625  6064 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:02:36.953011  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 134574us
I20260709 13:02:36.911962  6012 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:36.963860  1702 raft_consensus.cc:2368] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13.
I20260709 13:02:36.929971  6141 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.989723  6168 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:36.963193  6170 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.032344  6113 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:37.032508  5735 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:37.044292  6119 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.044592  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.044938  6094 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.045367  6095 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.045473  6081 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.050361  1455 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 11, leader changed from <none> to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.092698  5761 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:36.900442  6147 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:36.941151  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 380669us
W20260709 13:02:37.105809  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 260759us
I20260709 13:02:37.244619  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:37.245810  6012 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.246212  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:37.178325  5560 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.268898  6139 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.115072  6115 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.272758  6115 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.276448  5988 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:37.276870  6012 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.277119  5988 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.277572  6012 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.251693  5761 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.252790  5640 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:37.253530  5883 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.254014  6122 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.914626  1635 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:37.250083  6123 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.240809  6073 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.255973  6117 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.256336  6081 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.241820  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.249200  5832 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:37.241382  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.123144  6071 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.120888  6104 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.287309  6169 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:37.364832  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:37.365543  1634 raft_consensus.cc:2376] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12. Re-sending same reply.
I20260709 13:02:37.264156  1455 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:37.367275  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.366873  1455 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 11, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:37.155418  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 299) took 16243 ms (client timeout 15000 ms). Trace:
I20260709 13:02:37.367940  2296 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
W20260709 13:02:37.138253  6182 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
I20260709 13:02:37.278352  5988 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.249097  6106 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.218946  6105 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.282770  6178 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:37.361183  6055 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:37.284287  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 223495us
I20260709 13:02:37.316471  6012 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.294306  6095 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.380110  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:37.287521  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 295441us
W20260709 13:02:37.288456  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 321408us
I20260709 13:02:37.309438  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 736) took 1116 ms. Trace:
I20260709 13:02:37.327220  6122 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.106506  6138 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:37.296468  6108 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.391561  1709 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:37.349097  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 427392us
I20260709 13:02:37.347654  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:37.392701  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:37.393035  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:37.393617  2288 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.349910  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:37.394152  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2609) took 1220 ms. Trace:
I20260709 13:02:37.394593  1635 rpcz_store.cc:276] 0709 13:02:36.173882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.174172 (+   290us) service_pool.cc:224] Handling call
0709 13:02:37.393942 (+1219770us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54001e8054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:37.394091 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666921344}
W20260709 13:02:37.121663  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.283605  6094 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:37.287016  6119 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.362627  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.363744  2293 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:37.263489  6194 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.364826  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:37.326596  6005 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.369318  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:37.308099  6089 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.370234  1455 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:37.417150  1455 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 9, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:37.372123  2297 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:37.383780  6189 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.362831  6052 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.363010  6143 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:37.283244  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.364773  6087 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.363199  5761 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.294555  5735 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:37.284852  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 469390us
I20260709 13:02:37.380841  1702 rpcz_store.cc:276] 0709 13:02:36.192889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.193380 (+   491us) service_pool.cc:224] Handling call
0709 13:02:37.296226 (+1102846us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400097254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:37.309355 (+ 13129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667646208}
W20260709 13:02:37.382769  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.368141  1456 rpcz_store.cc:269] 0709 13:02:20.911794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.916878 (+  5084us) service_pool.cc:224] Handling call
0709 13:02:37.155289 (+16238411us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:34.744099 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:34.744659 (+   560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:34.744695 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:02:34.744710 (+    15us) tablet.cc:662] Decoding operations
0709 13:02:34.745557 (+   847us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:34.745623 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:34.745650 (+    27us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:34.897688 (+152038us) tablet.cc:701] Row locks acquired
0709 13:02:34.897716 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:02:34.959172 (+ 61456us) write_op.cc:270] Start()
0709 13:02:34.959328 (+   156us) write_op.cc:276] Timestamp: P: 1783602154959108 usec, L: 0
0709 13:02:34.959359 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:02:34.996933 (+ 37574us) log.cc:844] Serialized 2845 byte log entry
0709 13:02:35.391884 (+394951us) op_driver.cc:464] REPLICATION: finished
0709 13:02:35.461828 (+ 69944us) write_op.cc:301] APPLY: starting
0709 13:02:35.467453 (+  5625us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:35.506877 (+ 39424us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:35.506995 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:36.109601 (+602606us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:36.152164 (+ 42563us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:02:36.152213 (+    49us) write_op.cc:312] APPLY: finished
0709 13:02:36.152473 (+   260us) log.cc:844] Serialized 73 byte log entry
0709 13:02:36.152643 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:36.152755 (+   112us) write_op.cc:454] Released schema lock
0709 13:02:36.152797 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:36.153154 (+   357us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:36.153179 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":73988,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":47906,"apply.run_cpu_time_us":13979,"apply.run_wall_time_us":704214,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33415,"num_ops":7,"prepare.queue_time_us":443168,"prepare.run_cpu_time_us":17849,"prepare.run_wall_time_us":417994,"raft.queue_time_us":178877,"raft.run_cpu_time_us":9186,"raft.run_wall_time_us":107603,"replication_time_us":361567,"thread_start_us":566507,"threads_started":4,"wal-append.queue_time_us":125757,"wal-append.run_cpu_time_us":11430,"wal-append.run_wall_time_us":2032377}]]}
W20260709 13:02:37.311482  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.445029  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:37.445955  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:37.416929  6206 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.447779  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.448588  1704 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:37.449121  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.379492  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:37.287587  6115 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.341708  6117 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:37.384404  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.348091  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:37.376854  5988 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.397310  2295 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:37.398229  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:37.386751  6173 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:02:37.404424  6094 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:37.407056  2288 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:37.377513  6195 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.379946  5560 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.391686  6106 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.366498  5465 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.520366  2289 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:37.427013  1455 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 6 to 10, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.427590  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:37.428114  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47278.9us
I20260709 13:02:37.437671  6005 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.385537  6178 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:37.494230  6064 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.391125  6108 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:37.386998  5640 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.391494  5832 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:02:37.394544  6170 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.440135  6143 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:37.474936  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 95503us
I20260709 13:02:37.387481  6055 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:37.447014  6198 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:02:37.458602  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:37.455518  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1077) took 1334 ms. Trace:
I20260709 13:02:37.456195  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:37.457105  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1873) took 1098 ms. Trace:
W20260709 13:02:37.493997  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47851.2us
I20260709 13:02:37.495882  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:37.496136  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:37.555308  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:37.446794  6087 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.554106  1709 rpcz_store.cc:276] 0709 13:02:36.358792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.535119 (+176327us) service_pool.cc:224] Handling call
0709 13:02:36.535402 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.401969 (+866567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.452926 (+ 50957us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54003c1754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:37.457045 (+  4119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968550784}
I20260709 13:02:37.502049  6213 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:37.509259  6105 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:37.518522  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.549088  6145 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.476229  5890 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.440668  6052 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.428207  6185 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:37.466024  6209 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.476311  5834 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.494822  6125 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.614233  5834 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.436002  6086 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:37.494517  6076 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:37.476400  6138 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.537948  6149 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:37.426321  6123 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.510962  6146 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:37.510627  6094 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.502516  6214 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.552553  6143 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.587167  6104 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.586207  6213 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.547195  6150 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:02:37.587132  6168 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:37.544929  6064 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.584055  6141 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:37.625523  6095 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.625861  6210 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:37.439648  6189 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.637418  6210 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.638927  6189 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.625727  6213 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.417956  6194 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.641422  6194 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.472292  6206 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.642067  6070 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:37.642452  6070 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:37.510849  6195 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.645946  6190 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:37.623510  6094 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.548738  6089 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.624336  6214 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.650753  6119 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.620990  6076 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.624197  6197 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.179s
I20260709 13:02:37.652413  6197 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.652731  6197 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.653183  6076 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:37.634469  6122 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.653558  6197 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.621519  6209 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.654774  6197 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.624872  6005 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.626355  6138 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.623251  6146 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.656606  6146 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.624135  6199 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:02:37.648484  6094 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.643256  6070 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.661607  6070 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:37.629101  6178 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.627728  6012 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.669467  6064 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.684839  6185 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:37.685338  6185 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:37.686234  6185 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.685545  6146 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:37.645689  6195 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.690439  6146 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.632565  6173 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:02:37.691895  6168 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:37.692267  6168 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.693135  6168 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.695549  6138 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.687516  6185 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.721805  6143 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.725136  6143 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.863075  6146 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.864424  6146 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.522008  1455 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 12, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 12 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:37.471405  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 91350.5us
I20260709 13:02:37.539848  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.511373  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.778406  1559 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:02:37.939981  1554 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.940164  1559 pending_rounds.cc:85] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:02:37.940732  1559 pending_rounds.cc:107] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:02:37.941128  1559 raft_consensus.cc:2889] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.941576  1559 pending_rounds.cc:107] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:37.941816  1559 raft_consensus.cc:2889] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.942188   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:37.516227  1551 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:37.944996  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:37.947099  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.527546  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.948698  1552 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.512751  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.524811  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:37.951099  1560 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.951180  6146 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.789150  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:37.503441  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 105287us
W20260709 13:02:37.953347  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164748us
W20260709 13:02:37.807601  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.956161  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.627919  5890 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.957633  6229 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.958371  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.958621  6229 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.959066  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:37.524713  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 97187.1us
I20260709 13:02:37.962142  6141 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.962877  6141 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:37.584550  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 126069us
I20260709 13:02:37.965719  6141 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.968621  1557 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:37.969714  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:37.970541  1557 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:37.556752  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:37.972049  1554 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:37.973860  1554 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:37.973778  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 416923us
I20260709 13:02:37.975090  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:37.551899  1703 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.560150  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.610350  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:37.988643  2291 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.991298  6115 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.558693  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.524510  5465 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.616070  6145 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.728564  6213 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:37.842970  1633 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:37.843405  1630 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:02:37.849227  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.852643  2296 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:37.853341  5735 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.623180  6108 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:37.625758  5834 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:37.631124  6104 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:37.928244  6005 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:37.928735  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2494) took 1548 ms. Trace:
I20260709 13:02:37.929124  6190 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:37.930580  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:37.931998  6197 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.937808  1455 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 10, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 10 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:37.971450  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:37.553884  2292 rpcz_store.cc:276] 0709 13:02:36.116470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.116588 (+   118us) service_pool.cc:224] Handling call
0709 13:02:36.157701 (+ 41113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.159389 (+  1688us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:36.181991 (+ 22602us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.288044 (+1106053us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.308863 (+ 20819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.308919 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:37.308988 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.450663 (+141675us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:37.450695 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.450782 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23035,"prepare.queue_time_us":40264,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":62161,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":42048,"spinlock_wait_cycles":3330432,"thread_start_us":1082636,"threads_started":2,"wal-append.queue_time_us":575640}
I20260709 13:02:37.992147  1553 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.995317  6214 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.023288  1551 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.029275  6117 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.032318  6070 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.032564  5890 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.033375  6125 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:37.992327  6138 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:38.037554  6081 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.676s	user 0.003s	sys 0.010s
W20260709 13:02:38.043937  6113 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.009s
W20260709 13:02:38.044677  5988 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.004s
I20260709 13:02:38.043493  6076 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:02:38.050171  6206 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.050521  2291 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:38.052150  6209 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.054343  6087 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.054626  5912 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.056413  6052 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.057298  5983 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:38.059208  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.092511  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 133753us
I20260709 13:02:38.097107  6070 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.069314  6168 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.102414  2295 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:38.057744  6188 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:38.103170  6197 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.091533  6115 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.059530  6195 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.107555  1552 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:38.057881  1635 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:38.107991  6145 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:37.959668  6229 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.109200  5883 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.783s	user 0.004s	sys 0.012s
W20260709 13:02:38.057935  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83177.4us
I20260709 13:02:38.059587  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.112316  1560 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:02:38.108963  1455 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 12, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 12 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.116319  1551 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:38.115366  1553 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:38.115522  6113 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
W20260709 13:02:38.091151  6220 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:02:38.123608  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72239.8us
I20260709 13:02:38.115326  6081 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.101563  6174 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.113359  6138 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:38.071409  6055 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.123901  5912 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:38.093818  1708 rpcz_store.cc:276] 0709 13:02:36.307027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.563905 (+256878us) service_pool.cc:224] Handling call
0709 13:02:37.855373 (+1291468us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400166054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:37.855681 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665131904}
I20260709 13:02:38.126097  5983 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:38.132536  6069 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:38.156376  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.155934  1703 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:38.156262  5890 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:38.163977  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.167531  1557 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:38.115311  6190 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.158535  6138 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:38.175906  6115 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.176054  6125 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.178419  1557 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:38.178514  6115 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.179019  5890 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.185853  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.186362  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:38.188342  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.186870  1627 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:38.189754  6192 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:38.170238  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:38.192710  1559 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:38.192946  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:38.196000  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.155846  6052 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:02:38.221931  1551 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:38.217192  1553 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:38.229727  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.187474  6174 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.234699  6230 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 13:02:38.168085  6005 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.193451  6195 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.258219  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.251294  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 588058us
I20260709 13:02:38.234324  6055 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:38.339022  6213 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:38.339408  6106 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.817s	user 0.014s	sys 0.004s
W20260709 13:02:38.342121  5761 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.895s	user 0.017s	sys 0.000s
I20260709 13:02:38.344817  6229 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.346217  6197 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.348248  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.337200  1633 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:38.219460  6069 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.356190  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:38.365330  6233 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.365855  1455 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:38.367273  6006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.015s	sys 0.026s
I20260709 13:02:38.367575  6012 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:38.367987  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.368700  6178 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:38.368870  6190 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.369240  6210 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:38.369778  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 198674us
I20260709 13:02:38.370352  6147 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:38.389217  6147 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:38.389804  6174 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.390463  6147 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.392100  6147 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.392475  6195 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.370288  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.228749  6192 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.369300  6108 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:38.384653  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 227823us
I20260709 13:02:38.386505  1455 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 13, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 13 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:38.387842  6069 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.388496  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:38.210841  5912 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.380295  6006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.015s	sys 0.026s
I20260709 13:02:38.384342  5890 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.405200  5465 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.404714  1455 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 6 to 9, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:38.407070  6073 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.079s	user 0.009s	sys 0.006s
I20260709 13:02:38.406847  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:38.407589  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.344820  6241 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:38.422204  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 66169.2us
W20260709 13:02:38.424168  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 213662us
I20260709 13:02:38.403257  6210 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:38.426517  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:38.427042  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 206953us
I20260709 13:02:38.427910  2293 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:38.428421  6005 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.428566  6073 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:38.408447  6197 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:02:38.429221  5640 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.429808  6190 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:38.430428  6195 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:38.431411  6081 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.444692  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:38.447408  6069 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:38.448444  5912 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.479266  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:38.480094  1704 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.483840  6073 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:38.485102  6221 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.485028  6073 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.485868  6221 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.474404  5465 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.488245  5465 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.490891  6055 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.448652  2295 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:38.491526  6055 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.492843  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.449203  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.450197  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:38.493594  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49124.1us
I20260709 13:02:38.365378  5983 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.356117  6104 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.492837  2260 leader_election.cc:400] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13.
I20260709 13:02:38.452142  6241 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.452924  6178 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.452905  2290 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:38.455349  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.415915  6229 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:02:38.454368  6138 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.457604  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.457683  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.502805  2294 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:38.459903  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.448709  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.460894  6224 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.461503  6123 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:38.462558  1555 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:38.462899  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:38.463846  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57180.8us
I20260709 13:02:38.465997  6192 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:38.466210  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.506986  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.507480  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.466144  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:38.466557  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.439381  1455 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 8, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.417265  6233 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.476140  1632 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:38.447619  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.451064  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.494689  2260 leader_election.cc:403] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.466328  6012 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:38.513177  6012 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:38.503527  1554 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:38.472187  6190 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:38.502496  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:38.474841  6210 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:38.516142  6190 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:38.520488  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:38.521800  6125 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13.
I20260709 13:02:38.522557  6006 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.523144  6006 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.525300  5988 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.525980  5988 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.532732  6190 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.538378  6012 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.541147  6236 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:02:38.541846  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 76344.5us
I20260709 13:02:38.541567  6224 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.543841  1702 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:38.545181  6178 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.546943  5983 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.552004  6195 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:38.552728  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89986.6us
I20260709 13:02:38.552847  6169 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 13:02:38.553596  5735 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:38.554069  5735 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.554450  5735 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.559046  6070 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.498021  6174 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:38.560074  6174 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.560513  6174 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.560763  1558 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:38.561035  1455 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 11, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.563674  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.564024  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.564703  6123 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:38.559693  6070 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:38.565368  6233 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.565995  6241 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:38.565776  6123 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.567227  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 136293us
I20260709 13:02:38.567483  2293 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:38.567922  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:38.568557  1555 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.476730  5640 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:38.570637  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123159us
I20260709 13:02:38.571559  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:38.572276  6239 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:02:38.572223  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:38.572971  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.574229  6070 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:38.551929  6106 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.575141  6106 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.575608  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.576340  6221 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:38.577445  6106 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.577615  1709 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:02:38.577855  6226 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:02:38.578265  6222 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:38.578682  6222 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:38.579739  6222 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.590516  6222 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.580343  1705 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 15 index: 6. (index mismatch)
I20260709 13:02:38.582770  1706 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:38.596385  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.597174  1702 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:38.562975  1455 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 8, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 8 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:38.553596  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:38.580065  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 876) took 1232 ms. Trace:
I20260709 13:02:38.582902  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:38.588862  1558 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:02:38.601503  1704 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:38.599324  1555 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
W20260709 13:02:38.604478  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50982.5us
I20260709 13:02:38.604712  6178 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:38.605265  6178 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:38.605296  6224 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.606539  6178 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.584628  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:38.605274  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:38.583999  5465 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.587738  5988 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:38.588999  5640 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.609099  6174 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.609905  5640 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:38.610297  5640 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.610386  1635 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:38.610682  5640 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.611299  6064 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:38.612041  6185 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:38.599947  2290 rpcz_store.cc:276] 0709 13:02:37.347623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.347853 (+   230us) service_pool.cc:224] Handling call
0709 13:02:38.579846 (+1231993us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54004c1654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:38.579988 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537596288}
I20260709 13:02:38.586025  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.619127  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 704) took 1074 ms. Trace:
I20260709 13:02:38.607710  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:38.543879  6235 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.142s	user 0.003s	sys 0.000s
W20260709 13:02:38.633217  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47271.6us
I20260709 13:02:38.584492  6192 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:38.626447  1455 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.584543  6070 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:38.640107  1551 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:38.644505  1555 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:38.640839  6233 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:38.665469  1707 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:38.664247  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:38.653342  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.665481  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.659837  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.654814  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.670674  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.654290  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.677065  1554 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:38.664083  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:38.675192  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.678087  1557 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.552752  6195 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.678544  1701 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:38.666208  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1049) took 1079 ms. Trace:
I20260709 13:02:38.655548  6076 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:38.655310  6192 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:38.644014  6081 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:38.618839  6108 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:02:38.654502  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.686215  1559 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:38.641043  6241 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.641096  6005 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:38.687683  1556 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
W20260709 13:02:38.653882  1523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.963s (SENT) [suppressed 18 similar messages]
I20260709 13:02:38.704053  1551 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:38.704288  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.706593  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.708935  1704 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:38.710842  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.711359  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.712278  1709 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:38.712471  6192 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.713049  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.715951  6070 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.714399  1704 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:38.718168  6190 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:38.721176  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 138371us
I20260709 13:02:38.667608  6012 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.724184  6055 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.715682  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:38.678248  6233 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:38.681330  5465 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:38.728293  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.646621  5890 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:38.734498  1455 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:38.731961  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.645139  6221 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:02:38.646414  5761 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:02:38.688411  6081 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:38.740649  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.739725  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.746630  1701 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:38.679128  2297 rpcz_store.cc:276] 0709 13:02:37.586986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.587242 (+   256us) service_pool.cc:224] Handling call
0709 13:02:37.588629 (+  1387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.136043 (+547414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.140987 (+  4944us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.195006 (+ 54019us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.508964 (+313958us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:38.508993 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.555897 (+ 46904us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.665696 (+109799us) raft_consensus.cc:1685] finished
0709 13:02:38.665723 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.665961 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51475,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":606504,"threads_started":2,"wal-append.queue_time_us":424792}
I20260709 13:02:38.744745  1455 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:38.688788  5988 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:02:38.756541  6006 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:38.757344  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.757941  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.758394  6195 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.762629  5735 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.764605  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.743733  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.765846  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:38.766219  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.767127  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.767329  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:38.767529  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.767777  1628 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:38.767170  1635 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:38.768882  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.768550  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.769800  1629 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:38.770262  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.765640  5640 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.767884  1632 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.769740  1630 raft_consensus.cc:2410] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:38.773394  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.765887  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.781080  1633 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:38.737151  6233 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:38.788005  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54167.8us
W20260709 13:02:38.777208  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70755.6us
I20260709 13:02:38.772128  1631 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:38.743234  6081 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.725631  6224 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:38.705484  6076 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.807370  1560 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:02:38.783209  6168 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.001s
I20260709 13:02:38.729264  6241 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.619585  1552 rpcz_store.cc:276] 0709 13:02:37.544899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.173491 (+628592us) service_pool.cc:224] Handling call
0709 13:02:38.230413 (+ 56922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.328625 (+ 98212us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:38.367015 (+ 38390us) log.cc:844] Serialized 14 byte log entry
0709 13:02:38.618787 (+251772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.618839 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:38.618892 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.618951 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.619065 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30717,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":14713,"thread_start_us":236609,"threads_started":1}
I20260709 13:02:38.814762  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.781798  5465 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:38.815578  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.789304  5912 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:38.817051  1455 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 6 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.818334  6197 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:38.852697  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.824374  1556 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:38.823748  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:38.854741  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 239321us
I20260709 13:02:38.743973  6005 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:38.857964  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:38.858646  1627 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.859548  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:38.810820  6174 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.890547  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.861028  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.861325  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.855177  1455 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 11, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:38.861593  6170 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:38.861891  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.862790  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:38.857967  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:38.863850  1707 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.826308  6195 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.865624  1706 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:38.868268  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.872880  6005 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:38.897635  1629 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:02:38.874248  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.874106  6241 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.880604  5465 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:38.881933  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.882344  1627 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:38.818840  6055 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.883473  5640 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.885339  5988 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:38.885885  6064 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:38.887295  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.893038  1557 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:38.956292  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:38.897688  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.898348  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:38.831646  5912 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:38.902385  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 258020us
I20260709 13:02:38.889676  6138 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:38.900039  6170 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.961771  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.962596  2291 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:38.956714  1705 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:38.950980  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.966971  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:38.956610  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:38.940676  6012 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:39.037278  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 179385us
I20260709 13:02:38.968446  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.908320  5761 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.040387  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.037348  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:39.051045  1708 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:39.048625  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.036495  1635 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:39.052307  2296 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:39.053972  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:39.054600  1709 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:39.053797  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:38.951628  5988 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:38.967751  5912 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.065131  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.067909  6213 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:39.068473  6213 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.069672  5640 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:39.070986  6213 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:38.944145  6104 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:02:39.072342  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.072922  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:38.958606  6055 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.076282  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:38.962304  6197 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:02:39.053495  1626 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:39.086242  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.062668  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:38.908691  6221 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 13:02:39.091156  2291 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
W20260709 13:02:39.052265  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 236130us
I20260709 13:02:39.082963  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.092581  1634 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:39.089864  1700 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:39.039868  6229 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.038774  6170 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:39.050797  5761 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.083755  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:39.087021  1630 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:39.097328  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.105368  1560 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:39.105515  1558 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:39.099063  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:38.946779  6195 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:39.101516  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64718.7us
I20260709 13:02:39.112399  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.100886  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.117959  1556 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:39.118808  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.086081  6138 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:39.127584  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.088956  6113 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:39.095363  6214 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:39.133991  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.134805  1560 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:39.136041  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.138634  1626 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:39.136765  1558 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:39.114603  5761 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:39.145718  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67384.6us
I20260709 13:02:39.124876  6190 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:39.122524  6147 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.144330  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:39.107597  6012 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:02:39.133625  6113 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.159107  1552 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:39.134069  6214 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:39.108518  6244 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:02:39.163936  2296 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:39.131690  6213 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:39.114089  6229 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.198239  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.153610  5983 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:39.163820  6113 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.174304  6190 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:39.201148  6222 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.004s
I20260709 13:02:39.196894  6214 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:39.220587  5983 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.265292  5983 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:39.225244  6076 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.220872  6253 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.230801  6213 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.193697  5988 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.259737  6064 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:39.270922  6064 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:39.271036  5988 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.271925  6064 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.161155  6147 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.263990  6244 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:39.275130  6244 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:39.276928  6195 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.281100  6229 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.268409  6190 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.269418  6253 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.283402  1555 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:02:39.220685  6174 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:02:39.258641  1632 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:39.288825  1555 pending_rounds.cc:85] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 6
I20260709 13:02:39.297744  1555 pending_rounds.cc:107] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:39.298013  1555 raft_consensus.cc:2889] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.310289  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:39.283946  6055 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:39.332742  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.333051  6252 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 13:02:39.362298  5983 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.364686  5983 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.322857  6244 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.369331  6244 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:39.371850  6249 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:02:39.373811  6251 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:39.370888  5761 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.377720  6213 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.381199  1552 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:39.382227  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.383358  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.385318  5735 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.387527  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.413142  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.416466  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.417392  1627 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:39.419106  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 258452us
I20260709 13:02:39.428678  1708 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:39.430236  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.433115  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.435565  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.416473  6055 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.440488  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.418886  5761 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.444556  1552 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:02:39.434590  6178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.001s	sys 0.025s
I20260709 13:02:39.467150  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.468035  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.430450  6070 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.473889  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.473124  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:39.473227  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.480796  2288 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:39.474717  2292 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:39.442565  6055 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:39.483829  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:39.482795  6233 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.000s
I20260709 13:02:39.485888  6055 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.490993  6233 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.493197  6233 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.430830  5735 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:39.495930  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:39.496320  6253 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.496502  1553 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:39.497282  6147 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:39.498914  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.499606  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.499967  2293 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:39.500823  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.502117  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:39.436990  6076 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:02:39.502352  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:39.499629  1631 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:39.502406  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:39.476749  6178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.002s	sys 0.026s
I20260709 13:02:39.488907  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.509604  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:39.510084  1632 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:39.529399  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:39.532856  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.536119  1553 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:39.537659  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:39.538416  1553 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:39.539966  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:39.540861  1627 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:39.541046  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.543519  6076 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:39.544435  6073 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:39.544838  6073 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:39.545768  6073 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.547513  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:39.548182  6213 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:39.548686  6213 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:39.549445  6229 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:39.549925  6229 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.549557  6213 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.550257  6229 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:39.547617  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:39.552484  6195 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:39.546517  6055 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.554803  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.555362  6253 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.556278  6005 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:39.478220  6070 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:39.572666  6070 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:39.574515  6259 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:39.560097  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.576174  1555 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:39.577229  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.578467  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.579088  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:39.504935  5640 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:39.580201  5640 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:39.580703  5640 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:39.561039  2292 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:39.442540  6214 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.569029  6174 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:39.570501  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:39.557931  6012 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:39.586341  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.590466  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.547186  6073 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:39.487318  5912 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.618957  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:39.640930  1633 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.015s	sys 0.047s
W20260709 13:02:39.643167  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 744785us
I20260709 13:02:39.586308  6253 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:39.645808  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.008s	sys 0.042s
I20260709 13:02:39.646682  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:39.650882  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:39.653967  1633 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.015s	sys 0.048s
W20260709 13:02:39.654392  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.008s	sys 0.043s
I20260709 13:02:39.683061  6174 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:39.649825  6081 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.832s	user 0.008s	sys 0.005s
I20260709 13:02:39.688969  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.692256  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.689517  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:39.653560  6253 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.716218  1553 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:39.593662  6170 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:39.741880  6270 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:02:39.745560  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53423us
I20260709 13:02:39.746435  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.690634  6104 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.748714  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.744000  6233 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.805447  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 371453us
I20260709 13:02:39.748610  6301 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:39.748471  6272 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.813930  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:39.811941  6113 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 13:02:39.825902  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:39.834758  1553 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:39.810916  6301 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:39.796245  6104 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:39.818444  6277 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:39.818948  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 317362us
I20260709 13:02:39.651883  5735 raft_consensus.cc:2764] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:39.867652  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2508) took 1199 ms. Trace:
W20260709 13:02:39.873710  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46354.2us
I20260709 13:02:39.874550  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:39.888469  1706 rpcz_store.cc:276] 0709 13:02:38.668506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.671264 (+  2758us) service_pool.cc:224] Handling call
0709 13:02:39.867370 (+1196106us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:39.867568 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429971200}
W20260709 13:02:39.891747  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 144106us
I20260709 13:02:39.815572  6285 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:39.931166  1553 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:39.939325  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.691385  6012 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:39.936618  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:39.945124  1559 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.820333  6233 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.951236  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.952365  1553 raft_consensus.cc:2376] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13. Re-sending same reply.
I20260709 13:02:39.939239  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:39.818619  5465 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:02:39.829049  6113 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.697052  6081 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:39.958578  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.951506  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 262099us
W20260709 13:02:39.969743  1523 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.164s	user 0.027s	sys 0.046s
W20260709 13:02:39.970136  1523 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.027s	sys 0.046s
W20260709 13:02:39.970417  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.31659e+06us
I20260709 13:02:39.963953  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:39.892408  6055 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.941586  6197 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.896168  6064 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:39.982287  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:39.901839  6214 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:39.941825  6190 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:39.987095  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:39.823997  6272 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:39.987658  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 98791.8us
W20260709 13:02:39.988864  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 114440us
I20260709 13:02:39.956120  6285 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:40.000550  6295 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:02:40.001108  6174 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:40.003115  1704 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:40.008064  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 319142us
I20260709 13:02:40.008503  1552 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:40.004182  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:39.957683  5912 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:39.962118  6125 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:39.956933  6274 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:40.029155  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:40.018182  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:40.030450  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:40.030556  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:40.037532  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:40.038570  1553 raft_consensus.cc:2410] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:02:40.005528  1523 connection.cc:256] RPC call timeout handler was delayed by -1.194723947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:02:40.007592  6178 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:40.017400  6265 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:02:40.055116  1559 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:40.032188  6125 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:40.029946  6005 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:40.057359  1559 pending_rounds.cc:85] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 4
I20260709 13:02:40.038865  6213 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:40.072559  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54458.1us
I20260709 13:02:40.048362  6178 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.043511  6284 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:40.005770  6197 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.053234  6170 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.115607  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:40.081970  5988 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.810s	user 0.006s	sys 0.007s
I20260709 13:02:40.018988  6190 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:39.984368  6301 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.105399  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:40.132454  1553 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 5. Preceding OpId from leader: term: 15 index: 6. (index mismatch)
I20260709 13:02:40.135957  1559 pending_rounds.cc:107] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:02:40.137030  1559 raft_consensus.cc:2889] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.139130  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:40.118176  6178 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:40.150321  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 119890us
I20260709 13:02:40.151489  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:40.122253  6311 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:40.125308  5912 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:40.137437  1559 pending_rounds.cc:107] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 13:02:40.138827  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 150390us
W20260709 13:02:40.135965  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 105607us
I20260709 13:02:40.200686  1559 raft_consensus.cc:2889] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.204137  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.205938  1635 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:40.207126  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:40.114379  6064 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:02:40.215875  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:40.216336  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:40.216508  1635 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:40.138410  5465 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.120230  6005 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.218865  1701 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:40.066529  6274 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:02:40.238996  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:40.245319  1632 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:40.229125  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.192200  6301 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:40.157069  6311 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.217097  6264 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.769s	user 0.000s	sys 0.005s
W20260709 13:02:40.228256  6303 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:02:40.289124  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.291810  1630 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:40.213889  6297 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:40.211342  6284 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.212355  6214 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:40.261137  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2481) took 1172 ms. Trace:
I20260709 13:02:40.227312  6268 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:40.256601  6244 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.887s	user 0.008s	sys 0.004s
I20260709 13:02:40.229620  6104 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.299485  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.230829  5912 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:40.261128  5640 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.001s
I20260709 13:02:40.313736  1632 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:40.294700  6297 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.302016  1556 rpcz_store.cc:276] 0709 13:02:39.079559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.132473 (+ 52914us) service_pool.cc:224] Handling call
0709 13:02:39.150021 (+ 17548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.421084 (+271063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.421131 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.425375 (+  4244us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.676301 (+250926us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:39.886778 (+210477us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.979180 (+ 92402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.979340 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.251471 (+272131us) raft_consensus.cc:1685] finished
0709 13:02:40.251495 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.251646 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":245393,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":22606,"thread_start_us":744072,"threads_started":4,"wal-append.queue_time_us":311920}
I20260709 13:02:40.304653  6081 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:40.322244  6305 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:02:40.260273  6311 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:40.323086  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.292436  6055 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.323889  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:40.356634  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 230716us
I20260709 13:02:40.252183  5465 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.310166  6268 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.354972  1634 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:40.322964  6297 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:40.328549  6244 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:40.427191  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.428453  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:40.438476  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:40.438706  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68667us
I20260709 13:02:40.339650  6104 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:40.328735  6274 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:40.322276  6322 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:40.455029  1635 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:02:40.455506  1635 pending_rounds.cc:85] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 4
I20260709 13:02:40.455789  1635 pending_rounds.cc:107] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:02:40.322712  6284 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:40.456126  1635 raft_consensus.cc:2889] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.369357  6113 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:40.457795  1554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.017s	sys 0.015s
W20260709 13:02:40.477610  1554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.017s	sys 0.016s
W20260709 13:02:40.386197  6295 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:02:40.341751  6081 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.472983  1635 pending_rounds.cc:107] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:02:40.439599  6326 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.495517  1635 raft_consensus.cc:2889] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.324611  5640 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.401440  6268 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.500653  6309 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:40.501185  6309 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.501639  5640 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.502347  1705 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:40.261235  6125 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.503347  6125 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:40.503670  6309 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.362182  6233 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.370476  6317 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.495069  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2487) took 1181 ms. Trace:
I20260709 13:02:40.387105  6286 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.477123  6316 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.468622  6272 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.504010  6081 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.511327  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.454612  6322 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.369992  6055 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.448107  6104 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.295097  6285 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.535892  1634 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:40.536226  1633 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:40.519582  1554 rpcz_store.cc:276] 0709 13:02:39.313317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.313463 (+   146us) service_pool.cc:224] Handling call
0709 13:02:39.337931 (+ 24468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.338132 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:39.739360 (+401228us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.494690 (+755330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.494745 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:40.494797 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.494879 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.495006 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34017,"prepare.queue_time_us":990365,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":21867,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"thread_start_us":1105085,"threads_started":2}
W20260709 13:02:40.538043  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 225328us
I20260709 13:02:40.563418  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.484485  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:40.454330  6244 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.494000  6274 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:40.563746  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.581833  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1875) took 1088 ms. Trace:
I20260709 13:02:40.582365  1552 rpcz_store.cc:276] 0709 13:02:39.493021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.554516 (+ 61495us) service_pool.cc:224] Handling call
0709 13:02:40.581573 (+1027057us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:40.581764 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988580096}
I20260709 13:02:40.592487  6274 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.561848  6104 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.555410  6322 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:40.599326  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114948us
I20260709 13:02:40.616132  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:40.619338  1634 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:02:40.619645  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.624223  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:40.581537  6190 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.650822  1632 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:40.599829  6310 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.661433  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:40.589710  6233 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:40.665148  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:40.712188  1634 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:40.671140  6306 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
W20260709 13:02:40.687649  6329 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:40.688130  6277 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:40.690928  5761 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.692090  6293 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.692428  1552 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:02:40.704618  6294 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
I20260709 13:02:40.743049  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 729) took 1183 ms. Trace:
I20260709 13:02:40.512804  6326 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.633855  5735 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:40.690845  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.628082  6309 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:40.628707  6275 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.715198  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.662853  6005 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.706730  5465 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:40.736768  6229 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.181s	user 0.015s	sys 0.000s
I20260709 13:02:40.743526  1552 rpcz_store.cc:276] 0709 13:02:39.559856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.663241 (+1103385us) service_pool.cc:224] Handling call
0709 13:02:40.692174 (+ 28933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.742661 (+ 50487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.742975 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.559484  5912 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.696442  6190 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.542881  6316 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.542469  6286 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.772374  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1883) took 2318 ms. Trace:
I20260709 13:02:40.773744  6213 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:40.774276  6213 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:40.775381  6070 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.202s	user 0.013s	sys 0.001s
I20260709 13:02:40.775856  6070 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.773000  1703 rpcz_store.cc:276] 0709 13:02:38.453844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.454031 (+   187us) service_pool.cc:224] Handling call
0709 13:02:38.454363 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.580358 (+125995us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:38.892982 (+312624us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.413765 (+520783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.413838 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:39.413905 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.772026 (+1358121us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003afcd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:40.772096 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.772267 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187266,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":13229,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4447,"spinlock_wait_cycles":289120000,"thread_start_us":765577,"threads_started":2,"wal-append.queue_time_us":775356}
W20260709 13:02:40.777020  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2518) took 2034 ms (client timeout 2037 ms). Trace:
W20260709 13:02:40.777776  1704 rpcz_store.cc:269] 0709 13:02:38.745848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.752013 (+  6165us) service_pool.cc:224] Handling call
0709 13:02:40.776747 (+2024734us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001b6f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:40.776949 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788716928}
I20260709 13:02:40.781237  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2476) took 1706 ms. Trace:
I20260709 13:02:40.782761  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:40.784657  6292 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
I20260709 13:02:40.781824  1557 rpcz_store.cc:276] 0709 13:02:39.074900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.075041 (+   141us) service_pool.cc:224] Handling call
0709 13:02:39.075357 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.126380 (+ 51023us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:39.448402 (+322022us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.760447 (+1312045us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b18004296a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1
0709 13:02:40.762002 (+  1555us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b18004296a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:40.768817 (+  6815us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b18004296a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a
0709 13:02:40.768995 (+   178us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.769538 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.769703 (+   165us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.769739 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.769803 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.781046 (+ 11243us) raft_consensus.cc:1685] finished
0709 13:02:40.781067 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.781179 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18936,"prepare.queue_time_us":775263,"prepare.run_wall_time_us":41,"raft.queue_time_us":194906,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":181,"spinlock_wait_cycles":72535680,"thread_start_us":628620,"threads_started":2,"wal-append.queue_time_us":432763}
I20260709 13:02:40.537398  6317 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.785351  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2462) took 1911 ms. Trace:
W20260709 13:02:40.785983  6312 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 13:02:40.786409  6320 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.787267  6317 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.708599  6113 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:40.758693  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:40.760422  6326 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.788707  6326 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:40.789283  6326 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.791473  6326 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.761063  6314 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:40.795331  6314 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.797822  6314 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.798592  2288 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:40.793763  6213 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.806128  6213 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.737106  6277 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:40.765085  6123 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:40.766193  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:40.770582  6081 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.771581  6244 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.816267  6123 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.784554  2291 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:40.787019  6320 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.817960  6229 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.820060  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1879) took 1325 ms. Trace:
I20260709 13:02:40.820819  6320 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.821595  1555 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16.
I20260709 13:02:40.822916  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 732) took 1262 ms. Trace:
I20260709 13:02:40.827857  6197 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.764331  6190 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.696633  6310 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:40.747448  6302 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.355s	user 0.000s	sys 0.002s
I20260709 13:02:40.785779  1629 rpcz_store.cc:276] 0709 13:02:38.873741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.101312 (+227571us) service_pool.cc:224] Handling call
0709 13:02:39.130519 (+ 29207us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.651867 (+521348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.651899 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.681284 (+ 29385us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.838161 (+156877us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:40.018814 (+180653us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.102231 (+ 83417us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.102454 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.102554 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.785106 (+682552us) raft_consensus.cc:1685] finished
0709 13:02:40.785128 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.785264 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56676,"prepare.queue_time_us":244695,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":114,"raft.queue_time_us":324177,"raft.run_cpu_time_us":525,"raft.run_wall_time_us":80108,"thread_start_us":850946,"threads_started":4,"wal-append.queue_time_us":1061125}
I20260709 13:02:40.823339  1555 rpcz_store.cc:276] 0709 13:02:39.560687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.645929 (+ 85242us) service_pool.cc:224] Handling call
0709 13:02:40.822636 (+1176707us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002f9754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:40.822817 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684954496}
I20260709 13:02:40.743854  6233 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.751950  6293 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:40.729924  2297 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
I20260709 13:02:40.870719  2297 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
W20260709 13:02:40.763499  6278 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:02:40.787393  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2486) took 1643 ms. Trace:
I20260709 13:02:40.784997  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:40.784623  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.878337  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.794776  6318 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.409s	user 0.002s	sys 0.000s
I20260709 13:02:40.882413  6070 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.883576  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 728) took 1983 ms. Trace:
I20260709 13:02:40.884016  1551 rpcz_store.cc:276] 0709 13:02:38.899806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.899973 (+   167us) service_pool.cc:224] Handling call
0709 13:02:38.949010 (+ 49037us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.590083 (+641073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.590138 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.677736 (+ 87598us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.071437 (+393701us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:40.135580 (+ 64143us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.207737 (+ 72157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.765146 (+557409us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400309dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:40.765199 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.794059 (+ 28860us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:40.883345 (+ 89286us) raft_consensus.cc:1685] finished
0709 13:02:40.883371 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.883504 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78744,"prepare.queue_time_us":477243,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":267563,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":20389,"spinlock_wait_cycles":314607232,"thread_start_us":849014,"threads_started":3,"wal-append.queue_time_us":570062}
I20260709 13:02:40.886340  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:40.886981  1551 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.892143  1629 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:40.893972  1629 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:40.894964  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:40.896510  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 730) took 1336 ms. Trace:
I20260709 13:02:40.896888  1551 rpcz_store.cc:276] 0709 13:02:39.560153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.895705 (+1335552us) service_pool.cc:224] Handling call
0709 13:02:40.895988 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.896183 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.896212 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.896257 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:40.896297 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.896351 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.896457 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.897583  6233 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.898213  1551 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:40.898840  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 733) took 1331 ms. Trace:
I20260709 13:02:40.899473  6293 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.895687  6297 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.904018  6310 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.905241  6297 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.906231  6104 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:40.871688  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90395.8us
I20260709 13:02:40.881517  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:40.908056  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2466) took 1411 ms. Trace:
I20260709 13:02:40.908367  1709 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:40.908475  1628 rpcz_store.cc:276] 0709 13:02:39.496660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.500067 (+  3407us) service_pool.cc:224] Handling call
0709 13:02:39.500373 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.500705 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.500732 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.501189 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.847322 (+346133us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:39.847360 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.847453 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.907731 (+1060278us) raft_consensus.cc:1685] finished
0709 13:02:40.907781 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.907977 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60030,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5661,"thread_start_us":345747,"threads_started":1,"wal-append.queue_time_us":799610}
I20260709 13:02:40.912011  6286 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.913051  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:40.759604  6309 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:40.907191  6316 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:40.862964  6301 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.001s
I20260709 13:02:40.865314  1707 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:40.876123  1558 rpcz_store.cc:276] 0709 13:02:39.143915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.148805 (+  4890us) service_pool.cc:224] Handling call
0709 13:02:39.175959 (+ 27154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.592635 (+416676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.592667 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.619883 (+ 27216us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.818184 (+198301us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:39.818240 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.819659 (+  1419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.787167 (+967508us) raft_consensus.cc:1685] finished
0709 13:02:40.787190 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.787334 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67242,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":518231,"threads_started":2,"wal-append.queue_time_us":309279}
I20260709 13:02:40.929210  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:40.858795  6123 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:40.820515  1559 rpcz_store.cc:276] 0709 13:02:39.494186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.056803 (+562617us) service_pool.cc:224] Handling call
0709 13:02:40.057139 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.202303 (+145164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.202337 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.202834 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.355983 (+153149us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.585557 (+229574us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.585767 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.585848 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.819835 (+233987us) raft_consensus.cc:1685] finished
0709 13:02:40.819860 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.819993 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99869,"prepare.queue_time_us":205431,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":40,"raft.queue_time_us":104,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3965,"thread_start_us":232959,"threads_started":2,"wal-append.queue_time_us":470610}
I20260709 13:02:40.931099  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:40.899147  1551 rpcz_store.cc:276] 0709 13:02:39.567924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.897766 (+1329842us) service_pool.cc:224] Handling call
0709 13:02:40.898061 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.898627 (+   566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.898791 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.949653  1552 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:40.945576  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.006s	sys 0.010s
W20260709 13:02:40.950574  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.007s	sys 0.010s
W20260709 13:02:40.945355  6174 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.020s	sys 0.010s
I20260709 13:02:40.961719  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:40.964669  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:40.966464  6147 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:40.967005  6147 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.967456  6147 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:40.970569  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.971932  6293 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.972527  6293 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.974938  6293 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.975230  6320 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.975977  6320 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.976872  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:40.978524  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:40.978744  6320 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:40.979987  6195 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:40.980386  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:40.979525  6320 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:40.975560  6293 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.981168  6195 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.981487  6293 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:40.982678  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:40.983641  6195 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:40.984175  6195 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.984606  6195 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:40.946471  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:40.755892  5761 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:40.988399  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57616.4us
I20260709 13:02:40.962445  1709 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:40.989543  6055 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.991127  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2490) took 1431 ms. Trace:
I20260709 13:02:40.994776  6229 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:40.999027  6268 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:40.916528  6108 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:40.991706  1553 rpcz_store.cc:276] 0709 13:02:39.559268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.205862 (+646594us) service_pool.cc:224] Handling call
0709 13:02:40.208467 (+  2605us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.309941 (+101474us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:40.451954 (+142013us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.695757 (+243803us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.705232 (+  9475us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.705548 (+   316us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.705622 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.705754 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.990821 (+285067us) raft_consensus.cc:1685] finished
0709 13:02:40.990860 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.991050 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105402,"prepare.queue_time_us":237295,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":8997,"raft.queue_time_us":119235,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8425,"thread_start_us":164586,"threads_started":1,"wal-append.queue_time_us":451535}
I20260709 13:02:41.037122  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.004042  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:41.007491  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 735) took 1389 ms. Trace:
I20260709 13:02:41.043900  1559 rpcz_store.cc:276] 0709 13:02:39.618029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.950655 (+1332626us) service_pool.cc:224] Handling call
0709 13:02:41.006826 (+ 56171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.007045 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.007083 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.007138 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:41.007183 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.007251 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.007392 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:41.010581  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 891) took 1578 ms (client timeout 2083 ms). Trace:
W20260709 13:02:41.045506  2297 rpcz_store.cc:269] 0709 13:02:39.431746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.436864 (+  5118us) service_pool.cc:224] Handling call
0709 13:02:41.010328 (+1573464us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400770e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:41.010496 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499373824}
I20260709 13:02:41.046764  1700 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:41.022219  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:41.023241  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:40.963060  6174 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.021s	sys 0.010s
W20260709 13:02:41.082697  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 389178us
W20260709 13:02:41.084699  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 157267us
I20260709 13:02:40.757869  6275 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.096645  1557 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:41.096931  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:41.097616  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2493) took 1346 ms. Trace:
I20260709 13:02:40.985592  6293 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.098133  1557 rpcz_store.cc:276] 0709 13:02:39.751220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.032683 (+1281463us) service_pool.cc:224] Handling call
0709 13:02:41.096195 (+ 63512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.097290 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.097545 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.016815  5983 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:41.104386  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:41.106532  6299 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.406s	user 0.002s	sys 0.000s
I20260709 13:02:41.107602  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1903) took 2141 ms. Trace:
I20260709 13:02:41.108104  1705 rpcz_store.cc:276] 0709 13:02:38.966023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.060854 (+ 94831us) service_pool.cc:224] Handling call
0709 13:02:39.868187 (+807333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.502722 (+634535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.106322 (+603600us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:41.107331 (+  1009us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54001cbbd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:41.107505 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3256175616}
W20260709 13:02:41.109400  6291 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.418s	user 0.001s	sys 0.000s
I20260709 13:02:41.109658  6104 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.085390  6174 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.112942  6174 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:41.113682  1700 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:02:41.114727  6178 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.921s	user 0.009s	sys 0.000s
I20260709 13:02:41.115142  6178 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.022939  6123 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.118266  1627 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:41.121124  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.130456  1634 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:41.132823  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:41.136624  1633 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
W20260709 13:02:41.139643  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84233.8us
W20260709 13:02:41.144385  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107387us
I20260709 13:02:41.149168  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.149837  1700 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:41.150803  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:41.151443  2297 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:41.156948  2263 leader_election.cc:400] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16.
W20260709 13:02:41.157282  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60476.1us
I20260709 13:02:41.157323  2263 leader_election.cc:403] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.030505  1552 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:41.122432  6233 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:41.116779  5983 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.165485  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:41.166941  2291 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:41.167954  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1866) took 1682 ms. Trace:
I20260709 13:02:41.170145  1601 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.140116  6104 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.177345  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:41.211593  6106 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:41.211422  6104 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.212059  6106 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.213097  6106 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.214123  6104 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.177747  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:41.184787  6055 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:41.185477  6178 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.186676  6268 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.138484  6123 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.187939  6190 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.189739  6338 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:41.141616  6340 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:41.167258  6253 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.266s	user 0.008s	sys 0.007s
I20260709 13:02:41.193784  5761 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.194447  6147 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.199460  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.199440  6195 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:41.200410  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.255410  1556 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:41.200852  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.201934  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:41.256148  1557 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:41.202301  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.257062  1552 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:41.203680  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:41.205046  6275 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.166568  6197 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:02:41.159699  6108 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:02:41.210197  5983 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.236021  5465 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:41.215617  6339 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:41.140929  5735 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:41.218859  2295 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:02:41.228729  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.008s	sys 0.028s
I20260709 13:02:41.230365  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:41.319347  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.009s	sys 0.028s
I20260709 13:02:41.236934  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:41.256567  6190 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.168596  6233 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:41.233692  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:41.168488  1560 rpcz_store.cc:276] 0709 13:02:39.485929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.043616 (+557687us) service_pool.cc:224] Handling call
0709 13:02:40.058952 (+ 15336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.118546 (+ 59594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.118581 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.132268 (+ 13687us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.370563 (+238295us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:40.705905 (+335342us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.166116 (+460211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.166218 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.166275 (+    57us) raft_consensus.cc:1685] finished
0709 13:02:41.166290 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.167875 (+  1585us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150963,"prepare.queue_time_us":210090,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":114018,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":2780,"thread_start_us":763069,"threads_started":3,"wal-append.queue_time_us":313453}
W20260709 13:02:41.252589  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.031s	sys 0.019s
I20260709 13:02:41.158594  6125 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:02:41.253535  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:41.258045  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54446.5us
W20260709 13:02:41.326330  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.011s	sys 0.054s
W20260709 13:02:41.326779  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.011s	sys 0.054s
W20260709 13:02:41.236338  6138 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.082s	user 0.003s	sys 0.018s
I20260709 13:02:41.214633  6106 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:41.274353  1631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.000s	sys 0.018s
I20260709 13:02:41.310654  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:41.343432  2297 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:41.314692  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 144665us
I20260709 13:02:41.344835  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:41.241700  6338 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.320560  1633 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:41.238515  2291 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:41.323918  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.031s	sys 0.019s
W20260709 13:02:41.323889  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 90320.3us
I20260709 13:02:41.346886  2291 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:41.317267  6073 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.701s	user 0.007s	sys 0.008s
W20260709 13:02:41.347383  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 686010us
I20260709 13:02:41.347388  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:41.348193  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:41.272642  6309 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:41.345511  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:41.345960  1631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.019s
I20260709 13:02:41.330003  1557 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:41.350901  1560 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:41.321187  6302 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:02:41.350615  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:41.264581  6253 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.365540  1709 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:41.347944  2291 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.351161  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.365114  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.365330  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.375448  2291 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:41.375960  1707 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:41.329547  6138 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.184s	user 0.003s	sys 0.019s
W20260709 13:02:41.377080  2289 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.027s
I20260709 13:02:41.378222  1700 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:02:41.350066  6285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.040s
I20260709 13:02:41.380059  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:41.380422  2289 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.000s	sys 0.027s
I20260709 13:02:41.380286  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.384286  1707 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:41.378718  1556 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 13 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:02:41.352147  6296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.006s	sys 0.019s
I20260709 13:02:41.381481  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:41.356737  5988 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.017s
I20260709 13:02:41.381362  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.344669  6275 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:41.375725  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:41.389307  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:41.381714  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.394281  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.395650  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.346971  6268 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:41.350526  6343 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:41.360644  6178 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.435480  1707 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.429716  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:41.435257  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:41.403668  6296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.006s	sys 0.020s
I20260709 13:02:41.404379  1702 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.350719  6338 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:41.377095  5465 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:41.327272  6233 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.439339  1555 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 9: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:41.447000  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57760.7us
W20260709 13:02:41.447623  6274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.030s	sys 0.076s
W20260709 13:02:41.448175  6274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.030s	sys 0.076s
I20260709 13:02:41.448786  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:41.365042  6309 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:41.450305  6309 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:41.439474  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43913.1us
I20260709 13:02:41.452378  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:41.378064  6310 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:41.401820  6285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.015s	sys 0.041s
I20260709 13:02:41.444912  1709 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:41.318362  6340 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:02:41.445569  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:41.439546  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63903.9us
I20260709 13:02:41.449028  1628 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:41.458386  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:41.459219  1552 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:41.459908  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:41.382736  6073 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.466470  6064 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:41.392331  6125 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.382810  6301 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:41.406414  6277 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.004s
I20260709 13:02:41.445019  6178 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.530109  6338 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.489496  6268 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.532567  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.533210  1560 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:41.501806  5465 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.534677  5465 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.468173  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:41.524511  6277 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.538223  1708 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.539503  6277 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.525516  6123 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.510468  6326 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:41.404158  6275 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:41.466468  6073 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.552212  6326 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.565062  6147 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:41.566382  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 106016us
I20260709 13:02:41.566524  6073 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:41.404496  5988 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.008s	sys 0.017s
I20260709 13:02:41.567175  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.539408  6195 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.568260  6195 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:41.568485  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.570814  6253 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:41.510049  6341 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:02:41.444975  6197 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:41.525293  6301 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:41.604712  1701 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.017s
W20260709 13:02:41.606652  1701 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.009s	sys 0.017s
I20260709 13:02:41.470633  1555 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:02:41.525352  6322 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.006s
I20260709 13:02:41.503098  1556 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:41.468735  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.381716  6138 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:41.519816  6108 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:41.563875  6275 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:41.469596  1557 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:41.565325  6326 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:41.591178  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46666.3us
W20260709 13:02:41.592281  1673 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.924s (SENT)
W20260709 13:02:41.599474  6311 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.154s	user 0.011s	sys 0.001s
W20260709 13:02:41.600309  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 130775us
I20260709 13:02:41.542309  6355 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:41.428393  6293 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:41.602754  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 154054us
I20260709 13:02:41.488407  6274 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:41.606535  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 453337us
I20260709 13:02:41.613001  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:41.647151  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.647954  1700 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.547222  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:41.616830  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47162.6us
I20260709 13:02:41.648666  1709 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.617076  1552 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:41.623627  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.649729  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:41.623744  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:41.445482  6343 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.555831  6125 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.604244  6285 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:41.604027  6197 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:41.650800  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:41.628335  6268 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:41.651659  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:41.666904  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.667999  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:41.668568  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 437127us
I20260709 13:02:41.681947  1706 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.640664  6355 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:41.652643  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60400us
I20260709 13:02:41.488451  5735 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.655542  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:41.620005  6195 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:41.627697  5983 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:41.630371  6214 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:41.620541  6301 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:41.625931  6322 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.646075  6108 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:41.687922  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 448387us
I20260709 13:02:41.690056  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2650) took 1259 ms. Trace:
I20260709 13:02:41.658461  6106 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:02:41.681131  5640 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:41.691596  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.695327  1628 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:41.637491  6354 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:02:41.684028  6346 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:41.638695  6311 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:41.701604  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.692380  5912 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.683593  6338 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:41.683756  6104 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:41.708267  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:41.708537  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:41.696729  1630 rpcz_store.cc:276] 0709 13:02:40.441109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.441333 (+   224us) service_pool.cc:224] Handling call
0709 13:02:40.477777 (+ 36444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.510974 (+ 33197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.511017 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.554776 (+ 43759us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.899103 (+344327us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b48001bc258. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:40.921648 (+ 22545us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:41.424805 (+503157us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.425112 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.425218 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.685670 (+260452us) raft_consensus.cc:1685] finished
0709 13:02:41.689834 (+  4164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.689992 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25859,"prepare.queue_time_us":71646,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":323343,"raft.run_cpu_time_us":1487,"raft.run_wall_time_us":151611,"spinlock_wait_cycles":105066752,"thread_start_us":808034,"threads_started":2,"wal-append.queue_time_us":586482}
I20260709 13:02:41.802060  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:41.849480  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 140989us
W20260709 13:02:41.710417  1705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.009s	sys 0.016s
W20260709 13:02:41.851231  1705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.009s	sys 0.018s
I20260709 13:02:41.689004  6355 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:41.868909  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:41.719502  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.873971  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 165776us
I20260709 13:02:41.879065  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:41.699643  6178 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:41.803689  6341 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:41.849246  6356 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.884793  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.884527  6108 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:41.885932  6357 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:02:41.804170  6070 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:41.707412  6310 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:41.707135  5640 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.892422  1551 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:41.711205  6277 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:41.896032  6360 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:02:41.896924  6326 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:41.898126  2261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.013s
W20260709 13:02:41.949009  2261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.009s	sys 0.014s
I20260709 13:02:41.873744  6174 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:41.951057  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.906962  1553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.009s	sys 0.016s
I20260709 13:02:41.909734  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.734292  6320 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.157s
I20260709 13:02:41.927114  6070 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:41.954723  6320 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:41.955134  6320 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.955379  6070 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.955981  6320 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.956820  6070 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:41.957278  6320 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:41.959184  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2491) took 2399 ms. Trace:
W20260709 13:02:41.910552  2292 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.010s	sys 0.014s
I20260709 13:02:41.886484  6274 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:41.959733  2292 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.010s	sys 0.014s
W20260709 13:02:41.961395  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.925166  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45982.6us
W20260709 13:02:41.952134  1553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.009s	sys 0.016s
I20260709 13:02:41.963667  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:41.963884  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 736) took 1265 ms. Trace:
I20260709 13:02:41.709996  6197 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.949713  6310 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.959587  1554 rpcz_store.cc:276] 0709 13:02:39.559562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.606780 (+1047218us) service_pool.cc:224] Handling call
0709 13:02:40.652108 (+ 45328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.795798 (+143690us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:40.796130 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.253303 (+457173us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.266833 (+ 13530us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.267096 (+   263us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.267156 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.376850 (+109694us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54003fb954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:41.376906 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.958871 (+581965us) raft_consensus.cc:1685] finished
0709 13:02:41.958983 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.959124 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3682,"prepare.queue_time_us":1835,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":200,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":17626,"spinlock_wait_cycles":329300864,"thread_start_us":456531,"threads_started":1,"wal-append.queue_time_us":808720}
I20260709 13:02:41.980736  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:41.931422  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.017s	sys 0.018s
W20260709 13:02:41.981424  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.018s	sys 0.019s
W20260709 13:02:41.931267  2289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.015s
I20260709 13:02:41.877372  5912 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:41.926386  6106 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:41.989486  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2655) took 1378 ms. Trace:
I20260709 13:02:41.949828  5640 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:41.706482  6138 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:41.981979  2289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.016s
I20260709 13:02:41.950189  6285 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.959968  6274 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.992112  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:42.005146  1701 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:41.995298  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:41.997040  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1065) took 1215 ms. Trace:
I20260709 13:02:41.999325  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:42.007957  1706 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:41.960235  6343 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.003296  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:42.009227  1552 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 10: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:41.989899  1626 rpcz_store.cc:276] 0709 13:02:40.611072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.693291 (+ 82219us) service_pool.cc:224] Handling call
0709 13:02:40.709472 (+ 16181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.709730 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:41.230729 (+520999us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.987735 (+757006us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.987812 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:41.987867 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.987935 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.989427 (+  1492us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5779,"prepare.queue_time_us":283381,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"thread_start_us":1238702,"threads_started":2}
I20260709 13:02:42.005952  2289 rpcz_store.cc:276] 0709 13:02:40.781166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.782992 (+  1826us) service_pool.cc:224] Handling call
0709 13:02:40.783323 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.783554 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:41.402840 (+619286us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.996630 (+593790us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.996681 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:41.996733 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.996800 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.996980 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":874699,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":1193871,"threads_started":2}
I20260709 13:02:42.010222  1709 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:41.979681  6125 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:41.986682  6174 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:41.989810  6106 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.038249  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2492) took 2441 ms. Trace:
I20260709 13:02:42.035593  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.010735  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:41.961081  6277 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:41.927176  6356 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:42.057600  1700 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:42.006654  6274 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:42.059407  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.089213  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.104725  1552 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:42.104851  1709 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
W20260709 13:02:42.087702  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 413350us
W20260709 13:02:42.014854  5761 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.021s	sys 0.011s
I20260709 13:02:42.107079  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.056522  1551 rpcz_store.cc:276] 0709 13:02:39.596781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.949789 (+1353008us) service_pool.cc:224] Handling call
0709 13:02:40.950218 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.945961 (+995743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.037939 (+ 91978us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54003098d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:42.038184 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166158592}
I20260709 13:02:42.030719  6147 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:42.036692  6309 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.150179  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.151396  1708 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:42.117051  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.155911  1551 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:42.157184  1701 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:42.037655  6372 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.061296  6106 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:42.103531  6331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.004s	sys 0.025s
I20260709 13:02:42.126030  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.167040  1552 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:42.073599  6195 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.114524  5761 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.022s	sys 0.013s
I20260709 13:02:42.075446  6322 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.111943  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:42.118525  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 435898us
I20260709 13:02:41.964212  1553 rpcz_store.cc:276] 0709 13:02:40.698502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.085305 (+386803us) service_pool.cc:224] Handling call
0709 13:02:41.085637 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.085838 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:41.118432 (+ 32594us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.209043 (+ 90611us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b180037dd40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:02:41.963534 (+754491us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.963585 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:41.963625 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.963693 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.963830 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32426,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3126,"raft.queue_time_us":1772,"spinlock_wait_cycles":26045696,"thread_start_us":754259,"threads_started":1}
I20260709 13:02:42.152379  6371 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:42.171629  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:42.132290  6346 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:42.175505  6195 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.136376  6381 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:42.169515  6331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.005s	sys 0.025s
I20260709 13:02:42.163128  6311 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:42.177583  6073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.008s	sys 0.016s
W20260709 13:02:42.219825  1628 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.003s
I20260709 13:02:42.223337  1628 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:42.166395  6214 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:42.004776  6197 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:42.225178  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1921) took 1052 ms. Trace:
I20260709 13:02:42.233775  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:42.225117  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.226059  1707 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:42.159825  6309 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.129652  6174 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.234155  1705 rpcz_store.cc:276] 0709 13:02:41.172889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.173055 (+   166us) service_pool.cc:224] Handling call
0709 13:02:41.187691 (+ 14636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.188070 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:41.851857 (+663787us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.224761 (+372904us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.224828 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.224877 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.224976 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.225120 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321913,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":1047,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":993172,"threads_started":2}
I20260709 13:02:42.220432  6347 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:42.238332  6214 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:42.168912  6372 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.006803  6285 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.271426  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.241789  6310 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.188164  6346 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.295758  1554 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:42.222290  6381 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.303074  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.301818  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1074) took 1067 ms. Trace:
I20260709 13:02:42.306298  2291 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:42.298900  6366 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:02:42.305024  2292 rpcz_store.cc:276] 0709 13:02:41.234864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.234996 (+   132us) service_pool.cc:224] Handling call
0709 13:02:41.272927 (+ 37931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.273242 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:41.961220 (+687978us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.301467 (+340247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.301542 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:42.301603 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.301670 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.301773 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372550,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"thread_start_us":940789,"threads_started":2}
W20260709 13:02:42.304598  6369 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 13:02:42.327994  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 212677us
I20260709 13:02:42.299146  6331 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.300585  6322 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:42.309305  6370 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:02:42.309201  6123 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.341877  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.345645  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:42.276350  6347 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.336541  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.294315  6388 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:02:42.330386  5912 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.372664  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:42.292577  6389 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:02:42.333196  6277 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:42.342154  6379 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
I20260709 13:02:42.367965  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.362634  1706 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:02:42.374650  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 203105us
W20260709 13:02:42.354300  6374 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:02:42.399648  6390 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:42.401232  6372 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.389580  6371 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:42.404980  6233 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.904s	user 0.010s	sys 0.001s
I20260709 13:02:42.404906  6371 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.405521  6233 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.403239  6214 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.340835  6331 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.409843  6331 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.410426  6331 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:42.410938  6331 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.411486  6214 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:42.412108  6383 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:02:42.376111  1709 raft_consensus.cc:2368] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11.
I20260709 13:02:42.343739  6326 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:42.382308  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.396775  6392 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:42.398655  6195 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.412180  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1910) took 1620 ms. Trace:
I20260709 13:02:42.420099  1704 rpcz_store.cc:276] 0709 13:02:40.791564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.879707 (+ 88143us) service_pool.cc:224] Handling call
0709 13:02:40.921970 (+ 42263us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.933295 (+ 11325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.933345 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:41.017056 (+ 83711us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.605826 (+588770us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:41.605874 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.605942 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.410067 (+804125us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b24001d4878. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:02:42.411855 (+  1788us) raft_consensus.cc:1685] finished
0709 13:02:42.411882 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.412113 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8380,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11237,"spinlock_wait_cycles":22808960,"thread_start_us":485245,"threads_started":1,"wal-append.queue_time_us":811761}
I20260709 13:02:42.340221  6322 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.397096  6346 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.418452  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.407205  6274 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.387207  6123 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:42.434455  1709 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:42.440922  6361 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.560s	user 0.004s	sys 0.000s
I20260709 13:02:42.443112  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.445180  1702 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:42.450096  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82222.1us
I20260709 13:02:42.450996  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.452133  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.454597  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:42.436431  6355 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.003s
I20260709 13:02:42.461201  6347 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.467691  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.473055  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.473338  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.473723  2293 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:42.474071  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.474659  2295 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:42.473117  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.473659  6407 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:42.476004  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.476023  6407 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.476632  6407 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:42.480717  6405 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.481280  6405 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:42.483247  6382 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.307s	user 0.001s	sys 0.002s
W20260709 13:02:42.483994  6391 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:02:42.485168  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2482) took 1112 ms. Trace:
I20260709 13:02:42.485641  1633 rpcz_store.cc:276] 0709 13:02:41.372449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.372659 (+   210us) service_pool.cc:224] Handling call
0709 13:02:41.408073 (+ 35414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.434878 (+ 26805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.434911 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.440235 (+  5324us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.007052 (+566817us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:42.007092 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.007162 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.482538 (+475376us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:42.484830 (+  2292us) raft_consensus.cc:1685] finished
0709 13:02:42.484856 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.485097 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":12231,"spinlock_wait_cycles":12904704,"thread_start_us":466949,"threads_started":1,"wal-append.queue_time_us":636210}
W20260709 13:02:42.489861  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.490900  6070 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.002s
W20260709 13:02:42.493561  6394 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:02:42.444613  6285 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.468134  5912 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:42.504925  6147 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:42.507319  5912 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.507706  5912 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:42.461508  5988 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:42.506671  6285 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.512471  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.513185  1554 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:42.392792  5465 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:02:42.514353  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.514431  5465 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:42.514910  5465 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:42.515651  5465 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.518546  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:42.519502  6320 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.004s
I20260709 13:02:42.519748  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.537796  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:42.539734  2290 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:42.540074  1633 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:42.520694  6373 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.441s	user 0.000s	sys 0.001s
I20260709 13:02:42.521106  1556 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:42.524210  6309 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.525076  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:42.544925  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.551175  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:42.551808  1635 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:42.526683  6322 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.512987  6285 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:42.442081  6274 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.528987  6331 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:42.531318  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.566875  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.531723  6343 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.532469  6268 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:42.567699  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:42.570616  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.571219  1632 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:42.572100  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.572657  1628 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:42.536469  1630 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:42.536731  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 758) took 1070 ms. Trace:
I20260709 13:02:42.537068  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.576975  1626 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
W20260709 13:02:42.538034  6380 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 13:02:42.518757  6301 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.003s
I20260709 13:02:42.520242  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.425261  6233 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:42.487278  5983 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.000s
W20260709 13:02:42.519353  6367 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 13:02:42.546717  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.551671  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.636771  6285 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:42.637198  1629 raft_consensus.cc:2410] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:42.527383  2291 raft_consensus.cc:2410] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:42.543366  2292 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:42.563697  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.564381  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:42.567040  1553 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
W20260709 13:02:42.532591  6275 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.894s	user 0.012s	sys 0.000s
I20260709 13:02:42.428869  6402 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 13:02:42.457311  6123 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:42.507676  6147 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:42.569118  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:42.578686  1635 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:42.433907  6326 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:42.576110  1707 rpcz_store.cc:276] 0709 13:02:41.466446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.481752 (+ 15306us) service_pool.cc:224] Handling call
0709 13:02:42.225820 (+744068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.236568 (+ 10748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.536552 (+299984us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400042c50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:42.536677 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710127488}
I20260709 13:02:42.561062  6070 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:02:42.610828  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.692385  1631 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:42.616030  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.613511  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.590898  6356 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:43.084638  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.626448  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.632236  1559 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:42.632090  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:42.633255  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.630903  6238 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:02:42.635185  6106 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:42.598467  5465 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:42.740531  1703 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:42.637598  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:43.096300  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.096634  1552 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.097476  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.098019  2293 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:43.098773  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.746949  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:42.942176  6253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.007s	sys 0.007s
W20260709 13:02:43.100126  6253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.008s	sys 0.008s
I20260709 13:02:42.646366  6405 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:42.919871  6377 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:43.101115  6377 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:43.101394  6106 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.101922  6377 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.103129  6377 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:42.806463  6190 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.478s	user 0.015s	sys 0.000s
I20260709 13:02:43.104099  6190 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16.
I20260709 13:02:42.647421  6301 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.074126  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.630613  6389 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:02:43.106887  6301 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.075017  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:42.963964  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.007s	sys 0.024s
W20260709 13:02:43.107491  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.008s	sys 0.025s
I20260709 13:02:42.581408  6073 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:43.039848  6378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.010s	sys 0.024s
I20260709 13:02:43.107780  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.108145  6073 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.908269  5640 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
I20260709 13:02:43.083988  6381 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:43.076095  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.076633  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 512303us
I20260709 13:02:42.566216  6352 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:43.108532  6073 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.115124  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:43.108212  6378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.010s	sys 0.026s
I20260709 13:02:43.115916  6316 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:43.116423  6316 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:43.116789  6316 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.099704  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:42.836226  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.118948  2293 raft_consensus.cc:2376] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 10. Re-sending same reply.
I20260709 13:02:42.538534  6355 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:42.985364  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 404486us
I20260709 13:02:43.014595  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:42.935529  1523 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.963s (SENT)
I20260709 13:02:43.123624  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.078931  2295 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.079909  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.124708  1557 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:42.618685  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:42.737064  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.125680  1704 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:42.638113  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.685815  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:42.647943  1702 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:02:43.019523  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 411012us
I20260709 13:02:42.625088  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:42.658535  2260 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.695s (SENT)
I20260709 13:02:42.883936  6309 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.094626  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.094820  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.140409  1556 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:43.104918  1552 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:43.094475  6275 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.118641  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:43.145037  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 209557us
I20260709 13:02:42.995604  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.147291  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:43.119405  1705 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:43.119407  1553 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:43.110544  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:42.503520  6407 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.108964  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.149055  2289 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:43.149262  2290 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:43.120851  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.150533  1708 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:43.122044  1633 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.002s
W20260709 13:02:43.128273  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 469760us
I20260709 13:02:43.084230  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:43.109834  6238 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:43.159327  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.129464  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.112457  6352 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:43.170126  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.138525  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.093904  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.172185  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.143298  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.172816  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.132243  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.154556  1556 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:02:43.155182  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78321.3us
I20260709 13:02:43.127511  2288 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:43.093711  6214 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:43.133744  6311 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:43.151505  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.177969  1708 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.157356  1553 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:43.157487  1633 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:43.132762  5912 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.003s
I20260709 13:02:43.130766  1555 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:43.134089  6285 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:43.157078  2290 pending_rounds.cc:85] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 10
I20260709 13:02:43.155673  6106 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.152745  6301 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:42.580915  6343 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.155910  6381 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.109496  5640 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.174494  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.170918  1705 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 12. Re-sending same reply.
I20260709 13:02:43.225512  2289 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.226004  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 756) took 1789 ms. Trace:
I20260709 13:02:43.226454  1703 rpcz_store.cc:276] 0709 13:02:41.436566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.437671 (+  1105us) service_pool.cc:224] Handling call
0709 13:02:41.465165 (+ 27494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.105466 (+1640301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.225764 (+120298us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:43.225946 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2932502784}
I20260709 13:02:43.204241  2290 pending_rounds.cc:107] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:02:43.205955  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.158869  6238 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:43.230525  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.224308  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.234762  1704 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:43.235826  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.188932  1556 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 13:02:43.165686  6388 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:43.227470  2290 raft_consensus.cc:2889] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:43.177697  5735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.009s	sys 0.021s
I20260709 13:02:43.243181  1708 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:02:43.153606  6309 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.256458  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.267647  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.205436  6356 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.182266  6310 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:43.278847  6310 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:43.277678  1701 raft_consensus.cc:2368] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13.
I20260709 13:02:43.267627  2293 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.273766  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.282631  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:43.227550  6214 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:43.287659  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.278831  6356 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:43.271473  5735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.010s	sys 0.021s
I20260709 13:02:43.288306  2292 raft_consensus.cc:2376] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12. Re-sending same reply.
I20260709 13:02:43.192618  6311 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.284139  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.250969  6340 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.279855  6310 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.292042  1702 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.267856  6388 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.294950  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 142282us
I20260709 13:02:43.167976  1631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:43.305750  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.306017  1631 raft_consensus.cc:606] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:43.216301  5912 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:43.300107  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 145772us
I20260709 13:02:43.300258  1558 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:43.223649  6106 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:43.300691  1523 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.539s (SENT)
I20260709 13:02:43.310232  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:43.288722  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.306564  2292 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:43.232374  6301 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.250537  6381 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.310925  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:43.312965  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.318717  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.329030  6355 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.330241  6073 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.331496  1708 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:43.332944  5640 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.256744  6123 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.334695  6301 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.336413  6275 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.337278  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.308610  6388 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:43.354548  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53883.3us
W20260709 13:02:43.354502  2260 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.873s (SENT)
W20260709 13:02:43.359697  2260 leader_election.cc:336] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.894s (SENT)
I20260709 13:02:43.361995  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:43.309657  6268 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:43.360200  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:43.364084  1523 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.574s (SENT)
I20260709 13:02:43.387745  2292 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:02:43.388505  2260 leader_election.cc:336] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.918s (SENT)
I20260709 13:02:43.395308  6356 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:43.397858  1523 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.583s (SENT)
I20260709 13:02:43.307332  6340 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.399106  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.398365  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.356592  6277 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:43.403501  1703 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:43.405287  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:43.407064  6417 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:02:43.410032  6428 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:02:43.410408  6413 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:02:43.306821  6352 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.420866  6340 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.421175  5912 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.422478  6106 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.364179  6431 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:43.426851  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:43.413717  6277 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:43.464644  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66813.6us
W20260709 13:02:43.465529  1523 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.611s (SENT)
I20260709 13:02:43.465708  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:43.466524  1523 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.628s (SENT)
I20260709 13:02:43.467043  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.468014  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.468798  1635 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:43.414011  6322 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:43.396733  6407 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.415652  6311 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:43.392719  6389 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.393201  6233 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:43.493146  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.450198  2288 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:43.413017  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:43.461804  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.494566  1633 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:43.466526  1707 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:43.400507  6309 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.423719  6431 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:43.486723  1673 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.083s (SENT)
W20260709 13:02:43.489096  1523 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.796s (SENT)
I20260709 13:02:43.489393  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.582767  1626 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:43.490406  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:43.583331  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 96660us
I20260709 13:02:43.404803  6297 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:43.491609  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.585299  1631 raft_consensus.cc:2410] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:43.493023  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.493155  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.585620  1700 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:43.585947  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:43.586532  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 467926us
I20260709 13:02:43.496860  2292 raft_consensus.cc:2368] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:02:43.501255  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.501255  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.537712  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.397157  6268 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:43.538403  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.537985  2260 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.982s (SENT)
I20260709 13:02:43.610710  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1105) took 1156 ms. Trace:
I20260709 13:02:43.613287  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:43.616129  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.617034  2295 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:43.617621  2289 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:02:43.573073  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.487368  5735 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:43.314100  6238 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.402571  6214 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.586786  1633 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:43.428755  1629 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:43.570866  6012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.001s	sys 0.020s
I20260709 13:02:43.588074  2294 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:02:43.631076  1706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.011s	sys 0.022s
W20260709 13:02:43.632264  1706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.022s
W20260709 13:02:43.630216  1524 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.531s (SENT)
W20260709 13:02:43.589818  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 100755us
I20260709 13:02:43.590770  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:43.592597  1628 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
W20260709 13:02:43.593974  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 458150us
I20260709 13:02:43.593632  1707 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:43.592780  6343 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.593343  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:43.575675  6073 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:43.618257  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.611102  2291 rpcz_store.cc:276] 0709 13:02:42.454168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.454324 (+   156us) service_pool.cc:224] Handling call
0709 13:02:43.610533 (+1156209us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b540027f454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:43.610659 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165329664}
I20260709 13:02:43.644969  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.500072  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:43.646027  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 146079us
I20260709 13:02:43.595110  6297 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:43.641973  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:43.619251  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.622283  2296 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:43.504316  6389 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.623483  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1113) took 1053 ms. Trace:
W20260709 13:02:43.623890  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83928.1us
W20260709 13:02:43.654227  2260 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.195s (SENT)
I20260709 13:02:43.654807  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.623909  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 734) took 4021 ms. Trace:
I20260709 13:02:43.626593  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.627700  1631 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:43.655654  1558 rpcz_store.cc:276] 0709 13:02:39.587874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.928280 (+1340406us) service_pool.cc:224] Handling call
0709 13:02:40.963248 (+ 34968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.309962 (+2346714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.608996 (+299034us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b540036ecd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:43.609218 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5374151936}
W20260709 13:02:43.636755  1523 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.836s (SENT)
I20260709 13:02:43.657128  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:43.660076  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.661361  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.661553  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.662122  1700 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 10: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:43.662312  1702 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
I20260709 13:02:43.662874  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.663272  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:43.637172  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46534.2us
I20260709 13:02:43.665746  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.666510  2296 raft_consensus.cc:2368] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:02:43.598331  1632 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:43.651288  6356 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:43.668329  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2522) took 1501 ms (client timeout 1531 ms). Trace:
W20260709 13:02:43.668738  1701 rpcz_store.cc:269] 0709 13:02:42.166805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.233434 (+ 66629us) service_pool.cc:224] Handling call
0709 13:02:43.668149 (+1434715us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001b7454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:43.668274 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2398184320}
I20260709 13:02:43.669902  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.670444  6389 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:43.648821  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.650595  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.651623  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:43.630882  6012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.002s	sys 0.020s
W20260709 13:02:43.652468  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59232.9us
I20260709 13:02:43.639235  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:43.620712  6326 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.676178  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:43.663476  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:43.653959  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:43.678472  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:43.681442  2260 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.114s (SENT)
W20260709 13:02:43.691319  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52154.2us
I20260709 13:02:43.667510  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.670948  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.698369  1701 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:02:43.699054  2260 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.142s (SENT) [suppressed 50 similar messages]
I20260709 13:02:43.654248  2295 rpcz_store.cc:276] 0709 13:02:42.570475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.610454 (+ 39979us) service_pool.cc:224] Handling call
0709 13:02:43.623400 (+1012946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:43.700925  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.702392  6381 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.702661  1633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:43.703263  1633 raft_consensus.cc:606] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
W20260709 13:02:43.704041  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50141us
I20260709 13:02:43.706033  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:43.707906  6238 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.707984  6285 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.708639  6285 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.709040  6285 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.698617  6375 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:43.712064  6350 raft_consensus.cc:2764] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:02:43.712348  6375 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:43.713428  6375 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.714536  5912 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:02:43.715322  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1893) took 1181 ms. Trace:
I20260709 13:02:43.715826  1552 rpcz_store.cc:276] 0709 13:02:42.533502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.141248 (+607746us) service_pool.cc:224] Handling call
0709 13:02:43.141618 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.142052 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:43.142093 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.142628 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.578140 (+435512us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:43.578198 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.578289 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.715033 (+136744us) raft_consensus.cc:1685] finished
0709 13:02:43.715060 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.715250 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":818,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":18175,"thread_start_us":399530,"threads_started":1,"wal-append.queue_time_us":552589}
I20260709 13:02:43.719065  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:43.720088  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.720744  2289 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:43.721038  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:43.721639  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.722816  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.723336  2289 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.724121  5465 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.724340  1631 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:43.703598  1633 raft_consensus.cc:614] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:43.698014  6311 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:43.782228  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 145484us
I20260709 13:02:43.690699  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:43.784086  1523 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.531s (SENT)
I20260709 13:02:43.784735  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:43.707535  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:43.780893  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.785890  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:43.787503  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61322.4us
W20260709 13:02:43.788195  6429 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:02:43.788679  6012 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:43.790123  6431 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.651433  6233 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:43.692615  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:43.812243  2291 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:02:43.841068  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.840678  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:43.791664  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.844154  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:43.793892  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 86446.6us
I20260709 13:02:43.852206  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.852564  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.840184  6435 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:02:43.690111  6178 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:02:43.696332  6073 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.157s
I20260709 13:02:43.911192  6178 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:43.911589  6178 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:43.912282  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.913026  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:43.839890  6233 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:43.914443  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.915041  2291 raft_consensus.cc:2376] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9. Re-sending same reply.
I20260709 13:02:43.915624  1556 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:43.915921  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.916795  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:43.916939  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:43.917219  6414 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:43.917425  2291 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.917747  6414 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:43.918291  6414 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.919870  6233 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:43.920002  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2518) took 1392 ms. Trace:
I20260709 13:02:43.920472  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:43.837369  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53319.7us
I20260709 13:02:43.920368  1554 rpcz_store.cc:276] 0709 13:02:42.527957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.110876 (+582919us) service_pool.cc:224] Handling call
0709 13:02:43.111317 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:43.111830 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.111872 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:43.112615 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 13:02:43.591508 (+478893us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:43.629498 (+ 37990us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.629986 (+   488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.630083 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.919753 (+289670us) raft_consensus.cc:1685] finished
0709 13:02:43.919789 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.919932 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62452,"prepare.queue_time_us":37616,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":1717,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":9447,"thread_start_us":478391,"threads_started":1,"wal-append.queue_time_us":804985}
I20260709 13:02:43.792661  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.924556  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:43.925282  6434 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:43.912462  6178 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:43.925372  6297 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.926862  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.724633  5465 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.927815  1554 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:43.910135  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.913610  1560 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.928836  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.929562  1554 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:43.929827  5465 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:43.917520  1556 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:43.931478  1552 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:02:43.932415  6411 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.048s	user 0.010s	sys 0.000s
I20260709 13:02:43.934207  6311 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:43.696569  6407 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:43.936836  6178 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:43.936754  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.937304  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.937915  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:43.938768  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.940434  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1084) took 1423 ms. Trace:
I20260709 13:02:43.941529  1708 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:02:43.942098  1708 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 7
I20260709 13:02:43.942386  1708 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:02:43.942696  1708 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.852785  6343 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:43.944047  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.945240  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:43.945613  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:43.945969  1633 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:43.947031  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.946770  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:43.947615  2288 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:02:43.948470  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1108) took 1492 ms (client timeout 1874 ms). Trace:
W20260709 13:02:43.949051  2294 rpcz_store.cc:269] 0709 13:02:42.456364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.456586 (+   222us) service_pool.cc:224] Handling call
0709 13:02:43.948309 (+1491723us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54005d02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:43.948419 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444409600}
W20260709 13:02:43.951013  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2495) took 1405 ms (client timeout 1860 ms). Trace:
I20260709 13:02:43.952029  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:43.948709  2288 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:43.944244  1558 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:43.898003  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:43.876343  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:43.692687  6418 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:02:43.690393  6277 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:44.007251  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.940888  2290 rpcz_store.cc:276] 0709 13:02:42.516823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.156558 (+639735us) service_pool.cc:224] Handling call
0709 13:02:43.156864 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.249149 (+ 92285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.249198 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.271233 (+ 22035us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.588047 (+316814us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:43.588105 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.588199 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.940204 (+352005us) raft_consensus.cc:1685] finished
0709 13:02:43.940229 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.940368 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":316502,"threads_started":1,"wal-append.queue_time_us":584276}
I20260709 13:02:43.826380  6423 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.965085  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 55053.6us
I20260709 13:02:44.023669  1560 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:43.948489  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:43.791832  5735 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:43.842938  6123 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.188s
I20260709 13:02:44.010928  2289 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:43.792006  6389 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:02:43.840425  6350 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:43.849742  6377 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.746s	user 0.002s	sys 0.010s
W20260709 13:02:44.012950  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221355us
I20260709 13:02:44.013768  1704 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:02:44.015162  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
W20260709 13:02:44.016646  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.035s	sys 0.044s
I20260709 13:02:44.017439  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.051348  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:43.966647  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.013599  6277 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:44.019811  6436 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.009966  1627 rpcz_store.cc:269] 0709 13:02:42.545251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.545366 (+   115us) service_pool.cc:224] Handling call
0709 13:02:43.950859 (+1405493us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54002986d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:43.950955 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332845824}
I20260709 13:02:43.793639  6355 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:44.026401  6343 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.812009  6070 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:44.039501  2294 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:44.065793  2294 pending_rounds.cc:85] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 8
I20260709 13:02:44.066155  2294 pending_rounds.cc:107] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:44.066447  2294 raft_consensus.cc:2889] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.012217  5988 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.041564  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:43.853965  6402 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.179s
I20260709 13:02:44.046749  1559 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:02:44.045055  5735 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:44.054821  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.855814  6388 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:43.875130  6415 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.198s
I20260709 13:02:44.063897  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.132421  1558 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:44.068007  1556 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.057333  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.133842  1633 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:02:44.044102  6350 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:44.018944  2291 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:44.065896  6309 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:44.057364  6277 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:44.058419  6435 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.063028  6436 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.080191  6356 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:44.081633  6147 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.987s	user 0.000s	sys 0.009s
I20260709 13:02:44.086838  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.150727  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.151026  1627 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:44.030730  6423 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.087899  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:44.153044  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65256.5us
I20260709 13:02:44.092718  1629 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:44.141335  6350 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.093724  1626 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:44.068041  6174 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:44.087913  6268 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.129271  6285 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.138880  1630 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:44.071359  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.164330  1554 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.047261  6437 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:02:44.060684  6123 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:44.120714  6316 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.002s
I20260709 13:02:44.154315  1629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:44.092288  5735 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:44.155061  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.027s	sys 0.026s
I20260709 13:02:44.156770  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.172892  1629 raft_consensus.cc:606] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:44.173156  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.027s	sys 0.027s
I20260709 13:02:44.173696  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:44.157285  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 432141us
I20260709 13:02:44.174387  1629 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:44.157382  2296 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:44.158313  1673 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.031s (SENT)
I20260709 13:02:44.177276  1629 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:02:44.167151  6310 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.858s	user 0.008s	sys 0.004s
I20260709 13:02:44.161051  6435 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:44.187493  1709 raft_consensus.cc:2410] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:44.169909  1627 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:44.174631  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.189142  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.072154  6070 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.167562  1630 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:44.182861  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 490289us
I20260709 13:02:44.153945  6147 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.193994  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.188333  1627 pending_rounds.cc:85] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 8
I20260709 13:02:44.196115  1627 pending_rounds.cc:107] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:02:44.196444  1627 raft_consensus.cc:2889] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.196508  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.172070  6123 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:44.247548  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.248461  1554 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:44.249029  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.206508  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:44.254264  1702 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.191327  5983 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:44.259622  1629 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:44.164614  6174 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:44.264842  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.184949  6435 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.247160  6389 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:44.252008  6322 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:44.184028  6233 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.181144  6316 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.250869  5988 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:44.204746  5465 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:44.277626  1634 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:02:44.278887  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 220066us
I20260709 13:02:44.284652  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.265609  6402 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:44.266287  6309 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:44.191634  6310 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.274919  6435 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.280331  5983 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:44.296743  1553 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:44.278966  6437 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:44.293704  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.281565  6355 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:44.303062  6388 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.303524  6355 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.304800  6316 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.289952  6309 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:44.317993  1556 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:44.319499  1559 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:44.310492  6427 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.982s	user 0.000s	sys 0.002s
I20260709 13:02:44.306531  6437 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.336016  6437 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:44.339931  6436 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:44.341781  6446 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:44.343827  6233 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.344008  6423 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:44.344796  6285 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.345373  6285 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:44.345824  6285 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.346803  6414 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.317459  6309 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:44.310878  6123 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.309487  6147 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.306981  5983 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:44.368337  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 161893us
I20260709 13:02:44.390923  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:44.351526  6445 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:02:44.399209  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:44.400004  1628 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.301415  6451 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:02:44.335367  6277 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.322187  6450 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
W20260709 13:02:44.412235  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 118604us
I20260709 13:02:44.392546  6285 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.397972  5465 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:44.422510  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 95187.5us
I20260709 13:02:44.426793  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:44.403576  6456 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.429883  6285 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.404260  6147 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:44.420586  6123 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.421016  6451 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:44.417238  6455 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.446854  1628 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:44.448961  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.454308  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.455080  1628 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:44.446589  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.456332  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.415782  6322 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:02:44.459527  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.456863  2293 raft_consensus.cc:2376] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11. Re-sending same reply.
I20260709 13:02:44.460386  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.462841  2295 raft_consensus.cc:2376] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 16. Re-sending same reply.
I20260709 13:02:44.436757  6070 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.471273  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:44.477826  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:44.478082  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.478505  1628 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:44.479547  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.440623  6147 consensus_queue.cc:1243] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:44.447854  6451 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:44.452179  6174 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.485529  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.492916  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.494074  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.494983  1556 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:02:44.497682  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.499128  1628 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:44.505558  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.511907  5465 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:44.512446  5465 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.514472  5465 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.503930  6350 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.515959  6414 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.516748  1552 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:44.517508  1557 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:44.519407  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.520064  1631 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:44.522893  1702 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
W20260709 13:02:44.523221  6443 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:02:44.524106  6310 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:44.532249  6465 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:44.443542  5983 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.541155  6451 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.542546  6451 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:44.489045  6454 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:02:44.554177  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.486878  6460 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:44.582881  1631 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:44.467775  6459 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:02:44.493785  6470 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:44.548787  6174 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:44.589659  6277 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.560729  6377 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:44.601060  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:44.604599  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 531416us
I20260709 13:02:44.568013  6436 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:44.607276  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.611397  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.608235  1523 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.729s (SENT)
I20260709 13:02:44.609201  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.584192  6460 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.620474  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1929) took 1511 ms. Trace:
I20260709 13:02:44.620931  1706 rpcz_store.cc:276] 0709 13:02:43.108964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.111901 (+  2937us) service_pool.cc:224] Handling call
0709 13:02:43.112210 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.112390 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:43.632785 (+520395us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.620136 (+987351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.620190 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:44.620232 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.620294 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.620416 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":446522,"prepare.queue_time_us":222638,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"thread_start_us":1060392,"threads_started":2,"wal-append.queue_time_us":650309}
I20260709 13:02:44.622241  6435 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.603682  6377 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:44.626905  6377 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.605211  6174 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:44.628767  1552 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:44.628728  6174 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.630230  1552 pending_rounds.cc:85] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 4
I20260709 13:02:44.630517  1552 pending_rounds.cc:107] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:02:44.630766  1552 raft_consensus.cc:2889] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.632495  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2519) took 2104 ms. Trace:
I20260709 13:02:44.632992  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2525) took 1471 ms. Trace:
I20260709 13:02:44.632874  1555 rpcz_store.cc:276] 0709 13:02:42.528229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.124377 (+596148us) service_pool.cc:224] Handling call
0709 13:02:43.130539 (+  6162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.217484 (+ 86945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.632308 (+1414824us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b54003fb450. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:44.632440 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13456640}
I20260709 13:02:44.633466  1553 rpcz_store.cc:276] 0709 13:02:43.161177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.204552 (+ 43375us) service_pool.cc:224] Handling call
0709 13:02:43.204885 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.301475 (+1096590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.632779 (+331304us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001edcd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:44.632935 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510592768}
W20260709 13:02:44.640126  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 374713us
W20260709 13:02:44.641695  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.590302  6470 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:44.642876  6470 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.644290  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1943) took 1215 ms. Trace:
I20260709 13:02:44.644829  1705 rpcz_store.cc:276] 0709 13:02:43.429310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.430723 (+  1413us) service_pool.cc:224] Handling call
0709 13:02:43.471527 (+ 40804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.471830 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:43.496209 (+ 24379us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.056132 (+559923us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.594086 (+537954us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b18004c1c20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:44.595586 (+  1500us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18004c1c20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1
0709 13:02:44.643582 (+ 47996us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.643660 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:44.643731 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.644065 (+   334us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.644215 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":531157,"prepare.queue_time_us":26452,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":1864,"raft.queue_time_us":551402,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":14032,"spinlock_wait_cycles":25013120,"thread_start_us":603265,"threads_started":2,"wal-append.queue_time_us":667455}
I20260709 13:02:44.620919  6460 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.655081  6277 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:44.703862  6456 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.705789  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.707299  2290 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:44.709285  6455 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.709591  6350 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.710227  6435 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.719158  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.719842  2295 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.724498  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.721411  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.639751  6309 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.727058  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.731216  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.729650  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.732225  2290 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:44.735072  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93035.6us
I20260709 13:02:44.734254  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.661021  6253 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 13:02:44.738184  1523 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.031s (SENT)
I20260709 13:02:44.753403  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.760903  1553 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:44.735934  6238 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:44.764894  6470 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.767201  6470 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:44.767787  6456 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:44.768042  6350 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:44.733083  6104 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:44.770174  5640 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:44.768333  6456 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:44.771517  1523 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.226s (SENT)
I20260709 13:02:44.774015  2295 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:44.774600  2290 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:44.769550  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 128309us
I20260709 13:02:44.778231  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.756453  6073 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:44.754417  6455 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.727727  6277 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.780771  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.784566  1553 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:44.790251  6238 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.792380  6277 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.794085  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2520) took 2265 ms. Trace:
I20260709 13:02:44.794502  1557 rpcz_store.cc:276] 0709 13:02:42.528491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.129943 (+601452us) service_pool.cc:224] Handling call
0709 13:02:43.130313 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.517925 (+1387612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.793850 (+275925us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003faa54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:44.794020 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3189822336}
I20260709 13:02:44.798494  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:44.801618  1673 leader_election.cc:400] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:02:44.802245  1673 leader_election.cc:403] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.801788  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1125) took 1302 ms. Trace:
I20260709 13:02:44.804443  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.806332  1557 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.807734  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.790280  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.792421  5640 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:44.809917  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.809984  5640 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.809127  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.772217  6104 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.817181  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.791182  6238 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.776209  6415 raft_consensus.cc:2764] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:44.805151  2288 rpcz_store.cc:276] 0709 13:02:43.499635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.014505 (+514870us) service_pool.cc:224] Handling call
0709 13:02:44.014839 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.059069 (+ 44230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.059116 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.059175 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:44.059229 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.801595 (+742366us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b54005a0fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:44.801638 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.801738 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3507,"raft.queue_time_us":30586,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":92706048}
I20260709 13:02:44.833870  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:44.840713  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.841307  1553 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.836261  5640 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.847182  5640 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:44.834542  2295 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:44.738619  6476 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:02:44.756892  6138 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:02:44.831053  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.808318  6268 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:44.771214  6423 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:02:44.828634  6478 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:02:44.811758  6073 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.858173  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:44.834343  6104 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.866079  1557 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:44.845398  6076 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.785712  6356 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:02:44.874701  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:44.874785  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.885943  1559 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:44.838274  6426 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:02:44.834822  6350 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:02:44.886883  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:44.859256  6268 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.854681  6435 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:44.809257  6455 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.846221  6352 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:44.884645  6476 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:44.883985  6178 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:44.898748  1557 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:44.915192  1557 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:44.887302  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53502.2us
I20260709 13:02:44.916276  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:44.872795  6285 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:44.901491  6277 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:44.901789  6487 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:44.883555  1553 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:44.927338  6487 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.928423  1553 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:44.928247  6487 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.929468  6487 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:44.906306  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2561) took 1246 ms. Trace:
I20260709 13:02:44.931200  1702 rpcz_store.cc:276] 0709 13:02:43.659427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.854146 (+194719us) service_pool.cc:224] Handling call
0709 13:02:44.906151 (+1052005us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540016c954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:44.906249 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567809280}
I20260709 13:02:44.859382  6070 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:02:44.919147  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:44.931985  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 73860.2us
I20260709 13:02:44.935019  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:44.940657  1559 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:44.941336  1557 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:44.942596  1559 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:44.965102  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46025us
I20260709 13:02:44.960212  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.943728  1559 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:44.967154  2289 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:44.967980  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.969560  2289 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:44.970178  1559 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:44.970798  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.971709  2289 raft_consensus.cc:2368] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13.
I20260709 13:02:44.911840  6476 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:44.918591  6435 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:44.981980  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:44.986390  2289 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:44.899432  6310 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:02:44.924780  6426 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:44.986069  6147 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.998342  6420 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.422s	user 0.000s	sys 0.008s
I20260709 13:02:44.968258  6178 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.000599  6178 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.000824  6460 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.922781  5983 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.990348  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:44.933498  6437 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.597s	user 0.002s	sys 0.008s
I20260709 13:02:44.959868  6073 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.004366  6309 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:45.026607  6437 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.027895  6476 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.959897  6285 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:45.029197  6476 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.029829  6437 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.029652  6285 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:44.989861  6492 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:45.033219  6123 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.001s
I20260709 13:02:44.990753  6268 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.996294  6435 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.034771  6268 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.998008  6147 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:44.985316  1559 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:44.979245  6277 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:44.903834  6456 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.025282  2289 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:45.025487  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:45.055411  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:44.937721  6253 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:02:45.058929  6455 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:44.961275  6070 consensus_queue.cc:1243] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:45.016878  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:45.057171  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.039618  5983 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.058280  6106 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:45.063308  1599 leader_election.cc:400] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13.
I20260709 13:02:45.085628  1599 leader_election.cc:403] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.069717  2290 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:45.082787  2289 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:02:45.090539  1674 connection.cc:256] RPC call timeout handler was delayed by -2.06698327700002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:02:45.090992  1674 connection.cc:256] RPC call timeout handler was delayed by -2.0597832770000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 13:02:45.091265  1674 connection.cc:256] RPC call timeout handler was delayed by -2.0588832770000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:02:45.091523  1674 connection.cc:256] RPC call timeout handler was delayed by -2.0525832769999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:02:45.091804  1674 connection.cc:256] RPC call timeout handler was delayed by -1.9418832769999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 13:02:45.092085  1674 connection.cc:256] RPC call timeout handler was delayed by -1.8014832769999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:02:45.092425  1674 connection.cc:256] RPC call timeout handler was delayed by -1.6430832770000166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
I20260709 13:02:45.078830  6436 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:45.092730  1674 connection.cc:256] RPC call timeout handler was delayed by -1.567483277000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
I20260709 13:02:45.093120  6436 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:45.093168  1674 connection.cc:256] RPC call timeout handler was delayed by -1.55218327700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
I20260709 13:02:45.093639  6436 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:45.093729  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2038832769999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2519s for a response
I20260709 13:02:45.094623  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.098881  6104 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.105172  6147 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.106050  6277 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:45.110404  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.113500  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.059410  6253 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:45.131201  6253 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.115236  6426 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:45.143927  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.116640  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60199.3us
I20260709 13:02:45.084630  6492 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.117306  6470 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:45.114753  6460 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:45.118489  6487 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.119539  6104 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:45.122483  6138 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:45.123462  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:45.126403  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:45.146080  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:45.128739  5983 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.084734  6352 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:45.089743  6106 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.116489  6423 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:45.155642  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 288908us
W20260709 13:02:45.158176  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 178381us
I20260709 13:02:45.161790  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:45.162408  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 329253us
I20260709 13:02:45.165628  2289 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:45.209304  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:45.211357  2289 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:45.207755  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46047.5us
I20260709 13:02:45.213423  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:45.219623  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.230195  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.231938  5983 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:45.232600  5983 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.234148  6309 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:45.234716  5983 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:45.235664  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.237591  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.237717  6437 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:45.238199  1633 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.238308  6437 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:45.243932  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:45.245007  1557 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.246037  1633 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:45.247972  6104 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:45.248790  6104 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:45.249322  6104 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.247619  6487 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:45.239507  6437 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.253319  6437 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.255618  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.256536  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.257136  1632 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:45.258052  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.258750  1626 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.259270  1551 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:45.238404  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.231647  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.261615  1551 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:45.263520  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.263919  6487 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.212409  2289 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.266934  2289 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.228508  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:45.268388  1626 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:02:45.272163  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.177526  6426 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.210114  6268 raft_consensus.cc:2764] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:45.207553  6106 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:45.199467  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 73101.6us
I20260709 13:02:45.267977  5912 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:45.271502  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.178079  6147 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:45.301899  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:45.280982  6123 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:45.316999  6123 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:45.209882  6459 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:45.175673  6436 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.325798  1557 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:45.328896  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:45.159415  6492 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.334128  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.266134  6316 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:02:45.339105  2290 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:45.334460  1633 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:45.200057  6340 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:02:45.340173  2289 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
W20260709 13:02:45.340693  6419 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.868s	user 0.005s	sys 0.009s
W20260709 13:02:45.348963  6494 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:02:45.340886  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.343175  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:45.350664  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1079) took 3045 ms. Trace:
I20260709 13:02:45.351138  2297 rpcz_store.cc:276] 0709 13:02:42.305127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.305253 (+   126us) service_pool.cc:224] Handling call
0709 13:02:42.334781 (+ 29528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.336595 (+  1814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.336626 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.344974 (+  8348us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.584105 (+239131us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b180036a660. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:42.948517 (+364412us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:42.948586 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.948689 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.339970 (+2391281us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:45.350371 (+ 10401us) raft_consensus.cc:1685] finished
0709 13:02:45.350391 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.350548 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43308,"raft.queue_time_us":46495,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2229,"spinlock_wait_cycles":15453440,"thread_start_us":364187,"threads_started":1,"wal-append.queue_time_us":972998}
I20260709 13:02:45.336138  6147 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:45.355398  6147 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.347132  2289 pending_rounds.cc:85] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 9
I20260709 13:02:45.359982  2289 pending_rounds.cc:107] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:45.360998  2289 raft_consensus.cc:2889] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.367708  5983 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.373149  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.320206  6104 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.206138  6253 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.335451  5912 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:45.338248  6435 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:45.318554  6123 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.383118  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.405323  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.369164  6451 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:45.337917  6459 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.392361  6253 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.386924  6489 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:45.450875  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.378799  5983 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.458168  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:45.458853  2290 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:45.455937  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.472898  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.473654  1702 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:45.474968  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.475689  1708 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.478152  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.483140  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.473712  1700 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:45.490654  1706 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:45.430819  6238 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.493492  1707 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:45.456707  6489 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.466971  6309 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:45.513870  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1126) took 1854 ms (client timeout 1729 ms). Trace:
W20260709 13:02:45.504268  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 161147us
W20260709 13:02:45.514362  2296 rpcz_store.cc:269] 0709 13:02:43.659551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.667227 (+  7676us) service_pool.cc:224] Handling call
0709 13:02:45.512495 (+1845268us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b5400770e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:45.513797 (+  1302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048273664}
W20260709 13:02:45.513811  1523 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.729s (SENT)
W20260709 13:02:45.517400  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 74662.4us
W20260709 13:02:45.521636  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 177669us
W20260709 13:02:45.535959  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 212228us
I20260709 13:02:45.526345  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:02:45.541746  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.514278  6489 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.514133  1702 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:45.503700  6238 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:45.555584  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.559293  1702 pending_rounds.cc:85] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 8
W20260709 13:02:45.568845  2260 leader_election.cc:336] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.912s (SENT)
I20260709 13:02:45.521646  6309 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:45.483624  5912 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.554044  2290 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:45.570140  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.588231  1708 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:45.590660  6500 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:45.571421  6436 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.592451  6436 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.569130  1702 pending_rounds.cc:107] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:45.592938  6436 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.593405  1702 raft_consensus.cc:2889] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.593986  6459 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.595132  6436 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:45.595323  2260 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.941s (SENT)
I20260709 13:02:45.595814  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.597374  5640 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:45.597886  5640 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.598809  5640 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:45.601765  6502 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:02:45.574954  6496 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:02:45.574869  6309 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.541666  6435 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:45.560125  6147 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:45.607278  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 91844.1us
W20260709 13:02:45.607543  1524 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509): Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.729s (SENT)
I20260709 13:02:45.608078  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:45.578863  6238 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:45.573180  6495 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:45.555241  6492 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:45.581310  6476 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.552s	user 0.005s	sys 0.007s
W20260709 13:02:45.604559  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 148726us
I20260709 13:02:45.553928  6514 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:45.615651  6106 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:45.619987  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 356463us
I20260709 13:02:45.624558  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.628598  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.630927  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:45.633536  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.642064  1626 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:02:45.642352  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62954.2us
I20260709 13:02:45.645480  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.650245  1708 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:45.652276  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.653100  1633 raft_consensus.cc:2368] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14.
I20260709 13:02:45.654202  1634 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:45.657339  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:45.658286  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.659754  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.660621  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.662166  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:45.662528  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54203us
I20260709 13:02:45.662824  2297 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:45.663184  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.666680  1626 raft_consensus.cc:2368] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:45.667273  1633 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:45.668870  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.670688  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.673903  6268 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.674576  6268 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:45.675217  6268 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:45.676934  6493 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:02:45.677112  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 924) took 1219 ms. Trace:
W20260709 13:02:45.677943  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 1933) took 2567 ms (client timeout 2874 ms). Trace:
W20260709 13:02:45.678455  1709 rpcz_store.cc:269] 0709 13:02:43.110072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.271426 (+161354us) service_pool.cc:224] Handling call
0709 13:02:45.662691 (+2391265us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:45.677851 (+ 15160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014872448}
I20260709 13:02:45.672020  6073 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:02:45.677176  6510 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:45.675220  1628 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:02:45.681573  1628 pending_rounds.cc:85] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 11
I20260709 13:02:45.681802  1628 pending_rounds.cc:107] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:02:45.682165  1628 raft_consensus.cc:2889] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.684705  6311 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:45.685103  6311 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:45.685916  6311 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.687063  6311 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:45.645452  6514 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.689877  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.690754  6522 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.693460  6491 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.694475  6104 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:45.696131  6459 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.696589  6106 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.696839  6459 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:45.697525  6523 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:45.698359  6343 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:45.698850  6343 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.699498  5983 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.700192  5983 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:45.700593  5983 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:45.700803  6343 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:45.656788  6492 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:02:45.701538  5983 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:45.637699  6495 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.667798  6519 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:02:45.702389  6492 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:45.707176  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:45.677803  2291 rpcz_store.cc:276] 0709 13:02:44.457429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.457554 (+   125us) service_pool.cc:224] Handling call
0709 13:02:44.457851 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.466544 (+  8693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.466576 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.803428 (+336852us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b2c00047568. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:02:44.803844 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.200728 (+396884us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.230017 (+ 29289us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.231227 (+  1210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.231305 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.676791 (+445486us) raft_consensus.cc:1685] finished
0709 13:02:45.676900 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.677049 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35006,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":858,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":202,"spinlock_wait_cycles":7629056,"thread_start_us":396527,"threads_started":1,"wal-append.queue_time_us":613414}
W20260709 13:02:45.732092  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 173601us
W20260709 13:02:45.677814  6504 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:02:45.736212  2297 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:45.736630  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:45.635332  6435 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:45.738386  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 143075us
W20260709 13:02:45.723400  6470 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.001s
I20260709 13:02:45.748268  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:45.754057  2260 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 3.173s (SENT)
I20260709 13:02:45.755105  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.756752  2297 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:45.757195  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:45.749995  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 102407us
W20260709 13:02:45.751181  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.731549  6492 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.722769  6499 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.723264  6238 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:45.751711  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 91175.8us
I20260709 13:02:45.759600  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.760023  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.761525  1708 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:45.673774  6512 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:02:45.761425  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.762850  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.766731  1708 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:45.744231  6522 raft_consensus.cc:2764] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:02:45.763466  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.771214  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.764430  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.764731  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:45.768791  2260 leader_election.cc:336] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.001s (SENT)
I20260709 13:02:45.767226  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.772135  1704 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:45.772336  1700 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:45.772584  1701 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:45.768301  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.769575  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.750686  6268 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:45.703878  6517 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:45.775317  1708 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:45.765746  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 158233us
I20260709 13:02:45.774648  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.779349  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:45.780318  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.780475  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.782181  1701 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:45.774176  1703 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:45.784191  1705 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:45.784397  1555 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:45.756227  6104 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:45.758198  6470 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13.
I20260709 13:02:45.790213  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.792933  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.822682  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.793493  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.823666  1552 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:45.824779  1552 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:45.752884  6178 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:02:45.761933  6238 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.794834  1560 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:45.796329  1559 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:45.788326  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.787494  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.791628  1704 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:45.797399  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:45.797873  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.799103  1709 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:45.800323  1707 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:45.827905  1701 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:45.801158  6495 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.804623  1631 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:45.805830  1630 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:45.807307  6456 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:45.808202  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.762248  6521 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.812390  6514 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:45.814184  6489 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.758349  6510 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:45.819864  1551 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:45.830695  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61964.6us
I20260709 13:02:45.860164  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:45.860814  2260 leader_election.cc:336] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.076s (SENT)
I20260709 13:02:45.861253  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:45.861701  2260 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.126s (SENT)
I20260709 13:02:45.862087  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:45.862221  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:45.848553  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51223.2us
W20260709 13:02:45.849421  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 73297.5us
I20260709 13:02:45.796480  6523 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:45.850461  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.833603  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.848964  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.821128  6268 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:45.822395  6104 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.855469  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:45.880606  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.785878  6415 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:02:45.864930  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:45.831127  1633 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:45.877003  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.877179  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.878377  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.879228  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.880499  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.857695  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.849502  6512 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.881861  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:45.880772  1704 raft_consensus.cc:2376] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13. Re-sending same reply.
I20260709 13:02:45.858824  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:45.857628  6493 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:45.885788  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:45.886615  1709 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:45.886965  1706 raft_consensus.cc:2376] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 16. Re-sending same reply.
I20260709 13:02:45.960341  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.961444  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.961723  1633 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:45.888146  1630 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:45.888633  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:45.772195  6499 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.965521  6499 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:45.974102  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:45.887070  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:45.980105  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 93113.4us
I20260709 13:02:45.885155  6104 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:45.855348  6522 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:45.854921  6123 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:45.949398  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:45.854285  6533 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.877852  6513 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:45.884186  6343 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.883982  6238 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:45.945194  6515 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.052359  1631 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:45.881956  6523 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.027729  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:45.991855  6073 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:46.057844  6431 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:46.058472  6431 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.060523  6377 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:46.062214  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:46.062868  2297 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:46.063863  6343 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.066187  6343 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.066478  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:46.066746  6515 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.067931  6537 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.069761  6515 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:46.072351  1523 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.585s (SENT)
I20260709 13:02:45.990707  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:46.030261  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 147856us
I20260709 13:02:46.031082  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 754) took 1249 ms. Trace:
I20260709 13:02:45.981953  6524 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:02:46.086338  1553 rpcz_store.cc:276] 0709 13:02:44.782115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.929184 (+147069us) service_pool.cc:224] Handling call
0709 13:02:44.929514 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.929911 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.929947 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.930580 (+   633us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.145303 (+214723us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.200058 (+ 54755us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.200341 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.200451 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.030842 (+830391us) raft_consensus.cc:1685] finished
0709 13:02:46.030865 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.031014 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63240,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":3608,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"spinlock_wait_cycles":45341952,"thread_start_us":214169,"threads_started":1,"wal-append.queue_time_us":522507}
W20260709 13:02:46.030496  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 165669us
W20260709 13:02:46.090588  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 99967.2us
I20260709 13:02:45.992751  6531 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:46.033721  6530 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:46.034021  6522 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.038946  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.889732  6535 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.171s
I20260709 13:02:46.044476  1706 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:45.978677  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:45.890803  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 115368us
I20260709 13:02:45.869962  6529 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.128s
I20260709 13:02:46.029498  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.070640  6513 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:46.075888  5640 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.994653  6510 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:02:46.078639  1630 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:46.076066  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:46.081569  2297 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:46.099610  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.056434  6123 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:46.070709  6526 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:46.106521  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.109680  1704 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 16 because replica has last-logged OpId of term: 15 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 5.
I20260709 13:02:46.114101  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.076380  6489 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.050002  6268 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.120493  1700 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:46.047593  6512 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:02:46.121496  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.129289  1707 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:46.122385  2288 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:46.124087  1706 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:46.088629  6238 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.096822  6536 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.098125  6415 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:46.112617  1631 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:46.101459  6070 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:46.127182  6510 consensus_queue.cc:1243] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:46.139937  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.119215  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:46.103618  6178 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:02:46.144989  2289 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:46.107463  6073 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:02:46.161787  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.172511  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.173812  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:46.163165  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:46.113432  6147 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:02:46.169664  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.170753  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51677.9us
I20260709 13:02:46.177152  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:46.154868  6529 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:46.161796  6526 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.160214  6070 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.189479  1631 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:46.180455  6435 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:46.137454  6534 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:02:46.241645  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.176343  6489 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:46.262707  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 122944us
I20260709 13:02:46.262877  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.263525  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:46.265995  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:46.266929  1631 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:46.244735  6419 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:46.196189  6070 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:46.270934  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 137172us
I20260709 13:02:46.281250  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.154384  6536 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:46.268985  6311 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.581s	user 0.008s	sys 0.007s
I20260709 13:02:46.269164  6253 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.253103  6316 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:46.173750  6123 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.272482  6456 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:46.242883  6524 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:46.261824  6535 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:02:46.280229  6515 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:46.282649  6489 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:46.196586  6526 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.292944  6316 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.438591  6559 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:46.442127  6550 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.366408  1523 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.883s (SENT)
I20260709 13:02:46.450380  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.450330  6550 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.451193  6377 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:46.451370  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 85026us
I20260709 13:02:46.451725  6377 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.452428  6550 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:46.452930  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.453321  6431 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.453490  6343 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:46.453657  6377 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:46.441450  6541 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.313s
I20260709 13:02:46.291271  6253 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.459331  6543 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.459996  6253 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:46.459887  6515 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.292729  6549 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.461113  6549 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.462126  6515 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.443732  6526 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:46.445456  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.463173  6549 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:46.445677  6538 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:02:46.465101  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 931) took 1147 ms. Trace:
I20260709 13:02:46.465474  2296 rpcz_store.cc:276] 0709 13:02:45.317597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.542883 (+225286us) service_pool.cc:224] Handling call
0709 13:02:45.557147 (+ 14264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.655956 (+ 98809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.656024 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.665607 (+  9583us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.790958 (+125351us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:45.791004 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.791077 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.464870 (+673793us) raft_consensus.cc:1685] finished
0709 13:02:46.464895 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.465036 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115148,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":48,"thread_start_us":116545,"threads_started":2,"wal-append.queue_time_us":204145}
W20260709 13:02:46.474419  6547 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
W20260709 13:02:46.477006  6551 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:02:46.479494  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.480856  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:46.481451  1708 raft_consensus.cc:2376] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 12. Re-sending same reply.
I20260709 13:02:46.191093  6513 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.487062  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.475445  6511 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.797s	user 0.001s	sys 0.001s
W20260709 13:02:46.498313  1523 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.042s (SENT)
I20260709 13:02:46.436474  6522 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.489303  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:46.446354  6545 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 13:02:46.443114  6548 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
W20260709 13:02:46.447918  6539 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:02:46.463730  6549 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:46.558207  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 276986us
I20260709 13:02:46.534266  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.448385  6316 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.563395  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.563920  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:46.449218  6536 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.524631  6559 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:46.523238  6123 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:46.561755  6377 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.587383  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.588644  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.441529  6540 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.398s
I20260709 13:02:46.474633  6104 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.581475  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:46.562265  6526 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:46.593935  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.595046  1708 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:02:46.578940  6549 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:46.578733  6553 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:46.460009  6543 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.446115  6489 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.614804  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 351314us
I20260709 13:02:46.620280  1702 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:46.617929  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.617975  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.620959  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:46.611459  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.623144  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2517) took 1368 ms. Trace:
W20260709 13:02:46.630255  6527 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.938s	user 0.009s	sys 0.003s
I20260709 13:02:46.633407  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1142) took 1307 ms. Trace:
I20260709 13:02:46.634444  6557 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:46.616900  6561 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.637506  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:46.638875  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.639444  1704 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:46.641355  1704 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:46.641990  6104 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.643076  1704 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:46.643437  1704 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:46.644219  1704 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.645081  1704 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.645596  1702 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:46.614167  6559 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:46.591296  6377 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.633810  2290 rpcz_store.cc:276] 0709 13:02:45.325762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.584355 (+258593us) service_pool.cc:224] Handling call
0709 13:02:45.584800 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.585122 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.585157 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.585709 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.646524 (+ 60815us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.646566 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.646638 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.629637 (+982999us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:46.633092 (+  3455us) raft_consensus.cc:1685] finished
0709 13:02:46.633115 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.633342 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68456320,"thread_start_us":60437,"threads_started":1,"wal-append.queue_time_us":106256}
I20260709 13:02:46.653864  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:46.659040  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:46.654582  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:46.623677  1635 rpcz_store.cc:276] 0709 13:02:45.254703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.255717 (+  1014us) service_pool.cc:224] Handling call
0709 13:02:45.256082 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.256431 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.256458 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.257048 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.543790 (+286742us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.543829 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.543894 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.474655 (+930761us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:46.615737 (+141082us) raft_consensus.cc:1685] finished
0709 13:02:46.615801 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.623042 (+  7241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6472,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":74,"spinlock_wait_cycles":64939904,"thread_start_us":286343,"threads_started":1,"wal-append.queue_time_us":378406}
I20260709 13:02:46.658074  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:46.660125  1553 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:46.659756  1708 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
W20260709 13:02:46.656795  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 469463us
I20260709 13:02:46.660668  1708 pending_rounds.cc:85] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 6
I20260709 13:02:46.662135  1708 pending_rounds.cc:107] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:46.662360  1708 raft_consensus.cc:2889] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:46.662672  2291 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:46.608848  6549 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:46.664224  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.647668  6525 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:46.668951  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:46.671519  1628 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:46.675482  1708 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:46.676064  1708 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:46.647715  6559 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.613243  6526 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.687835  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.695549  1523 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.578s (SENT)
I20260709 13:02:46.680986  1708 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.670924  6525 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:46.614488  6553 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.634516  6070 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.703269  1708 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:46.707151  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 85814.2us
I20260709 13:02:46.709566  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:46.637598  6561 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.715095  1554 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:46.706523  6541 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:46.714915  6238 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.000s
I20260709 13:02:46.736485  6070 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:46.737689  1701 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:46.741578  6562 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:46.742303  6562 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.749575  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 770) took 1105 ms. Trace:
I20260709 13:02:46.750227  1626 rpcz_store.cc:276] 0709 13:02:45.644603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.645206 (+   603us) service_pool.cc:224] Handling call
0709 13:02:46.749236 (+1104030us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b54003ded54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:46.749469 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46561024}
W20260709 13:02:46.755293  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2515) took 1517 ms (client timeout 1843 ms). Trace:
W20260709 13:02:46.755867  1627 rpcz_store.cc:269] 0709 13:02:45.237500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.238175 (+   675us) service_pool.cc:224] Handling call
0709 13:02:46.755094 (+1516919us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002986d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:46.755227 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364697856}
W20260709 13:02:46.761838  6528 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.715s	user 0.001s	sys 0.001s
I20260709 13:02:46.768921  6377 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.771972  6526 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.773247  6238 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.773980  6523 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.773839  6238 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.779641  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 158794us
I20260709 13:02:46.731990  6524 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:46.783620  1708 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:46.782415  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:46.798751  1708 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:46.800005  1708 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:46.787118  6076 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:02:46.723625  6565 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
W20260709 13:02:46.809062  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.817622  1708 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:46.818617  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 107818us
I20260709 13:02:46.822327  1708 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:46.823697  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.804167  6524 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.743515  6562 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.827705  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:46.806167  6570 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:46.825506  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:46.815265  6557 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:46.830600  2290 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:46.834815  6553 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.837431  6492 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.805668  6560 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 13:02:46.776854  6525 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.778116  6559 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:46.849913  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111892us
I20260709 13:02:46.832693  6238 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:46.850785  1523 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 3.005s (SENT)
I20260709 13:02:46.840240  6571 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:02:46.846634  6570 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.848165  6565 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:46.853188  6464 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.888091  6464 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.888868  6578 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:46.889348  6578 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.890163  6578 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.890939  6464 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.872334  6525 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.896502  1708 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
W20260709 13:02:46.885059  6456 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.004s
I20260709 13:02:46.919938  6456 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:46.902454  6521 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.031s	user 0.011s	sys 0.000s
I20260709 13:02:46.902487  6455 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.921154  6521 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:46.921278  6455 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.921974  1708 pending_rounds.cc:85] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 8
I20260709 13:02:46.922526  1708 pending_rounds.cc:107] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:02:46.922784  1708 raft_consensus.cc:2889] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:46.923290  1709 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:46.891461  6578 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:46.923864  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2590) took 1041 ms. Trace:
I20260709 13:02:46.903024  6524 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.924245  1704 rpcz_store.cc:276] 0709 13:02:45.883079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.674064 (+790985us) service_pool.cc:224] Handling call
0709 13:02:46.674434 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.674719 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.674753 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.679961 (+  5208us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.860077 (+180116us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:46.860132 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.860230 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.923654 (+ 63424us) raft_consensus.cc:1685] finished
0709 13:02:46.923674 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.923808 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":149359,"threads_started":1,"wal-append.queue_time_us":208696}
I20260709 13:02:46.925328  6519 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:46.925736  6519 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:46.903379  6549 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.926478  6519 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:46.928407  6549 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.904059  6561 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.904373  6464 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.855379  6541 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:46.906915  6570 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.929502  6464 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:46.909229  6572 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:46.909554  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2519) took 1654 ms. Trace:
I20260709 13:02:46.932744  6464 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:46.932760  1629 rpcz_store.cc:276] 0709 13:02:45.255255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.255373 (+   118us) service_pool.cc:224] Handling call
0709 13:02:46.909345 (+1653972us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:46.909492 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686477568}
I20260709 13:02:46.923552  6455 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.934399  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2678) took 1046 ms. Trace:
I20260709 13:02:46.934859  1633 rpcz_store.cc:276] 0709 13:02:45.888214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.962469 (+ 74255us) service_pool.cc:224] Handling call
0709 13:02:45.962788 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.133848 (+171060us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:46.263196 (+129348us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.709358 (+446162us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.712321 (+  2963us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.712487 (+   166us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:46.712786 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.814681 (+101895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.814804 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.934187 (+119383us) raft_consensus.cc:1685] finished
0709 13:02:46.934210 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.934340 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":312666,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":148160,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":3468,"spinlock_wait_cycles":3088768,"thread_start_us":789532,"threads_started":4,"wal-append.queue_time_us":503869}
W20260709 13:02:46.941112  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.941632  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.942065  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.944098  6070 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.944788  6070 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:46.962344  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:46.970960  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.974133  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:46.977519  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:46.979002  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:46.913414  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 85811.8us
I20260709 13:02:46.980211  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.984534  2293 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:46.947351  6070 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:46.976120  6456 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:46.986946  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.972358  6519 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.988829  2296 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:46.992564  2292 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:46.989665  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:46.989886  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.977087  6464 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:46.991060  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:46.975401  6455 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:46.990236  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:46.995512  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:46.994203  6497 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:46.986235  6513 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:47.002293  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.011394  1708 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:46.985602  6523 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.017032  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:47.020097  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:47.021241  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.022015  1704 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:47.026186  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.028342  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.029229  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:47.032588  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.033674  2262 leader_election.cc:400] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14.
I20260709 13:02:47.034127  2262 leader_election.cc:403] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:02:47.034834  6451 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14.
I20260709 13:02:47.035955  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:47.021474  1630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:47.036221  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:47.036881  1630 raft_consensus.cc:606] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:47.037297  1630 raft_consensus.cc:614] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:47.037449  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.038879  1633 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:47.004761  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:46.998759  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.008395  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1088) took 2100 ms. Trace:
I20260709 13:02:47.009229  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.040419  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.041083  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.040433  2295 rpcz_store.cc:276] 0709 13:02:44.908366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.909611 (+  1245us) service_pool.cc:224] Handling call
0709 13:02:44.909991 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.039243 (+129252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.039288 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.039346 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:45.039399 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.008143 (+1968744us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400610dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:47.008192 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.008334 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3477,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4504,"spinlock_wait_cycles":296325504}
I20260709 13:02:47.042658  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.043435  2295 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:46.976629  6525 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.045164  6525 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.040683  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:47.048424  2262 leader_election.cc:400] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:47.048794  2262 leader_election.cc:403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:47.048238  6451 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:46.983654  6535 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:47.049902  6123 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15.
I20260709 13:02:47.050112  6525 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.051097  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.041796  2293 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:47.055074  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:47.056017  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.056638  1708 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:47.051378  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:47.057893  1627 raft_consensus.cc:606] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:47.000015  6456 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:47.010269  6455 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:47.059294  1627 raft_consensus.cc:614] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:47.058159  6076 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:47.035990  6553 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:02:47.060755  2296 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:47.023063  6070 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:47.073266  1708 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:47.058153  6570 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:47.052446  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47832us
I20260709 13:02:47.037081  6536 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:47.075209   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:47.051462  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:47.035526  6377 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:47.062870  6455 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.067277  6316 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.067003  6497 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:47.061744  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48380us
I20260709 13:02:47.081159  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.036193  6523 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.076617  6536 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.079905  6431 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:47.087461  6553 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:47.117728  6497 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.119100  6285 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:02:47.124099  6456 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.129594  6535 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:47.131038  6535 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:47.133188  6356 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:02:47.137940  6464 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:47.138769  6464 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.139564  5983 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:47.139218  6464 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.117898  6316 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.144443  6464 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.145828  6523 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.146296  6316 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.146829  6497 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.151683  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.151984  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:47.152534  1628 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:47.153290   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:47.154188  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.154863  1629 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:47.161867  1628 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:47.161990  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.081076  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:47.164746  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.168175  6356 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.168370  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.119899  6536 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:47.178879  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97884.3us
I20260709 13:02:47.126111  6553 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.180006  6253 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:47.172554  1628 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:47.183789  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:47.078405  6570 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.183988  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.186702  2297 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:47.203048  1635 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:47.199453  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:47.207121  5640 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:47.194133  6553 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:47.214962  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 98381.2us
I20260709 13:02:47.198206  6268 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:47.199563  6377 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:47.205255  6350 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:47.228276  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 783) took 1464 ms. Trace:
I20260709 13:02:47.228765  1705 rpcz_store.cc:276] 0709 13:02:45.764093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.764213 (+   120us) service_pool.cc:224] Handling call
0709 13:02:47.222005 (+1457792us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b5400097254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:47.228201 (+  6196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27127680}
I20260709 13:02:47.199263  6238 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:02:47.186167  6592 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:47.188230  6356 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.217623  6253 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:47.220252  6268 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.216336  6523 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:02:47.231472  6377 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.246533  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.270884  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.271364  6356 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.272428  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.235599  6350 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.253794  1631 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:02:47.283733  1631 pending_rounds.cc:85] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 6
I20260709 13:02:47.284067  1631 pending_rounds.cc:107] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:47.255321  6513 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:47.255934  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 768) took 1621 ms. Trace:
I20260709 13:02:47.257295  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.244336  6570 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:47.281694  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.284322  1631 raft_consensus.cc:2889] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:47.285718  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:47.259096  6070 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:47.290073  1629 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:47.288373  1632 rpcz_store.cc:276] 0709 13:02:45.639247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.639558 (+   311us) service_pool.cc:224] Handling call
0709 13:02:45.643298 (+  3740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.643735 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.643793 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.644385 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.936054 (+1291669us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b180036c400. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:47.078959 (+142905us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:47.078998 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.079078 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.238348 (+159270us) raft_consensus.cc:1685] finished
0709 13:02:47.238369 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.255849 (+ 17480us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25538,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":29963,"spinlock_wait_cycles":66166016,"thread_start_us":142704,"threads_started":1,"wal-append.queue_time_us":1537682}
I20260709 13:02:47.295194  1704 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.297250  1629 raft_consensus.cc:1217] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:02:47.297916  1629 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:47.299149  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.300779  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.295974  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:47.302846  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:47.303645  1635 raft_consensus.cc:2376] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 13. Re-sending same reply.
I20260709 13:02:47.268968  6268 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:47.307013  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:47.309685  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.309899  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:47.310721  1709 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:47.342568  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.283744  6076 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.371469  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.372555  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.378403  1708 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:02:47.379938  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.406845  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:47.288080  6253 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.448987  1635 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:47.460429  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.462296  1704 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:47.463668  1704 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:47.466462  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:47.293648  6356 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:47.474431  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 275069us
W20260709 13:02:47.476264  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 105434us
I20260709 13:02:47.477934  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.478327  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.478185  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:47.488840  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:47.301748  6070 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:47.311034  6316 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:47.311829  6526 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:47.444677  6123 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.477054  6253 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.483844  6598 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:02:47.478946  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:47.460942  6238 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.509296  1709 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:47.477893  6456 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.488322  6464 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:47.491418  6350 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.521103  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:47.618068  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1113) took 1494 ms. Trace:
I20260709 13:02:47.461577  6076 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:47.620275  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 99294.4us
I20260709 13:02:47.622229  2289 rpcz_store.cc:276] 0709 13:02:46.124066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.124235 (+   169us) service_pool.cc:224] Handling call
0709 13:02:46.124554 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.171704 (+ 47150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.617730 (+1446026us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b5400350054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:47.617986 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46113536}
I20260709 13:02:47.511445  6316 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:47.629810  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:47.490549  6356 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.634548  2294 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:47.619997  6489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.016s	sys 0.048s
I20260709 13:02:47.494818  6455 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.611004  6451 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.619364  6570 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.627460  6559 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:47.514319  6123 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.657913  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:47.632153  6350 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.524863  6238 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.613729  6464 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.642388  6598 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.649181  6489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.016s	sys 0.048s
I20260709 13:02:47.611253  6456 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:47.680060  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2568) took 1846 ms (client timeout 1585 ms). Trace:
I20260709 13:02:47.681123  1702 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:47.632998  6536 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.649188  6070 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:47.724050  6601 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:02:47.656343  6451 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.665745  6350 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.669602  6553 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.661096  6570 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.724681  6253 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.744045  6525 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:47.713786  6076 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.735085  6455 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.736277  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.737376  6604 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.738236  6277 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.844852  6277 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.846123  6570 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.841068  6076 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.688163  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.847391  2288 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:02:47.847045  6076 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:47.739162  5735 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:02:47.849712  6076 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:47.850402  6076 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:47.850827  6076 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.851262  6076 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.838258  6525 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.657038  6559 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.715309  6238 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.835863  6070 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:47.858038  6559 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.858201  6238 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:47.858780  6238 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.758116  6543 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:47.745553  1700 rpcz_store.cc:269] 0709 13:02:45.834012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.858567 (+ 24555us) service_pool.cc:224] Handling call
0709 13:02:47.679831 (+1821264us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:47.679969 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371285760}
I20260709 13:02:47.718626  6531 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:47.723049  6356 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.743266  6491 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:02:47.713100  6377 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:47.754717  6451 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.755405  6350 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.878436  6456 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.721539  6464 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.758517  6534 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:47.763981  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2565) took 1931 ms (client timeout 1729 ms). Trace:
W20260709 13:02:47.773411  2260 connection.cc:256] RPC call timeout handler was delayed by -1.032239092999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 13:02:47.880703  1700 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:47.881260  2260 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.078s (SENT)
W20260709 13:02:47.880877  1701 rpcz_store.cc:269] 0709 13:02:45.832993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.833276 (+   283us) service_pool.cc:224] Handling call
0709 13:02:47.763714 (+1930438us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:47.763891 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989896576}
I20260709 13:02:47.881765  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:47.774485  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2547) took 1994 ms. Trace:
I20260709 13:02:47.882619  1551 rpcz_store.cc:276] 0709 13:02:45.789210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.819301 (+ 30091us) service_pool.cc:224] Handling call
0709 13:02:45.819653 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.860037 (+ 40384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.774123 (+1914086us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400109120. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e7a02 00007f70ce9e3dd6 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:02:47.774407 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580430208}
I20260709 13:02:47.775383  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1109) took 1714 ms. Trace:
I20260709 13:02:47.717303  6123 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:47.761552  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2684) took 1617 ms. Trace:
I20260709 13:02:47.800532  6253 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.801962  6553 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:47.803620  6536 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.732174  6316 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.823935  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.905927  2295 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:47.825845  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.743096  6592 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
W20260709 13:02:47.835614  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.908718  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.909225  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.908929  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:47.909762  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.909996  2292 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:02:47.841231  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.857311  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.687672  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.860855  6238 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.847064  6277 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:47.875393  6356 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.815955  6526 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.884104  2291 rpcz_store.cc:276] 0709 13:02:46.061268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.066197 (+  4929us) service_pool.cc:224] Handling call
0709 13:02:47.775190 (+1708993us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:47.775315 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369371904}
I20260709 13:02:47.887068  1631 rpcz_store.cc:276] 0709 13:02:46.144339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.286921 (+142582us) service_pool.cc:224] Handling call
0709 13:02:46.294651 (+  7730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.294791 (+1000140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.761290 (+466499us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b5400343ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:47.761486 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204941568}
I20260709 13:02:47.894816  1700 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:02:47.900346  6253 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.907415  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.892019  6451 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:47.936769  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.872674  6531 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:47.942842  1700 pending_rounds.cc:85] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 4
I20260709 13:02:47.958038  1700 pending_rounds.cc:107] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:02:47.958323  1700 raft_consensus.cc:2889] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:47.952481  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.021s	sys 0.036s
I20260709 13:02:47.761814  6598 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:47.955708  6268 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
I20260709 13:02:47.892449  6377 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:47.960279  6268 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.960834  6253 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.918587  6553 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.942214  6356 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:47.961886  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.021s	sys 0.036s
I20260709 13:02:47.961145  6377 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.965415  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:47.966076  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2544) took 2187 ms. Trace:
I20260709 13:02:47.966533  1557 rpcz_store.cc:276] 0709 13:02:45.778610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.778729 (+   119us) service_pool.cc:224] Handling call
0709 13:02:45.778993 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.966516 (+187523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.966561 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.967243 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.113877 (+146634us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:46.113926 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.953969 (+1840043us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54002b98d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:47.954049 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.954108 (+    59us) raft_consensus.cc:1685] finished
0709 13:02:47.954131 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.966013 (+ 11882us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6092,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":113,"spinlock_wait_cycles":430245248,"thread_start_us":145033,"threads_started":1,"wal-append.queue_time_us":193988,"wal-append.run_cpu_time_us":3493,"wal-append.run_wall_time_us":1306718}
I20260709 13:02:47.969821  6322 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:47.992360  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.021s	sys 0.043s
W20260709 13:02:48.049214  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.021s	sys 0.043s
I20260709 13:02:47.957348  6238 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.960871  6598 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:48.054004  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 575097us
W20260709 13:02:48.057025  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.067427  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.066068  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:47.950953  6526 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.066723  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.019s
W20260709 13:02:48.074667  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.000s	sys 0.020s
W20260709 13:02:48.075546  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 597394us
I20260709 13:02:48.080937  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2589) took 2222 ms. Trace:
I20260709 13:02:48.081382  1702 rpcz_store.cc:276] 0709 13:02:45.878526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.660907 (+782381us) service_pool.cc:224] Handling call
0709 13:02:47.680792 (+1019885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.746352 (+ 65560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.080737 (+334385us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001b7950. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:48.080879 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344735616}
W20260709 13:02:48.084447  6611 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 13:02:48.084828  6606 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.347s	user 0.014s	sys 0.005s
I20260709 13:02:48.055445  6451 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.979097  6553 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.088142  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.056665  6238 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.109844  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.008s	sys 0.012s
W20260709 13:02:48.111522  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.009s	sys 0.012s
I20260709 13:02:48.089054  6617 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:48.099166  6104 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:02:48.097329  6451 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.176308  6238 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.162807  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:48.179543  2297 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.171046  6268 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:47.892683  6534 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:47.872992  6604 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.057863  6531 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.196501  6617 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:48.207085  6377 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:48.108260  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 371558us
W20260709 13:02:48.171785  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65110.6us
W20260709 13:02:48.295001  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.296752  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:48.300580  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.301966  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:48.303454  2297 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:48.305179  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:48.312397  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.328460  6104 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:48.330129  6268 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:48.234134  6525 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.227557  6531 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:48.244611  6604 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:48.231609  6614 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:48.408149  1553 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:48.408821  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:48.216094  6615 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:02:48.411128  6268 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.231333  6617 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.412904  6630 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.414024  6630 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:48.415427  6489 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:48.416709  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 782) took 1230 ms. Trace:
I20260709 13:02:48.209805  6356 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.417227  1628 rpcz_store.cc:276] 0709 13:02:47.185629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.190545 (+  4916us) service_pool.cc:224] Handling call
0709 13:02:47.191228 (+   683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.191626 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.191658 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.219494 (+ 27836us) log.cc:844] Serialized 501 byte log entry
0709 13:02:47.966105 (+746611us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:47.966152 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.966229 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.415931 (+449702us) raft_consensus.cc:1685] finished
0709 13:02:48.415971 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.416372 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":706446,"threads_started":1,"wal-append.queue_time_us":850722}
I20260709 13:02:48.420142  2293 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:48.379879  6627 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:48.424778  6451 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.425294  6627 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.412984  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1968) took 1125 ms. Trace:
I20260709 13:02:48.429171  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.429697  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.430447  6104 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.433498  6630 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.384351  6534 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:48.416337  6489 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:48.377897  6350 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.007s
I20260709 13:02:48.434492  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1971) took 1061 ms. Trace:
I20260709 13:02:48.428537  1704 rpcz_store.cc:276] 0709 13:02:47.287087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.464428 (+177341us) service_pool.cc:224] Handling call
0709 13:02:47.464752 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.464967 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.465000 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.465437 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.049726 (+584289us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:48.049767 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.049827 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.412415 (+362588us) raft_consensus.cc:1685] finished
0709 13:02:48.412449 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.412699 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":583968,"threads_started":1,"wal-append.queue_time_us":643095}
W20260709 13:02:48.434417  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 245218us
W20260709 13:02:48.451818  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 155597us
I20260709 13:02:48.434273  6104 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.453567  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:48.448854  1709 rpcz_store.cc:276] 0709 13:02:47.372946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.661980 (+289034us) service_pool.cc:224] Handling call
0709 13:02:47.681300 (+ 19320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.098433 (+417133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.098463 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.103131 (+  4668us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.379933 (+276802us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:48.380076 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.380274 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.432292 (+ 52018us) raft_consensus.cc:1685] finished
0709 13:02:48.432312 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.432515 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":253641,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":3817,"spinlock_wait_cycles":101632,"thread_start_us":635469,"threads_started":2,"wal-append.queue_time_us":274927}
I20260709 13:02:48.455376  6630 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:48.452211  6604 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:48.456118  6356 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:48.435817  6526 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:48.440979  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 925) took 3982 ms. Trace:
I20260709 13:02:48.439595  6513 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:48.461706  2294 rpcz_store.cc:276] 0709 13:02:44.458420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.458567 (+   147us) service_pool.cc:224] Handling call
0709 13:02:45.513945 (+1055378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.635319 (+2121374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.439209 (+803890us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400770e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:48.440454 (+  1245us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400770e50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:48.440738 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7284767232}
I20260709 13:02:48.440047  6310 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:48.464293  6350 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.461748  6630 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.461400  6604 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:48.474864  6630 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:48.475221  6604 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.488261  6543 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:48.482650  6574 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.664s	user 0.005s	sys 0.022s
I20260709 13:02:48.482856  6268 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:48.512171  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.522935  6543 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.528760  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.529906  6492 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.534566  6535 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:48.533341  6627 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.537333  6535 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.537796  6535 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.532765  6513 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:48.540371  6513 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.541155  6543 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:48.543066  6513 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:48.551153  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122408us
I20260709 13:02:48.552688  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.554344  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.555020  6627 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:48.555178  6635 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:02:48.557197  6285 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:48.568257  6285 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:48.557857  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:48.561264  6356 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.562479  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1970) took 1274 ms. Trace:
I20260709 13:02:48.569628  6285 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.572870  6285 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:48.572566  6356 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.573508  1551 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:48.574930  6603 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:48.575103  6356 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.575390  6603 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.575881  6603 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:48.578795  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 954908us
I20260709 13:02:48.580602  6571 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:48.581929  6571 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:48.582387  6571 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:48.274520   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.571362  1708 rpcz_store.cc:276] 0709 13:02:47.287393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.479025 (+191632us) service_pool.cc:224] Handling call
0709 13:02:47.500530 (+ 21505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.063391 (+562861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.063441 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.080788 (+ 17347us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.399121 (+318333us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:48.404879 (+  5758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.561742 (+156863us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54003c1254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:48.561807 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.561900 (+    93us) raft_consensus.cc:1685] finished
0709 13:02:48.561923 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.562232 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":230234,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":331513728,"thread_start_us":486765,"threads_started":2,"wal-append.queue_time_us":101368}
I20260709 13:02:48.607908  6268 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:48.616976  5983 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:48.618127  5983 consensus_queue.cc:1243] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:48.620293  6513 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:48.621280  6513 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.629536  6322 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:48.630146  6322 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:48.633599  5640 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.634779  6076 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.015s
I20260709 13:02:48.634204  6322 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.636281  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.637174  1553 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:48.637594  6513 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:48.638055  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.638460  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.638715  6492 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:48.606158  6356 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:48.634218  5640 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.639432  6492 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.642318  1703 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:02:48.640568  1555 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:48.643244  6492 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:48.640513  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86313.5us
I20260709 13:02:48.641110  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.648360  6356 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.636298  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:48.660429  6602 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.921s	user 0.005s	sys 0.000s
I20260709 13:02:48.662731  6356 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:48.663968  6070 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.774s	user 0.022s	sys 0.005s
W20260709 13:02:48.668817  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.671097  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2541) took 1507 ms. Trace:
I20260709 13:02:48.635299  6076 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.671558  1634 rpcz_store.cc:276] 0709 13:02:47.163293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.165669 (+  2376us) service_pool.cc:224] Handling call
0709 13:02:47.165967 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.169578 (+  3611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.169623 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.169995 (+   372us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.291097 (+121102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:47.291156 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.291250 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.659170 (+1367920us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:48.668491 (+  9321us) raft_consensus.cc:1685] finished
0709 13:02:48.668519 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.670867 (+  2348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3229,"raft.queue_time_us":14204,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":4122368,"thread_start_us":110985,"threads_started":1,"wal-append.queue_time_us":558737}
I20260709 13:02:48.594012  6571 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.647877  6604 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.676616  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:48.674922  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:48.686959  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72117.5us
I20260709 13:02:48.690187  6571 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.690402  6070 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:48.691149  6070 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:48.692147  6426 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.692502  6070 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.692915  6426 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.694376  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.695417  6426 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.695569  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.696300  5912 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:48.698451  6076 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:48.699054  6559 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.013s
I20260709 13:02:48.698987  6489 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:48.700242  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.700970  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:48.701936  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:48.696321  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.702226  6526 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Committing config change with OpId 10.8: config changed from index 3 to 8, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:48.704176  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:48.704303  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:48.707882  6553 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:48.709311  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.710637  6604 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.716046  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.721093  2295 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:48.685303  6322 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:48.728257  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:48.720314  2263 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.736s (SENT) [suppressed 26 similar messages]
I20260709 13:02:48.736598  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.742688  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:48.746424  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 163291us
W20260709 13:02:48.750984  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.752178  1559 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:48.754812  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.759905  2289 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:48.721946  6630 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.771886  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.773303  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:48.773615  1558 raft_consensus.cc:1217] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Deduplicated request from leader. Original: 10.7->[11.8-11.8]   Dedup: 11.8->[]
W20260709 13:02:48.774891  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.726182  6559 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.777169  1558 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:48.776010  2288 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:02:48.781211  1555 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:48.782428  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.783349  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:02:48.784106  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 113173us
I20260709 13:02:48.784801  6648 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:48.785159  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:48.786090  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2546) took 1572 ms. Trace:
I20260709 13:02:48.787206  1554 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:48.793560  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.795389  6076 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.796126  6076 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.798734  6076 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.798919  2292 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.796144  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:48.801036  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:48.802611  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.792490  1558 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:48.804633  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.806120  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:48.806401  1708 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:48.806911  1702 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:48.808883  1558 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:48.809520  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.810304  6550 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.006s	sys 0.010s
I20260709 13:02:48.809561  6535 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.810679  6550 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.007s	sys 0.010s
I20260709 13:02:48.811396  1557 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:48.814103  6603 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.815203  6535 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.839619  6621 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:48.840199  6621 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:48.815550  6598 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:48.841383  6621 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.819100  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:48.766716  6426 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.811349  1558 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:48.820160  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1972) took 1339 ms. Trace:
I20260709 13:02:48.858132  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:48.820017  6648 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.822449  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:48.825085  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:48.867285  6495 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:48.867982  6495 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:48.868530  6495 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.829186  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:48.829912  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:48.815546  5640 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.832455  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1980) took 1349 ms. Trace:
I20260709 13:02:48.881567  1701 rpcz_store.cc:276] 0709 13:02:47.482961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.898392 (+415431us) service_pool.cc:224] Handling call
0709 13:02:47.898676 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.283523 (+384847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.283644 (+   121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.283758 (+   114us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:48.283851 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.832016 (+548165us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54003e2454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:48.832062 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.832253 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882816896}
I20260709 13:02:48.833668  5983 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:48.883651  5983 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:48.884357  5983 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.835321  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.889374  1559 raft_consensus.cc:2410] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:02:48.806456  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.859321  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.891860  1553 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:48.893816  5640 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.894687  5640 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.899575  5640 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:48.909554  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:48.912501  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.913775  5640 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:48.915315  5640 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:02:48.922765  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.917580  5640 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.943392  1706 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:48.944149  6350 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:48.842402  6598 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:48.945942  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123586us
I20260709 13:02:48.946969  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:48.947855  6322 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:48.790927  1633 rpcz_store.cc:276] 0709 13:02:47.213162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.276119 (+ 62957us) service_pool.cc:224] Handling call
0709 13:02:47.276439 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.276652 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:47.283991 (+  7339us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.749697 (+1465706us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.754531 (+  4834us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:48.763189 (+  8658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.775069 (+ 11880us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.785913 (+ 10844us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1026819,"prepare.queue_time_us":7602,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":998027,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":260,"thread_start_us":452144,"threads_started":2,"wal-append.queue_time_us":520244}
I20260709 13:02:48.948742  6617 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.948451  6322 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.943430  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:48.949823  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83723us
I20260709 13:02:48.935931  5912 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.940848  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2584) took 3092 ms (client timeout 1578 ms). Trace:
I20260709 13:02:48.964984  6519 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:48.965737  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.965046  1700 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:48.959182  6076 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.964707  6426 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.965519  5912 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:48.965155  6652 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:48.966971  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:48.977653  1706 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.858664  1707 rpcz_store.cc:276] 0709 13:02:47.480868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.678182 (+197314us) service_pool.cc:224] Handling call
0709 13:02:47.678585 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.077130 (+398545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.077165 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.077668 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.285044 (+207376us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:48.364437 (+ 79393us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.388693 (+ 24256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.819581 (+430888us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b540010d1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:48.819629 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.819680 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:48.819826 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.820003 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138086,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":226630,"raft.run_cpu_time_us":609,"raft.run_wall_time_us":29685,"spinlock_wait_cycles":107905920,"thread_start_us":635065,"threads_started":3,"wal-append.queue_time_us":160208}
W20260709 13:02:48.962633  1703 rpcz_store.cc:269] 0709 13:02:45.838667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.850774 (+ 12107us) service_pool.cc:224] Handling call
0709 13:02:48.847737 (+2996963us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b5400165b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:48.931368 (+ 83631us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6337282304}
I20260709 13:02:48.959081  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:48.956776  6553 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.949349  6617 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.975478  6621 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:48.964228  6598 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.990083  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:48.993939  1705 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.994081  6355 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.972025  6076 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:48.991927  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:48.998816  1557 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:02:49.002161  1706 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:49.002743  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.004118  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.004752  1558 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:49.005089  1551 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:49.005894  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.006269  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.007019  1552 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:49.015475  6616 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.016377  6616 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.018745  6322 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.023561  6560 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:49.025424  5912 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:49.032616  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 213662us
I20260709 13:02:49.032346  6648 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:48.989266  5465 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:02:49.034870  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.035418  6630 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.035820  1559 raft_consensus.cc:2410] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:02:49.032356  6570 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:49.037228  1706 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:49.038285  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.039561  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:49.040482  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.040860  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.041425  6426 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.041862  1706 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:49.042692  6559 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.042811  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.043988  6559 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:49.044149  6652 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.044137  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.044680  6559 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.045010  6652 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.045420  1703 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:49.046380  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.046455  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.047020  6652 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.047063  1703 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:49.047596  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:49.048266  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.049274  1703 raft_consensus.cc:2410] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:49.049852  6617 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.050630  6553 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.004173  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.052682  1555 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:02:49.052763  1633 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:49.053179  6553 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.054581  5465 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:49.055174  5465 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.057554  5465 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.058259  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.058900  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:49.060865  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.061218  1703 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:49.062464  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.062980  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.064303  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.064599  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.065500  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.065915  1701 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:49.066051  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.066721  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.067832  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.068534  1709 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:49.068748  1700 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:49.063089  1706 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:49.069782  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.070452  1707 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:49.071877  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:49.072500  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.073120  1559 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:49.073194  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.073896  1553 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:49.074103  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:49.074357  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.074651  6616 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.071877  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:49.075445  6076 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:49.075755  1700 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.079360  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.088099  2288 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 13:02:49.091027  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49372.8us
I20260709 13:02:49.094590  2291 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:49.097657  6355 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:49.099948  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.099257  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.101934  2293 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:49.046932  6559 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.120026  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.048277  6603 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:49.128290  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66880.2us
I20260709 13:02:49.129277  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.134075  1704 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:49.162621  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.163878  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.164598  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.165977  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:49.168038  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 221198us
I20260709 13:02:49.172616  2292 raft_consensus.cc:2393] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:49.180218  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.165608  6553 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:02:49.186437  2293 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:49.190279  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.191828  6540 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:49.192575  6540 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.191682  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.194351  2291 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:49.194895  6540 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.195309  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.137332  5912 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:02:49.196457  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.196535  6534 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:49.197034  2291 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.197141  6534 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:49.198374  6534 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.200321  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.200992  2295 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.201211  1454 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:49.204428  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:49.204843  6322 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:49.205825  6544 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:49.206271  6544 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.206708  6544 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:49.207585  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.208495  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.209168  1701 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:49.209502  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.210326  6652 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:49.205494  6322 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.210971  6652 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.212001  6533 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.212146  6525 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:49.212586  6533 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.212801  6322 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.214038  6559 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.214598  6559 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.214955  6533 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.215708  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.216787  6559 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:49.217355  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.219879  6620 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.220108  6559 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.220549  2290 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:49.220505  6620 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:49.220697  6559 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.221899  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.222498  2290 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:49.222819  6620 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:49.223151  6544 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.223834  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.213173  6652 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.224754  1632 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:49.213728  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:49.227239  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 96743.3us
I20260709 13:02:49.223958  6533 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:49.223451  6559 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.230967  1629 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:49.231488  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.232611  1701 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:49.234611  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.164078  6617 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:02:49.235910  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.235419  1703 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 3 to 8, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:49.236522  1704 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:49.238897  1700 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:49.239174  1707 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:02:49.239512  1707 pending_rounds.cc:85] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 9
I20260709 13:02:49.239722  1707 pending_rounds.cc:107] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:02:49.240015  1707 raft_consensus.cc:2889] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.242645  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.244037  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.244800  2293 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.247720  1705 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:49.250654  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.256635  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.256639  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.257458  1633 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 9.
I20260709 13:02:49.259716  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.229929  6544 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.264933  2295 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:02:49.245463  6489 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.270648  1701 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:49.264935  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.281327  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.264209  1454 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 8 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:49.293524  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.264142  6534 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:49.297379  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 210) took 20272 ms (client timeout 15000 ms). Trace:
I20260709 13:02:49.298071  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.299001  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.305022  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.306025  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:49.267104  6526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50166 (request call id 10) took 6702 ms. Trace:
I20260709 13:02:49.262022  5912 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:49.308804  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.313644  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.272572  6540 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:49.309068  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.315171  1632 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.344777  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:02:49.314714  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.357158  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.357937  2292 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:49.356467  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46370.7us
I20260709 13:02:49.366662  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:49.277969  6559 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:02:49.277084  6533 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.383971  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.387496  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.293681  6489 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.392154  6355 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:49.392205  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.420176  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.421015  2295 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:49.391381  1628 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:49.394191  6592 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:02:49.396555  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.396875  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:49.400542  6665 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:49.401177  1701 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:49.401796  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.426678  2292 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:49.402400  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.403230  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:49.428447  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61643.3us
I20260709 13:02:49.429100  1700 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:49.401201  6603 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:02:49.404148  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:49.407533  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.415940  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:49.370334  6495 consensus_meta.cc:287] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.005s
I20260709 13:02:49.420290  1631 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:49.393414  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:49.423969  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.426827  1635 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:49.443760  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50436.3us
I20260709 13:02:49.446120  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.446756  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:49.447330  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88025.3us
I20260709 13:02:49.447520  6652 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:49.449596  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.450022  6544 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:49.450618  6544 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.450351  6495 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.451146  6544 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.452673  6495 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.454362  6525 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:49.455173  6553 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:49.455585  6553 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.455984  6553 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:49.456161  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 159667us
I20260709 13:02:49.380221  6616 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.383493  6550 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:49.447381  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
W20260709 13:02:49.462962  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.466732  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.388852  6540 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:49.470829  6350 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:49.474565  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.475042  1454 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 10, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 10 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:49.476354  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 181245us
I20260709 13:02:49.484850  6350 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:49.485586  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.310542  6526 rpcz_store.cc:276] 0709 13:02:42.551151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.163757 (+612606us) service_pool.cc:224] Handling call
0709 13:02:43.300272 (+136515us) log.cc:844] Serialized 501 byte log entry
0709 13:02:45.130106 (+1829834us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b4c00221900. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cccb0a20 00007f70cccc1a29 00007f70cccc213f 00007f70ccc862ab 00007f70cccb3f3c 00007f70cccb43fe 00007f70ccf4150b 00007f70ce940381 00007f70ce93e26b 00007f70ce944d53 00007f70ce944cd9
0709 13:02:49.232545 (+4102439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71539,"raft.queue_time_us":159704,"raft.run_cpu_time_us":4346,"raft.run_wall_time_us":1270925,"spinlock_wait_cycles":3596032,"thread_start_us":758241,"threads_started":2,"wal-append.queue_time_us":736024,"wal-append.run_cpu_time_us":1742,"wal-append.run_wall_time_us":1025575}
I20260709 13:02:49.477046  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.477720  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.480671  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.490451  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:49.494046  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49209.1us
I20260709 13:02:49.494323  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:49.501223  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51733.9us
I20260709 13:02:49.506837  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.508112  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.510574  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.383476  5912 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.483712  6540 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.520197  1628 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:49.511584  2291 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:49.520821  2295 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:49.461117  6620 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:02:49.528487  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50879us
I20260709 13:02:49.525763  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.485889  6350 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.536372  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.536393  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:49.539392  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53893.4us
I20260709 13:02:49.541870  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:49.509868  6550 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:49.525290  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:49.536077  1629 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:49.512936  6650 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:49.556280  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.518157  6519 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:49.552485  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.517792  6592 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:49.565732  1701 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.562036  1628 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:49.560027  1632 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:49.481683  6616 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.544646  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.563107  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.527908  6540 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.567137  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.574156  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:49.574452  1700 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:49.568779  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:49.568954  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.569324  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:49.569581  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 125103us
I20260709 13:02:49.582510  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.564906  6460 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:49.555922  6355 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:49.523979  6285 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:02:49.588186  1702 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:49.593861  6603 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:49.595623  6489 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.569967  6592 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.594717  1454 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 8 to 9, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 9 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:49.595785  5912 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:49.610965  6592 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:49.611231  6669 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.147s	user 0.000s	sys 0.002s
W20260709 13:02:49.611778  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45889.3us
I20260709 13:02:49.602327  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.647284  1632 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:02:49.616050  2290 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:49.605732  1631 raft_consensus.cc:2955] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 3 to 8, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:49.536038  6310 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:02:49.606560  6571 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.916s	user 0.007s	sys 0.006s
W20260709 13:02:49.659920  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51022.5us
I20260709 13:02:49.602826  6350 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.662102  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:49.663363  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:49.664340  6621 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:49.664881  6621 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:49.666086  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.666699  6070 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:49.667192  6070 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:49.667434  6616 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.668088  6070 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.669227  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.670499  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.671868  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.589265  6460 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:49.600426  6519 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:49.644403  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 186630us
I20260709 13:02:49.611697  1629 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:49.651115  6666 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:02:49.633100  6598 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:49.710482  6598 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:49.712110  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:49.711833  6598 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:49.653348  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 91986.3us
I20260709 13:02:49.713805  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.714097  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:49.300912  1455 rpcz_store.cc:269] 0709 13:02:28.918221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.918409 (+   188us) service_pool.cc:224] Handling call
0709 13:02:49.190742 (+20272333us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:39.820093 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:39.849307 (+ 29214us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:39.849398 (+    91us) write_op.cc:435] Acquired schema lock
0709 13:02:39.849433 (+    35us) tablet.cc:662] Decoding operations
0709 13:02:39.861081 (+ 11648us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:39.861199 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:39.861234 (+    35us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:02:40.385616 (+524382us) tablet.cc:701] Row locks acquired
0709 13:02:40.454019 (+ 68403us) write_op.cc:260] PREPARE: finished
0709 13:02:40.505486 (+ 51467us) write_op.cc:270] Start()
0709 13:02:40.549545 (+ 44059us) write_op.cc:276] Timestamp: P: 1783602160479713 usec, L: 0
0709 13:02:40.561693 (+ 12148us) op_driver.cc:348] REPLICATION: starting
0709 13:02:40.688238 (+126545us) log.cc:844] Serialized 8382 byte log entry
0709 13:02:40.809067 (+120829us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70cb8da0ac 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b
0709 13:02:43.311892 (+2502825us) op_driver.cc:464] REPLICATION: finished
0709 13:02:43.928797 (+616905us) write_op.cc:301] APPLY: starting
0709 13:02:43.928896 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:44.636670 (+707774us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:44.636722 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:45.588904 (+952182us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:45.589286 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:02:45.589331 (+    45us) write_op.cc:312] APPLY: finished
0709 13:02:45.589800 (+   469us) log.cc:844] Serialized 188 byte log entry
0709 13:02:45.759127 (+169327us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:45.785706 (+ 26579us) write_op.cc:454] Released schema lock
0709 13:02:45.801550 (+ 15844us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:45.802016 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:45.802050 (+    34us) write_op.cc:454] Released schema lock
0709 13:02:46.761170 (+959120us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
Metrics: {"mutex_wait_us":183698,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":555105,"apply.run_cpu_time_us":56983,"apply.run_wall_time_us":1873187,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91991,"num_ops":21,"prepare.queue_time_us":254155,"prepare.run_cpu_time_us":37769,"prepare.run_wall_time_us":1668959,"raft.queue_time_us":552484,"raft.run_cpu_time_us":24100,"raft.run_wall_time_us":313312,"replication_time_us":2733784,"spinlock_wait_cycles":51214976,"thread_start_us":1729110,"threads_started":5,"wal-append.queue_time_us":1403171,"wal-append.run_cpu_time_us":37610,"wal-append.run_wall_time_us":4339012}]]}
I20260709 13:02:49.665819  6621 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.675041  2292 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:02:49.718662  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.003s
W20260709 13:02:49.719137  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.015s	sys 0.003s
I20260709 13:02:49.719691  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.720126  6550 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:49.720640  1628 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:49.642580  6526 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:49.721525  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.722844  6526 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:49.725821  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.728116  6526 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:49.730031  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:49.732249  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.733923  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.734046  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.734795  1630 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:02:49.731671  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.737072  1633 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:02:49.741698  6598 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.609048  6651 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:02:49.746091  6533 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.748660  6519 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:49.749557  5983 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.864s	user 0.012s	sys 0.003s
I20260709 13:02:49.719007  2294 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:49.642819  6489 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:02:49.749564  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:49.732169  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:49.705570  6106 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:02:49.713910  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2555) took 1018 ms. Trace:
I20260709 13:02:49.712222  1454 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 8 to 12, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:49.744120  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:49.781116  2293 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.001s
I20260709 13:02:49.786370  2293 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:49.788857  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.702060  6460 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.793527  1632 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.795670  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.791054  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:49.782668  1560 rpcz_store.cc:276] 0709 13:02:48.712558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.748990 (+ 36432us) service_pool.cc:224] Handling call
0709 13:02:48.763566 (+ 14576us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:48.858174 (+ 94608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.858213 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:48.949482 (+ 91269us) log.cc:844] Serialized 52 byte log entry
0709 13:02:49.013256 (+ 63774us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b18004281a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70d28b858f 00007f70cb8f7571 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd
0709 13:02:49.245431 (+232175us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:49.245475 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.245539 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.713543 (+468004us) raft_consensus.cc:1685] finished
0709 13:02:49.713565 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.713812 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46650,"raft.queue_time_us":55891,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":78182,"spinlock_wait_cycles":8951168,"thread_start_us":255701,"threads_started":1,"wal-append.queue_time_us":518230}
I20260709 13:02:49.798111  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:49.800477  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.794229  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:49.805068  1701 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:49.806715  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.806617  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.812490  1629 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:49.784500  6322 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.009s	sys 0.026s
I20260709 13:02:49.782301  6651 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.786422  6526 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:49.801462  1454 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 5, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 5 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:49.841763  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47647.2us
I20260709 13:02:49.826805  1627 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:49.846885  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.860033  1443 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:02:49.841653  6534 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:02:49.843464  6322 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.010s	sys 0.026s
I20260709 13:02:49.824215  5912 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:49.824906  6106 consensus_queue.cc:1243] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:49.862108  5912 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.862727  5912 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.868829  6534 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.869642  6534 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.875114  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.884317  2292 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:49.886516  6495 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:49.887229  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:49.888813  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.890188  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:49.892293  6675 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.241s	user 0.005s	sys 0.002s
I20260709 13:02:49.895265  2292 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:49.750326  5983 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.814203  6525 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:49.898921  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.899978  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.807710  6620 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:49.905776  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:49.916923  1701 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:49.920272  1632 raft_consensus.cc:2410] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:49.849659  6559 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:49.927501  1627 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:49.928402  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.931499  6592 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.915371  6525 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.932691  6525 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:49.933789  6592 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.934481  5912 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.854794  6651 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.936103  6534 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:49.937188  5983 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.937704  6651 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:49.916517  6620 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:49.938974  6620 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.940112  6559 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:49.941277  6559 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:49.942253  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.944072  1551 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:49.944782  1634 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:49.948952  2291 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:49.953049  6671 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 13:02:49.953935  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 357017us
I20260709 13:02:49.957254  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.964156  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:49.965150  1632 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:49.978613  6540 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:49.978891  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:49.979822  1559 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:49.980088  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:49.980698  2293 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:49.981109  6534 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:49.981525  6534 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:49.981911  6534 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.984112  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:49.984863  1627 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.992740  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.993592  6076 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:49.994822  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:49.995297  6076 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:50.003235  6544 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.004s
I20260709 13:02:50.003618  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:50.004951  6553 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
I20260709 13:02:50.005682  5912 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:49.958801  6571 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:02:50.013468  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 94950.1us
I20260709 13:02:50.014716  6540 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.015358  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.022580  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.024056  6650 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:50.024945  6495 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:50.025759  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.029743  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.031729  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:50.032948  1634 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:49.953073  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:50.038795  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:50.044723  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.013s	sys 0.015s
W20260709 13:02:50.045500  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.014s	sys 0.015s
I20260709 13:02:50.088294  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.092206  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:50.053294  6426 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:50.105217  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 82750.1us
I20260709 13:02:50.037381  6355 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:50.107942  6679 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.457s	user 0.009s	sys 0.005s
I20260709 13:02:50.108240  1634 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:50.108996  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.109907  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:50.117075  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.031375  6544 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.031718  6553 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.130337  6620 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.104718  6592 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:50.143615  6534 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:50.120285  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.154050  1626 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.157459  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:50.140524  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 109292us
I20260709 13:02:50.176503  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:50.162971  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.165009  2296 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:02:50.167826  2295 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:50.182058  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.185491  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.185832  1630 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:50.081756  6460 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.188344  2297 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 13:02:50.202260  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:50.202001  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.203198  1634 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:50.182044  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.023s
W20260709 13:02:50.206629  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.015s	sys 0.023s
I20260709 13:02:50.207453  1627 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:02:50.168062  6603 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:50.214835  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.182819  6426 raft_consensus.cc:2764] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:02:50.204054  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.013s	sys 0.016s
W20260709 13:02:50.220021  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.013s	sys 0.016s
I20260709 13:02:50.187911  6616 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:50.229558  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:50.249166  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 704526us
I20260709 13:02:50.250077  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.250926  1627 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:50.252820  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 824) took 1038 ms. Trace:
W20260709 13:02:50.212211  6617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.007s	sys 0.019s
I20260709 13:02:50.257076  1628 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:50.267167  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:50.267944  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64873.9us
I20260709 13:02:50.216418  6603 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:50.219046  6592 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:50.202270  6489 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:50.257455  6076 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:50.281627  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.279301  6603 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:50.287734  6553 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.291049  6544 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:50.267432  6355 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.216725  6525 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:50.261693  6617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.008s	sys 0.020s
I20260709 13:02:50.282472  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.311039  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.306133  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:50.253393  1704 rpcz_store.cc:276] 0709 13:02:49.214074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.237164 (+ 23090us) service_pool.cc:224] Handling call
0709 13:02:49.237537 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:49.237971 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.237999 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:49.238518 (+   519us) log.cc:844] Serialized 52 byte log entry
0709 13:02:49.631218 (+392700us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:50.223264 (+592046us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.251735 (+ 28471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.251970 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.252158 (+   188us) raft_consensus.cc:1685] finished
0709 13:02:50.252179 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.252739 (+   560us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30615,"prepare.queue_time_us":253073,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":7803,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8109,"thread_start_us":841650,"threads_started":2,"wal-append.queue_time_us":610408}
W20260709 13:02:50.312112  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.010s	sys 0.030s
W20260709 13:02:50.316716  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.010s	sys 0.031s
W20260709 13:02:50.317047  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 681324us
I20260709 13:02:50.297418  6592 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.317948  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:50.281461  6621 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.324213  1634 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:50.326187  1634 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:50.305198  6076 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.339593  2292 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:50.340255  1634 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:50.348081  1634 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:50.358675  1629 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:50.342723  6076 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.373400  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.384421  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:50.338967  6533 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:02:50.360306  6460 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:50.359634  6690 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:50.400410  6650 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:02:50.408792  6525 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.410426  6648 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:50.412895  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.398554  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.375950  6519 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.429605  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 118654us
I20260709 13:02:50.436220  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:50.436650  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:50.436946  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.437460  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:50.438148  1701 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:50.395258  6621 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:50.425665  6690 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.451534  1701 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:50.451819  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:50.462886  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64437us
I20260709 13:02:50.426568  6534 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.467953  1631 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:50.471341  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:50.472675  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 98071.7us
I20260709 13:02:50.480754  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:50.481768  1706 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.431836  6648 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:50.483414  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70628.9us
W20260709 13:02:50.490140  6685 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:02:50.490646  6544 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:50.491092  6691 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:02:50.492878  6698 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:02:50.494405  6699 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:02:50.473453  6690 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:50.496443  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44751.4us
I20260709 13:02:50.497615  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.498850  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.466511  6070 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:50.499944  6560 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:50.501334  6560 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:50.444104  6702 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:50.494447  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2635) took 1091 ms. Trace:
I20260709 13:02:50.502338  6702 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.502759  6702 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.503113  6571 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:50.503571  6571 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:50.496552  2296 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:50.502485  1707 rpcz_store.cc:276] 0709 13:02:49.403498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.564920 (+161422us) service_pool.cc:224] Handling call
0709 13:02:49.565308 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.593069 (+ 27761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.593102 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.593735 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.731710 (+137975us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:02:50.027858 (+296148us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:50.027902 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.027983 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.494126 (+466143us) raft_consensus.cc:1685] finished
0709 13:02:50.494147 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.494386 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10781,"raft.run_cpu_time_us":2157,"raft.run_wall_time_us":138724,"spinlock_wait_cycles":2838400,"thread_start_us":433653,"threads_started":1,"wal-append.queue_time_us":700115}
I20260709 13:02:50.505882  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.508513  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.513052  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:02:50.515002  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.514451  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:50.432780  6519 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.512027  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.505275  5912 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:50.529834  1706 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:50.491160  6544 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.534168  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.511134  6525 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:50.542757  6076 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.543321  6648 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.544430  6648 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:50.545027  6687 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:50.545236  6076 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:50.545653  6687 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:50.547266  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.549121  6525 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:50.550417  6616 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.551167  6616 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:50.551831  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 1997) took 1152 ms. Trace:
W20260709 13:02:50.552275  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 346935us
I20260709 13:02:50.552484  6616 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:50.553036  6616 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.553261  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.555140  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.556332  6689 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:50.559064  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 279093us
I20260709 13:02:50.555938  6692 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:50.559906  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 241667us
I20260709 13:02:50.560335  6692 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:50.504472  6571 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.560949  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.553412  6616 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:50.564024  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.520207  6560 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:50.518636  6705 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:50.552219  1709 rpcz_store.cc:276] 0709 13:02:49.399440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.401615 (+  2175us) service_pool.cc:224] Handling call
0709 13:02:49.401914 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.605652 (+203738us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:50.043642 (+437990us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.391246 (+347604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.392045 (+   799us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:50.392124 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.551088 (+158964us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18004c10e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419
0709 13:02:50.551613 (+   525us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400133dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:50.551656 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.551759 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":278467,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":638,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":28286,"spinlock_wait_cycles":472292352,"thread_start_us":701663,"threads_started":2,"wal-append.queue_time_us":421928}
I20260709 13:02:50.538549  6544 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.559322  6688 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
W20260709 13:02:50.606650  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 89127.3us
I20260709 13:02:50.568454  6692 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:50.623679  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63901.2us
I20260709 13:02:50.578094  6076 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:50.626143  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:50.627332  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 93276.2us
I20260709 13:02:50.628708  2295 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:50.629354  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114977us
W20260709 13:02:50.629480  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68679.4us
I20260709 13:02:50.600909  6534 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:50.615095  6571 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.579294  6350 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:02:50.622262  6652 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.634845  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:50.639951  2289 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:02:50.637369  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83278.5us
I20260709 13:02:50.612941  6692 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.631446  6534 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:50.666385  6718 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:50.667110  6718 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.669745  6718 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:50.673610  6690 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:50.675812  6704 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:02:50.676309  6534 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.677397  6534 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.677628  6693 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.678171  6693 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.680414  6693 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:50.681442  6717 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:50.683207  6620 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.004s
W20260709 13:02:50.689255  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46514.8us
I20260709 13:02:50.690188  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.700008  6495 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:50.700209  6725 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:50.700551  6495 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:50.701013  6495 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.705545  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:50.707479  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53672.9us
I20260709 13:02:50.711637  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:50.633152  6076 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.717279  5912 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:50.727596  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.734616  6719 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:50.649632  6687 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.744478  6603 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:50.748322  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.715528  6725 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:50.757499  1628 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:50.759877  6718 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.663225  6728 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:50.762142  6728 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:50.764950  6701 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.332s	user 0.003s	sys 0.000s
I20260709 13:02:50.770385  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.771977  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.772754  1630 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.776693  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:50.781401  1626 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.005s
I20260709 13:02:50.781782  1626 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:50.790936  1635 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:02:50.795120  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.779311  6070 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:50.783684  6728 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:50.821314  1701 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:50.827569  1704 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:50.829082  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:50.795498  6519 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:50.831611  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 267665us
I20260709 13:02:50.784835  6621 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:02:50.864045  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 116741us
I20260709 13:02:50.864941  1630 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:02:50.866945  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.862937  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.820519  5912 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.874090  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:50.875221  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.874804  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.876855  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.877127  1632 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.878860  6426 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:50.879729  6426 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:50.880221  6426 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:50.881181  6426 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:50.912211  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:50.917337  6746 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:50.917934  6746 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:50.848207  6725 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:50.886484  6702 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:50.923463  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 211883us
I20260709 13:02:50.913895  6621 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:50.929165  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.919114  6746 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.935237  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.936617  1632 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:50.938655  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:50.945955  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.953306  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:50.943491  6070 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:50.956432  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:50.957001  1632 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:50.950316  6621 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:50.949751  6746 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.952104  6750 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:50.974334  1632 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:50.980408  6692 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:50.984428  6692 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.985816  6702 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:50.963629  6603 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:50.959651  6748 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.975137  6426 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.007493  6535 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:51.008056  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 381948us
I20260709 13:02:51.008505  6535 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:51.009246  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:50.999199  6748 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:51.010910  6748 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:51.012679  6350 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:51.016721  6700 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 13:02:51.007602  6736 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:51.011462  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 381167us
I20260709 13:02:51.021018  6598 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:51.022660  6697 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.633s	user 0.000s	sys 0.003s
I20260709 13:02:51.010821  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:51.008975  6535 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:51.027861  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 171188us
I20260709 13:02:51.029477  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:51.041109  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87904.2us
I20260709 13:02:51.026437  6350 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:51.042142  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.023445  6598 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:51.055599  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:51.050761  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2561) took 1112 ms. Trace:
I20260709 13:02:51.059182  1627 rpcz_store.cc:276] 0709 13:02:49.938009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.284516 (+346507us) service_pool.cc:224] Handling call
0709 13:02:50.309969 (+ 25453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.327962 (+ 17993us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:50.357894 (+ 29932us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.578171 (+220277us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.908073 (+329902us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b18003a6960. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:51.050214 (+142141us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18003a6960. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:51.050372 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.050430 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:51.050487 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.050571 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.050699 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78161,"prepare.queue_time_us":30687,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":562005,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":21879,"spinlock_wait_cycles":15215104,"thread_start_us":100733,"threads_started":1,"wal-append.queue_time_us":156858}
I20260709 13:02:51.051908  6322 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.038591  6535 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:51.052238  6739 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:51.052492  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:51.052914  6534 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.053287  6747 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.074371  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.054129  6592 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.040108  6355 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:51.036230  6495 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.047324  6350 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:51.088629  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 460466us
I20260709 13:02:51.070427  6688 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:51.090714  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:51.082073  2297 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
W20260709 13:02:51.087873  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.091457  2297 pending_rounds.cc:85] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 9
I20260709 13:02:51.091884  2297 pending_rounds.cc:107] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:02:51.092201  2297 raft_consensus.cc:2889] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.094261  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.095069  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2576) took 1105 ms. Trace:
I20260709 13:02:51.095489  1631 rpcz_store.cc:276] 0709 13:02:49.989768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.486524 (+496756us) service_pool.cc:224] Handling call
0709 13:02:50.516378 (+ 29854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.560006 (+ 43628us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:50.560321 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.094717 (+534396us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.094768 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:51.094822 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.094894 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.095008 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":456864,"prepare.queue_time_us":758,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2202,"thread_start_us":77115,"threads_started":1,"wal-append.queue_time_us":177011}
W20260709 13:02:51.098867  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46464.1us
I20260709 13:02:51.102558  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 1949) took 1434 ms. Trace:
I20260709 13:02:51.107484  1709 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:51.108053  1557 rpcz_store.cc:276] 0709 13:02:49.660434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.670260 (+  9826us) service_pool.cc:224] Handling call
0709 13:02:51.094640 (+1424380us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:51.094780 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2694891904}
I20260709 13:02:51.085729  6621 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:51.129420  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.094208  6736 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.096155  6688 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.134660  6736 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.136101  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2573) took 1147 ms. Trace:
I20260709 13:02:51.083528  6534 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.156980  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.157483  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.136935  6756 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.135933  6738 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:51.134902  6598 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:51.113266  6426 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.135461  6754 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.109814  6734 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:51.097496  6460 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:51.152669  1634 rpcz_store.cc:276] 0709 13:02:49.988790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.352866 (+364076us) service_pool.cc:224] Handling call
0709 13:02:50.353236 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.360747 (+  7511us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:50.617437 (+256690us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.135647 (+518210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.135709 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:51.135755 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.135868 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.136027 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":348672,"prepare.queue_time_us":35881,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":341498,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":110,"thread_start_us":383200,"threads_started":2,"wal-append.queue_time_us":236360}
W20260709 13:02:51.159297  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68657.6us
I20260709 13:02:51.146265  6693 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:51.159838  5912 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.160036  6106 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:51.159865  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57605.2us
I20260709 13:02:51.175820  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.204998  1627 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 10: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:51.204869  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:51.161993  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43936.1us
I20260709 13:02:51.177802  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.206069  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:51.210738  2295 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:51.139369  6688 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.152287  6355 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:51.130565  6553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.027s	sys 0.024s
I20260709 13:02:51.180418  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.110530  6652 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:02:51.185424  6544 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.575s	user 0.012s	sys 0.001s
I20260709 13:02:51.181190  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.183102  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.083316  6747 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.184307  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.142115  6519 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.185825  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 91639.5us
I20260709 13:02:51.164831  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.176754  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.214087  1630 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:51.214286  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.217599  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.184595  6070 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.182199  6598 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:51.181115  6617 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.187424  6734 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.160257  6720 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:02:51.377365  6720 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:51.377804  6720 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:51.378203  1707 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 13:02:51.378615  6720 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:51.379455  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 128855us
I20260709 13:02:51.379995  6737 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:51.380870  6737 consensus_queue.cc:1243] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:51.381317  6737 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:51.381639  6737 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:51.382452  6737 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.383026  6076 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:51.351195  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.201125  6460 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.202211  6736 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.189191  6693 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:51.388841  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2718) took 1359 ms. Trace:
I20260709 13:02:51.397758  2295 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:02:51.413580  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.217963  6553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.028s	sys 0.025s
I20260709 13:02:51.200337  6603 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.361858  2293 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:51.370970  6652 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:51.372634  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:51.185999  6738 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.252140  6718 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.190096  6426 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.383049  6734 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.357146  1634 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:51.356285  1629 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
W20260709 13:02:51.389353  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 334222us
I20260709 13:02:51.482939  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:51.252586  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46620.9us
W20260709 13:02:51.453507  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 331921us
I20260709 13:02:51.484095  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.485247  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:51.256767  6616 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.000s
I20260709 13:02:51.454602  1635 rpcz_store.cc:276] 0709 13:02:50.029924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.518093 (+488169us) service_pool.cc:224] Handling call
0709 13:02:50.537404 (+ 19311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.791397 (+253993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.385285 (+593888us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400343fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:51.388773 (+  3488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572398848}
I20260709 13:02:51.486439  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.375000  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:51.334865  6513 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.468803  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.470685  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.502203  1631 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:51.364969  6598 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:51.216266  6648 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.244s
I20260709 13:02:51.249660  6544 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.201036  6748 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.384454  6737 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.511516  6738 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.547987  6534 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.548614  6426 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:51.548794  6535 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.246657  6350 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.486904  6285 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.551185  6535 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.551993  6489 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:51.552476  6489 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.552835  6489 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:51.362406  6621 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.389427  6525 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.493355  1706 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:51.215948  5912 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.493927  6533 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:02:51.565850  1633 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:02:51.259291  6519 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.508183  6744 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:51.509859  6693 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.511988  6571 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:51.512686  6513 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.513609  6766 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:51.492273  6616 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.337852  6725 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:51.461737  6652 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:51.386519  6070 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.458487  6689 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:02:51.392122  6720 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:51.580021  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93635.2us
I20260709 13:02:51.582195  6513 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:51.582341  5912 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.514345  6355 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:51.584220  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.501263  6559 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:02:51.567302  6495 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.582546  6519 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.612763  6571 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.613647  6489 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.599362  6652 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.619905  6725 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.620952  6652 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:51.622306  6725 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.622961  6692 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:51.614794  6544 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:51.616554  6746 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:02:51.623911  6692 consensus_queue.cc:1243] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:51.613255  6748 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.624450  6513 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:51.624976  1704 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.613297  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.626240  6070 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.626935  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:51.627568  6744 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.627722  6535 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.615415  6571 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:51.633971  1704 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:51.630995  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:51.640623  1708 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:02:51.566808  6525 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.659211  6285 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.668097  6759 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:51.698287  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:51.699126  2291 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.670706  6460 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.584997  6736 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:02:51.672120  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.673640  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.679507  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.706490  1551 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.681128  1557 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:51.681196  6489 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.681855  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.682935  6616 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.683333  6706 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:51.684329  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:51.685879  6734 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.687484  6747 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.672168  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.624843  6513 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:51.679056  6688 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.693295  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.700340  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.720325  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:51.696287  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.721505  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:51.680025  6571 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:51.660315  6744 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:51.640448  6692 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:02:51.641736  6535 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:02:51.700363  6070 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:51.594789  6766 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.741545  2291 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:02:51.725479  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 133521us
I20260709 13:02:51.721259  2288 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:51.720741  1558 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:51.711896  1559 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:51.747371  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:51.743623  2262 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.568s (SENT)
I20260709 13:02:51.748854  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.749329  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:51.750273  1703 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:51.714344  6736 consensus_queue.cc:1243] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:02:51.767431  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.771818  1709 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:51.700950  6285 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.774127  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:51.777647  1707 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:51.738418  6460 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:51.774695  1556 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:51.772935  6771 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:51.779448  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:51.781651  6755 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.599s	user 0.004s	sys 0.002s
I20260709 13:02:51.811827  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.785356  6734 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.737725  6690 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.790791  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:51.822930  1702 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:51.793841  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.793386  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:51.792945  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.797030  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:51.825392  2295 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:51.825774  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.826926  1634 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:51.776834  6544 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.802764  6616 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:51.806391  6285 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:51.806962  6571 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.795308  6460 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.809371  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.810235  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.843360  1703 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:51.822959  1551 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:51.828998  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.829300  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:51.830374  2296 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:51.828455  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.846004  1634 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:51.774173  6744 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:51.754699  6070 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.808317  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.796059  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:51.855017  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59072.7us
I20260709 13:02:51.857887  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.860337  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.864573  1634 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:51.865382  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.847110  6460 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:51.869355  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.881016  1635 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [17.9-18.10]
I20260709 13:02:51.860749  6544 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:51.867352  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:51.886296  1551 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.890857  2295 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:51.888682  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.892921  1631 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:51.849355  6571 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.899627  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.902141  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.860589  6652 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:02:51.826663  6513 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.906841  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.891992  6544 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.917897  1629 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:51.896384  6771 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:51.905563  6776 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.929747  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:51.925141  1551 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:02:51.933166  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.934362  6553 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.934799  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:51.917894  6774 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:02:51.942406  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.914765  6544 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.946403  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:51.918638  6725 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:51.952628  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:51.945837  6778 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:51.924732  6525 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.957521  6070 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:51.935740  6553 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:51.936452  5912 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:51.967873  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:51.973723  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.970224  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:51.983065  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.974115  6513 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:51.971221  6688 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.971362  6553 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.988363  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:51.979212  6772 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:51.982707  6070 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.968650  6692 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:02:51.983474  6766 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:51.982589  6525 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:51.981482  6652 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:51.920902  6744 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:51.968490  6355 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:02:51.967844  6725 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:51.990198  6733 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.021368  6571 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.023149  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.015254  6652 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:52.024479  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.013208  6070 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.025144  1560 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:52.023864  1705 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:52.026728  6070 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:51.994166  6553 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:51.986728  6737 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:02:52.031279  6355 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:52.013051  6763 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:52.015110  6525 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:52.024482  6652 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.051595  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.054080  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.055228  6776 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.056109  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.056828  1630 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:02:52.058120  1630 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:52.058521  1630 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.058310  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.063005  2289 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:52.063825  6763 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:52.064354  6763 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.064828  6763 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.050258  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:52.035710  6355 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:52.069644  1635 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:52.072469  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.074900  2288 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.059592  1630 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.080766  1630 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.047508  6525 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:52.080415  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:52.085783  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.082041  6621 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.087996  6782 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:02:52.095463  1635 pending_rounds.cc:85] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 12
I20260709 13:02:52.096138  1635 pending_rounds.cc:107] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:02:52.096560  1635 raft_consensus.cc:2889] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.043633  6598 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.120402  6355 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.122196  6692 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.122439  6355 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:52.123008  6692 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.111296  6495 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:52.126135  6692 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:52.133270  6787 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.080s	user 0.004s	sys 0.000s
I20260709 13:02:52.134326  6598 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:52.105847  6766 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:02:52.143635  6725 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.146183  1628 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:52.156253  1551 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:52.160240  1630 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:52.167105  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.168044  1626 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:52.187515  2297 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:52.149673  6495 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:52.207070  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:52.227100  1560 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:52.248375  6460 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.252568  1631 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.219244  6748 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:02:52.261475  2262 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.142s (SENT)
I20260709 13:02:52.262144  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:52.263185  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.264298  1559 raft_consensus.cc:2376] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14. Re-sending same reply.
I20260709 13:02:52.267007  6550 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:52.267967  6460 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.278090  1628 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:52.281026  6553 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:52.271378  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 637180us
I20260709 13:02:52.283993  6495 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.291131  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:52.291899  1632 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.247854  6791 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:52.292526  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.296017  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.296764  1628 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:52.260618  6706 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:52.300230  6706 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.302582  6706 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.303943  6720 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:52.304387  6720 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.305395  6720 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.306423  6720 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.308944  6652 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.309728  6322 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:52.310133  6322 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.253510  6525 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.287418  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:52.285346  6763 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.314270  6763 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.311456  6652 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.308795  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.304286  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.315138  6525 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.321193  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.283504  6553 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.319481  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.322777  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 808) took 1173 ms. Trace:
I20260709 13:02:52.308831  6544 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.326413  6725 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:52.326992  1633 rpcz_store.cc:276] 0709 13:02:51.145481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.145970 (+   489us) service_pool.cc:224] Handling call
0709 13:02:52.318852 (+1172882us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54002760d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:52.319024 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818064128}
I20260709 13:02:52.327526  6693 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.328732  6544 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.325265  6706 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:52.311025  6322 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.328610  1559 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.341827  1708 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:52.349393  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:52.346954  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.331698  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:52.329879  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.330272  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.332165  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.346418  2288 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:52.365583  2297 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.340639  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.365664  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.368283  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:52.379196  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:52.328735  6772 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:52.333207  6736 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:52.372534  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.392900  1707 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.373953  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.394263  1557 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.380260  1626 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:02:52.383634  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.384390  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:52.389792  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 218747us
I20260709 13:02:52.396497  1632 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:52.397641  1633 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:52.339143  6737 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:52.404253  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.417276  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.426349  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.426694  1708 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:02:52.401492  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.401762  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.399595  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.428659  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:52.431908  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 227166us
I20260709 13:02:52.366492  6706 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.374013  6571 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.378947  6322 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.381441  6535 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:52.436640  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.378703  6734 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:52.404204  6705 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:02:52.439836  2295 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:52.441684  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:52.442181  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 235190us
I20260709 13:02:52.434298  6525 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:52.444644  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:52.434648  6736 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:52.446826  1708 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.447160  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:52.386139  6553 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:02:52.434293  6106 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:52.395999  6772 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.452147  1705 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.431418  6744 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:52.453756  1626 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.444754  6737 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.399542  6652 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:52.434130  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:52.454264  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:52.439807  6706 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.385182  6725 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:52.470533  6106 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.474193  6797 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:52.456017  6535 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.464953  6772 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.482937  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.485327  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.502205  1706 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:52.502528  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.505290  1552 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
I20260709 13:02:52.505604  1552 pending_rounds.cc:85] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 13
I20260709 13:02:52.505762  1552 pending_rounds.cc:107] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 13:02:52.505944  1552 raft_consensus.cc:2889] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.506515  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.457839  6746 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:52.486444  1705 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:02:52.486655  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:52.489286  1559 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:52.491053  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.520272  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.521170  1705 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.521960  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.522784  1705 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:52.523339  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.490217  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:52.461771  6525 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:52.497860  1554 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:52.498325  1626 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:02:52.509735  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.526230  6553 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:52.526450  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.527130  6571 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:52.527472  1558 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:52.528311  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.528514  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.528916  1702 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:52.529732  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.530519  1558 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:52.531194  6106 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.532972  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:52.490769  6725 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:52.540073  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.542570  1626 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:02:52.543419  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.544216  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.544667  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:52.536453  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81153us
I20260709 13:02:52.537119  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.536562  1632 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.691886  1706 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:52.531302  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.539345  1707 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.552994  6725 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.463127  6734 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:52.611743  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.491420  6744 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:52.493353  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61661.7us
I20260709 13:02:52.715899  1635 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:52.694996  1558 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:52.717154  1556 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:52.686944  1626 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.494529  6791 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.511687  6070 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:52.540551  6533 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:52.690850  1709 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:52.537654  1701 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.692741  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:02:52.695065  1706 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:52.789036  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:52.789508  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97002.2us
I20260709 13:02:52.789996  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.790241  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.790638  1557 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:02:52.707723  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.790262  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.791606  1707 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:52.720387  1558 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:52.792560  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.700181  1633 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:52.725114  6070 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.728682  6525 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.704545  6737 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.733517  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.800789  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.741736  6489 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.750918  6513 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:52.699142  6705 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:52.714069  6553 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.765748  6725 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.694245  6763 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:52.694550  6692 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.768337  6770 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:52.769403  6617 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:02:52.829667  6801 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:02:52.800756  6711 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:52.830376  6617 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:52.830734  6711 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.831205  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.831660  6711 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.783206  1702 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:52.837888  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.838861  1702 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.795365  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.842119  1707 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.797127  1708 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:52.799444  6070 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.803012  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:52.804381  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.854158  1560 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:52.713227  6535 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:52.817353  6746 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:52.856171  6746 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.821233  2289 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:02:52.821805  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.813344  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.858247  6746 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:52.688676  6754 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.859431  6754 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.862114  6754 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.824745  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:52.829692  6797 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.865180  6797 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.868213  1706 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:52.611263  6736 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.806936  6533 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.871482  6533 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:52.828783  6800 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:52.828680  6621 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.880052  6621 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.882436  6621 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.883000  6621 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:52.889528  1703 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:02:52.892866  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:52.897799  6744 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:52.908593  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:52.800582  6720 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:52.806138  6489 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:52.833648  6725 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.830742  6617 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:52.876405  6571 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.921607  6652 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:52.912494  6720 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:52.834721  6553 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:52.937140  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.920084  6705 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:52.833787  6692 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:52.949215  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.951256  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:52.952735  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44272.8us
I20260709 13:02:52.953070  2296 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:52.938898  6720 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.954556  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:52.959283  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.958792  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.963948  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.967494  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:52.970736  1633 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:52.927640  6571 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:52.973819  1551 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:52.974303  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:53.017655  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 247425us
I20260709 13:02:52.952800  6725 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:53.023177  1709 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.981251  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.972632  6763 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:52.971068  6797 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:53.045153  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:52.970506  6519 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.047034  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:53.049918  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:53.022650  6793 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:53.053064  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:53.058276  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:52.972723  6513 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:02:52.981652  6770 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:02:53.056377  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:53.059002  1629 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:52.973058  6652 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:02:53.062695  1634 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:52.955276  6489 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.072232  6489 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.019320  6772 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:53.118244  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.033926  6725 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:53.147950  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.116922  6535 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:53.173218  6744 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:53.054037  6519 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.068336  6793 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:02:53.354540  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 316157us
I20260709 13:02:53.176093  6770 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:53.178064  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59156.7us
I20260709 13:02:53.384373  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:53.385778  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:53.183702  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 779459us
I20260709 13:02:53.246565  5912 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:53.315156  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.056855  6748 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:53.323004  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 242814us
W20260709 13:02:53.324816  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 306161us
W20260709 13:02:53.325574  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.391009  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.321910  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 304122us
I20260709 13:02:53.388103  5912 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.174398  6553 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:52.972242  6617 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.174889  6772 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.057644  6705 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:53.382201  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:53.387428  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:53.512236  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.060s	sys 0.126s
W20260709 13:02:53.633167  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.063s	sys 0.150s
W20260709 13:02:53.556464  1560 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
W20260709 13:02:53.633725  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 246361us
W20260709 13:02:53.635463  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 253336us
W20260709 13:02:53.461931  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49687.9us
I20260709 13:02:53.460016  1626 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:53.414160  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:53.327000  5983 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.325528  6744 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:53.651113  1560 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:53.652123  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2604) took 1264 ms. Trace:
I20260709 13:02:53.652647  1560 rpcz_store.cc:276] 0709 13:02:52.387604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.853579 (+465975us) service_pool.cc:224] Handling call
0709 13:02:52.854005 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.651781 (+797776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.652050 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:53.654701  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 240585us
I20260709 13:02:53.340777  6746 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.337s
I20260709 13:02:53.649816  6513 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:02:53.678056  5983 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:53.680760  5983 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.681353  5983 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.681818  5983 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:53.597445  6811 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 13:02:53.600617  6808 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:02:53.596410  6816 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:02:53.694365  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2596) took 1318 ms (client timeout 1644 ms). Trace:
I20260709 13:02:53.326511  6798 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.596938  6812 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:02:53.340834  6737 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:53.616163  6706 consensus_meta.cc:287] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.111s	user 0.011s	sys 0.000s
I20260709 13:02:53.689872  1556 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:02:53.757792  1556 pending_rounds.cc:85] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 9
I20260709 13:02:53.758172  1556 pending_rounds.cc:107] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:02:53.758459  1556 raft_consensus.cc:2889] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:53.691126  1560 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:53.691927  5912 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.493306  6754 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:53.393684  6621 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:53.535208  6519 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:53.517613  6692 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:53.694818  1554 rpcz_store.cc:269] 0709 13:02:52.375610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.386471 (+ 10861us) service_pool.cc:224] Handling call
0709 13:02:53.689253 (+1302782us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:53.694297 (+  5044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161595136}
I20260709 13:02:53.596522  6571 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.376272  6763 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:53.383594  6770 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:52.440933   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:53.698222  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 312483us
I20260709 13:02:53.511667  6725 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.410413  6807 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:53.643999  6819 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:53.649525  6106 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:53.692987  1444 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8b2c3934407d422495728f84374c6b53 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:53.762470  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 800) took 1252 ms. Trace:
I20260709 13:02:53.807721  1560 rpcz_store.cc:276] 0709 13:02:52.510690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.690530 (+1179840us) service_pool.cc:224] Handling call
0709 13:02:53.690902 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.762085 (+ 71183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.762393 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:53.692816  5983 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.811230  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 804) took 1061 ms. Trace:
I20260709 13:02:53.811652  1554 rpcz_store.cc:276] 0709 13:02:52.749316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.800110 (+1050794us) service_pool.cc:224] Handling call
0709 13:02:53.810738 (+ 10628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.810915 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.810941 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.810984 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:53.811021 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.811075 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.811176 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:53.491568  6489 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.210s
I20260709 13:02:53.410203  6797 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.302s
I20260709 13:02:53.470770  6748 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.796082  6533 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.693590  6734 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:53.765187  6754 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.807277  1444 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 114565us
I20260709 13:02:53.836202  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 982) took 1091 ms. Trace:
I20260709 13:02:53.837703  6736 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:02:53.802023  6692 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:53.772337  6706 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:53.567929  6793 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:53.773141  6737 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:53.846278  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.000s	sys 0.022s
W20260709 13:02:53.943178  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.000s	sys 0.022s
I20260709 13:02:53.792723  6798 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:53.951560  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2589) took 2021 ms (client timeout 1914 ms). Trace:
W20260709 13:02:53.952075  1631 rpcz_store.cc:269] 0709 13:02:51.929860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.931897 (+  2037us) service_pool.cc:224] Handling call
0709 13:02:53.951256 (+2019359us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001c34d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:53.951489 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683942656}
I20260709 13:02:53.806977  6106 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:53.843014  2297 rpcz_store.cc:276] 0709 13:02:52.715820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.716050 (+   230us) service_pool.cc:224] Handling call
0709 13:02:52.724315 (+  8265us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.957197 (+232882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.957231 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.979221 (+ 21990us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.382596 (+403375us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:53.382668 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.382789 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.807333 (+424544us) raft_consensus.cc:1685] finished
0709 13:02:53.807360 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.807544 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20556,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":562728,"threads_started":2,"wal-append.queue_time_us":502560}
I20260709 13:02:53.851330  6533 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:53.914572  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.852100  6621 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:53.967234  6621 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.845726  6489 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:02:53.802057  6744 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.970870  6744 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.806721  6807 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.973469  6807 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:53.874411  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 224330us
W20260709 13:02:53.928241  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.001s	sys 0.013s
W20260709 13:02:53.974228  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.001s	sys 0.013s
I20260709 13:02:53.844756  6754 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:53.873632  6550 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.975092  6550 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.978108  6550 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:53.931870  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:53.932077  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.026s
W20260709 13:02:53.979954  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.012s	sys 0.026s
W20260709 13:02:53.933317  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 277790us
I20260709 13:02:53.980286  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:53.980801  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:53.934876  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 798) took 1471 ms. Trace:
I20260709 13:02:53.981230  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:53.981630  1551 rpcz_store.cc:276] 0709 13:02:52.463867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.974788 (+510921us) service_pool.cc:224] Handling call
0709 13:02:52.975115 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.975338 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:53.459327 (+483989us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.934131 (+474804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.934207 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:53.934288 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.934447 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.934748 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9001,"prepare.queue_time_us":574715,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":68,"raft.queue_time_us":86620,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4359,"spinlock_wait_cycles":36679680,"thread_start_us":911870,"threads_started":3,"wal-append.queue_time_us":390594}
I20260709 13:02:53.937309  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:53.924582  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.730s (SENT) [suppressed 27 similar messages]
I20260709 13:02:53.925064  6525 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:53.983542  1701 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:53.984023  6525 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:53.987119  6525 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:53.925612  6734 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:53.987967  6734 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:53.989305  6734 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:53.991108  6734 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:53.960335  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2595) took 1570 ms. Trace:
I20260709 13:02:53.991950  1559 rpcz_store.cc:276] 0709 13:02:52.375322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.547857 (+172535us) service_pool.cc:224] Handling call
0709 13:02:52.687963 (+140106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.708543 (+ 20580us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:53.022681 (+314138us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.943696 (+921015us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.945479 (+  1783us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:53.945554 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.945657 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.945826 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":616622,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"thread_start_us":1082920,"threads_started":2}
W20260709 13:02:53.980295  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 587118us
I20260709 13:02:53.931102  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:53.996081  1703 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:02:53.997478  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72982us
W20260709 13:02:53.938685  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 137761us
I20260709 13:02:54.001070  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.001895  1559 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 20 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:54.001155  1551 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:54.002995  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:54.009186  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:54.015836  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.017776  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.018462  1559 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:54.069247  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.069868  1632 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:54.073088  6616 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:54.073629  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:54.073769  6616 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.075127  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:54.088610  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72906.4us
I20260709 13:02:54.089677  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.091261  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.093065  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.096436  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:53.966403  6106 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.107967  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:54.084847  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 110524us
I20260709 13:02:54.107443  1559 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:53.892010  6770 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.088735  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 799) took 1603 ms. Trace:
W20260709 13:02:54.077960  6810 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:02:54.014612  6489 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.122227  1632 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:02:54.075578  6533 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:54.127266  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:54.137858  2295 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:54.077318  6835 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:53.935379  6763 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.077611  6819 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:54.135990  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:54.137115  1632 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:54.076422  6616 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:54.132769  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 200943us
I20260709 13:02:54.145017  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:54.114212  6692 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:54.120723  6830 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.206580  2289 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:54.120846  6797 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:54.136173  6070 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.289s	user 0.007s	sys 0.014s
I20260709 13:02:54.120834  6693 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:54.219810  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.225466  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:54.229228  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 136222us
I20260709 13:02:54.271317  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.210345  6692 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:54.272820  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45056.1us
I20260709 13:02:54.214542  6770 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:54.139032  6489 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:54.295540  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 278680us
I20260709 13:02:54.357800  6489 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:54.422765  1554 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
W20260709 13:02:54.347940  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76730.2us
I20260709 13:02:54.572449  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:54.216169  6855 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:54.118243  1555 rpcz_store.cc:276] 0709 13:02:52.485292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.406166 (+920874us) service_pool.cc:224] Handling call
0709 13:02:53.406491 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.406703 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:53.910715 (+504012us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.088234 (+177519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.088316 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:54.088390 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.088490 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.088655 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81923,"prepare.queue_time_us":165539,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":80105,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":19502,"thread_start_us":535951,"threads_started":2}
I20260709 13:02:54.575888  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:54.459955  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108717us
I20260709 13:02:54.200584  6848 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 13:02:54.201654  6835 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.511329  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2601) took 2134 ms. Trace:
I20260709 13:02:54.230332  6852 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:54.589258  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.604991  1555 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:54.605962  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.607446  1555 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:02:54.590080  1553 rpcz_store.cc:276] 0709 13:02:52.377112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.740998 (+363886us) service_pool.cc:224] Handling call
0709 13:02:52.741365 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.805391 (+ 64026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.805425 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:52.805468 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:52.805501 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.510988 (+1705487us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b54003faf54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:54.511057 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.511249 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146777984}
I20260709 13:02:54.609077  1555 pending_rounds.cc:85] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 10
I20260709 13:02:54.609943  1555 pending_rounds.cc:107] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:54.610620  1555 raft_consensus.cc:2889] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.510391  6833 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:54.616531  6798 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:54.617386  6798 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:54.617933  6798 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.622558  6835 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:54.623164  6835 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:54.624989  6831 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.518s	user 0.001s	sys 0.000s
I20260709 13:02:54.626727  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2745) took 1587 ms. Trace:
W20260709 13:02:54.627280  6844 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.496s	user 0.000s	sys 0.000s
I20260709 13:02:54.627111  1635 rpcz_store.cc:276] 0709 13:02:53.039216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.039377 (+   161us) service_pool.cc:224] Handling call
0709 13:02:53.054035 (+ 14658us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.055512 (+  1477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.055542 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.067405 (+ 11863us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.774170 (+706765us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:53.805880 (+ 31710us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.806113 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.806208 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.626490 (+820282us) raft_consensus.cc:1685] finished
0709 13:02:54.626512 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.626669 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42284,"prepare.queue_time_us":31485,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":207225,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":14194,"thread_start_us":448133,"threads_started":1,"wal-append.queue_time_us":838664}
I20260709 13:02:54.629489  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:54.295665  6692 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:54.637261  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2653) took 1813 ms. Trace:
W20260709 13:02:54.635140  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60208us
I20260709 13:02:54.641353  6835 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:54.646322  6720 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.667s	user 0.010s	sys 0.001s
I20260709 13:02:54.650195  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:54.648913  1706 rpcz_store.cc:276] 0709 13:02:52.823902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.825712 (+  1810us) service_pool.cc:224] Handling call
0709 13:02:52.867886 (+ 42174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.868785 (+   899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.636989 (+1768204us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b5400134cd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:54.637187 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95965440}
I20260709 13:02:54.225063  6070 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.222546  6797 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.663872  6863 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:54.664518  1709 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:54.665345  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.030s	sys 0.060s
W20260709 13:02:54.666368  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.030s	sys 0.061s
I20260709 13:02:54.221355  6830 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.631070  6846 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:54.647928  6736 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:54.687225  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.651382  6848 consensus_queue.cc:1243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:02:54.669009  6804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.492s	user 0.019s	sys 0.079s
W20260709 13:02:54.709412  6804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.528s	user 0.019s	sys 0.080s
I20260709 13:02:54.733916  6813 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:54.734493  6813 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:54.704643  6848 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.737619  6070 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:54.738237  6070 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:54.738716  6070 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:54.699450  6837 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.686s	user 0.000s	sys 0.001s
W20260709 13:02:54.743249  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2747) took 1693 ms (client timeout 1867 ms). Trace:
W20260709 13:02:54.743685  1626 rpcz_store.cc:269] 0709 13:02:53.049499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.049655 (+   156us) service_pool.cc:224] Handling call
0709 13:02:54.743066 (+1693411us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:54.743188 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941252352}
I20260709 13:02:54.745548  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:54.749809  6535 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.750433  6535 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:54.752671  6744 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.000s
I20260709 13:02:54.753233  6744 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:54.753633  6848 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:54.753727  6744 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.754277  6848 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.710152  6851 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:54.756717  6535 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:54.685446  6706 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:54.664530  6720 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.761284  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2655) took 1912 ms. Trace:
W20260709 13:02:54.757454  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.019s	sys 0.040s
I20260709 13:02:54.634658  6855 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:54.781556  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.019s	sys 0.040s
W20260709 13:02:54.794034  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.014s
W20260709 13:02:54.794385  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.010s	sys 0.014s
W20260709 13:02:54.794718  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 503116us
I20260709 13:02:54.780498  6798 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:54.802279  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.804454  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.805583  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.805502  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:54.684023  6846 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:54.763088  6754 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.788s	user 0.006s	sys 0.005s
I20260709 13:02:54.801659  6535 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:54.823187  1627 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:54.832122  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:54.803328  6744 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:54.781023  1702 rpcz_store.cc:276] 0709 13:02:52.834178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.839513 (+  5335us) service_pool.cc:224] Handling call
0709 13:02:54.745433 (+1905920us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:54.746408 (+   975us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20366592}
I20260709 13:02:54.803913  6856 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:54.735379  6813 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:54.849722  2293 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.001s	sys 0.013s
W20260709 13:02:54.857941  2293 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.001s	sys 0.013s
W20260709 13:02:54.782446  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.018s	sys 0.033s
W20260709 13:02:54.858829  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.018s	sys 0.034s
W20260709 13:02:54.793861  6793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.008s	sys 0.014s
I20260709 13:02:54.831483  6519 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.803236  6616 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:02:54.852841  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47417.8us
I20260709 13:02:54.830650  6652 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:54.874958  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:54.881320  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:54.826532  6754 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:54.865283  6793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.008s	sys 0.015s
I20260709 13:02:54.843674  6853 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:54.878826  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:54.931118  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52378us
I20260709 13:02:54.877642  1632 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:54.833268  6535 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:54.896507  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.000s	sys 0.012s
I20260709 13:02:54.896517  1635 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:02:54.859820  6835 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.892261  6851 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
W20260709 13:02:54.907289  6843 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:54.910166  6830 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:54.938189  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.002s	sys 0.014s
W20260709 13:02:54.938709  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.003s	sys 0.014s
I20260709 13:02:54.895712  6819 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:02:54.858894  6856 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.917217  6797 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:54.917677  6350 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:54.920624  6692 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:54.922619  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2608) took 2534 ms. Trace:
I20260709 13:02:54.925123  6621 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:54.922636  6846 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:54.932638  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.000s	sys 0.012s
I20260709 13:02:54.910605  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:54.940613  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:54.970149  1632 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:54.943475  1631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.000s	sys 0.011s
W20260709 13:02:54.971920  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.000s	sys 0.011s
W20260709 13:02:54.944159  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.006s	sys 0.005s
W20260709 13:02:54.945925  1705 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.147s	user 0.004s	sys 0.019s
W20260709 13:02:54.972540  1705 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.005s	sys 0.019s
W20260709 13:02:54.972617  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62117.5us
I20260709 13:02:54.973206  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2043) took 1988 ms. Trace:
I20260709 13:02:54.948837  1704 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:54.872364  6519 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:54.892861  6652 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:54.953042  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.025s
W20260709 13:02:54.987190  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.025s
W20260709 13:02:54.987607  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 631099us
I20260709 13:02:54.946010  6070 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:54.943827  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.011s	sys 0.000s
W20260709 13:02:55.000633  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.011s	sys 0.000s
W20260709 13:02:55.001561  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.002643  6535 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:55.003788  6870 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:55.004288  6848 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:55.005298  6867 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:55.006338  6737 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:55.007825  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.008486  6754 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:55.014540  6734 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.023s	user 0.011s	sys 0.000s
I20260709 13:02:55.015271  1551 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:55.017751  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2605) took 1002 ms. Trace:
I20260709 13:02:55.018136  6652 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:55.020159  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.014s	sys 0.014s
W20260709 13:02:55.020485  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.014s	sys 0.014s
I20260709 13:02:55.020983  6519 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:55.021761  6817 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.337s	user 0.000s	sys 0.017s
I20260709 13:02:54.935868  6845 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:54.973524  1705 rpcz_store.cc:276] 0709 13:02:52.985645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.996776 (+ 11131us) service_pool.cc:224] Handling call
0709 13:02:52.997206 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.997519 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:53.323154 (+325635us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.774830 (+451676us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.972871 (+1198041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.972928 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:54.972977 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.973041 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.973154 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":320222,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4188,"raft.queue_time_us":237855,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":41224,"thread_start_us":1911390,"threads_started":3}
I20260709 13:02:54.973855  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:54.986384  1559 raft_consensus.cc:2368] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13.
I20260709 13:02:54.953819  1556 rpcz_store.cc:276] 0709 13:02:52.388647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.863427 (+474780us) service_pool.cc:224] Handling call
0709 13:02:53.689418 (+825991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.772583 (+ 83165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.922348 (+1149765us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b540036ced0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:54.922536 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898430208}
I20260709 13:02:54.977589  5983 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:55.022497  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:54.952772  6866 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.024281  6875 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.024575  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2661) took 2002 ms. Trace:
W20260709 13:02:55.029277  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55482.3us
I20260709 13:02:55.023353  6070 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.986593  6835 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:54.975270  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.006s	sys 0.005s
W20260709 13:02:55.052386  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 840191us
I20260709 13:02:54.941205  6853 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:55.049690  6804 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:55.034250  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 1993) took 1038 ms. Trace:
I20260709 13:02:55.062901  1560 rpcz_store.cc:276] 0709 13:02:53.995633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.995742 (+   109us) service_pool.cc:224] Handling call
0709 13:02:53.996056 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.092145 (+ 96089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.092176 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.092646 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.859363 (+766717us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:54.859407 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.859477 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.033969 (+174492us) raft_consensus.cc:1685] finished
0709 13:02:55.034008 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.034193 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95689,"raft.queue_time_us":206780,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14468,"thread_start_us":657331,"threads_started":1,"wal-append.queue_time_us":745921}
I20260709 13:02:55.066558  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.050580  1709 rpcz_store.cc:276] 0709 13:02:53.022281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.025141 (+  2860us) service_pool.cc:224] Handling call
0709 13:02:53.045876 (+ 20735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.665023 (+1619147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.024270 (+359247us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003afcd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:55.024503 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3722086144}
I20260709 13:02:55.053251  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.008534  1708 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.032544  6706 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.055405  1632 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:55.028494  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.058666  1705 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:55.058815  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.059026  1635 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:55.018136  1627 rpcz_store.cc:276] 0709 13:02:54.014843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.016338 (+  1495us) service_pool.cc:224] Handling call
0709 13:02:54.016608 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.823748 (+807140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.017506 (+193758us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54001e8a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:55.017692 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854481792}
I20260709 13:02:55.028494  6793 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:55.093755  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.095368  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 983) took 2341 ms. Trace:
I20260709 13:02:55.067389  6875 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.068246  6776 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.102653  6776 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.104785  6875 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.069294  6856 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.047398  6525 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:55.071306  6853 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:55.058871  6835 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:55.072824  6106 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.941s	user 0.010s	sys 0.000s
I20260709 13:02:55.052834  6866 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.050912  5983 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.077824  6692 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:55.028781  6798 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.022403  6621 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.053664  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.090554  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.094252  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.067543  6836 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:02:55.105872  6489 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.126473  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:55.127494  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 234103us
I20260709 13:02:55.129160  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.129736  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.130300  1701 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:55.098573  1708 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:55.096505  2288 rpcz_store.cc:276] 0709 13:02:52.753861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.754123 (+   262us) service_pool.cc:224] Handling call
0709 13:02:52.754642 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.755141 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.755185 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.755872 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.981683 (+225811us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:52.981722 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.981789 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.078184 (+2096395us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b24003131d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:02:55.090079 (+ 11895us) raft_consensus.cc:1685] finished
0709 13:02:55.090104 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.095306 (+  5202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1757,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":15901056,"thread_start_us":225284,"threads_started":1,"wal-append.queue_time_us":629116}
I20260709 13:02:55.131659  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.132350  1701 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:55.062525  6720 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:55.106158  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:55.133289  1633 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:55.106958  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:55.107545  1555 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:55.110173  6835 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.108199  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.062212  6804 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.079929  6519 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.109741  1552 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:55.062345  6070 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.085685  6855 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:55.089685  6652 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.091197  6861 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:55.120126  2289 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:55.120674  1632 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:02:55.174264  1632 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:55.105494  6776 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:55.135857  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.175613  1701 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:55.132345  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2657) took 2176 ms (client timeout 1994 ms). Trace:
I20260709 13:02:55.161157  1627 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:55.177212  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.177995  1627 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 9.
W20260709 13:02:55.161119  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 107922us
W20260709 13:02:55.178475  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 124139us
I20260709 13:02:55.143110  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:55.094964  6734 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:02:55.178915  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.178858  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.111879  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.167547  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.181244  1556 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:55.181741  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.182006  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.182374  1560 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:55.182550  1556 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:02:55.184070  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:02:55.185603  1707 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:55.091019  6706 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.185871  6734 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:55.171010  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:55.135497  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.187081  1555 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:02:55.187613  6706 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.180600  1627 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:55.179075  1708 rpcz_store.cc:269] 0709 13:02:52.956116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.956282 (+   166us) service_pool.cc:224] Handling call
0709 13:02:55.007618 (+2051336us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:02:55.132274 (+124656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8217344}
I20260709 13:02:55.188818  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.188556  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:55.188357  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45357.9us
I20260709 13:02:55.186647  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.106310  6793 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.191416  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.206887  1708 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:55.190161  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:02:55.205662  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:55.207912  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.208526  1706 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:55.209266  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.126055  6621 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:55.213218  6873 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:55.163887  6804 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.113426  6525 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.207559  2288 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:55.215811  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:55.163926  6856 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.173357  6350 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.111951  6106 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:55.225934  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98138us
I20260709 13:02:55.236548  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.134654  6853 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:02:55.175839  6652 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:55.124606  6798 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.217365  6803 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:55.254992  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:55.165042  6489 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.213804  6873 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:55.218684  6737 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:55.257575  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2753) took 1251 ms. Trace:
I20260709 13:02:55.258615  6706 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:55.234992  6621 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:55.173553  6519 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:55.275172  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.264169  1629 rpcz_store.cc:276] 0709 13:02:54.006117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.006242 (+   125us) service_pool.cc:224] Handling call
0709 13:02:54.006553 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.123285 (+116732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.123317 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.138352 (+ 15035us) log.cc:844] Serialized 52 byte log entry
0709 13:02:54.939202 (+800850us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:55.141919 (+202717us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.235343 (+ 93424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.235429 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.257227 (+ 21798us) raft_consensus.cc:1685] finished
0709 13:02:55.257257 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.257496 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98244,"prepare.queue_time_us":320827,"raft.queue_time_us":847332,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":183,"thread_start_us":1001928,"threads_started":3,"wal-append.queue_time_us":873831}
I20260709 13:02:55.251524  6880 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.287855  1557 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:02:55.288570  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.256404  6803 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:55.290277  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53810.3us
I20260709 13:02:55.292907  6803 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:55.237778  6856 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:55.293218  6887 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:55.293685  6849 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:55.277560  6705 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.294099  6849 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.294504  6849 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:55.291574  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.297420  6848 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:55.297605  6862 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.290179  6877 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 13:02:55.298085  6848 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:55.298513  6848 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:55.298419  6862 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.292522  6819 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:55.299460  6819 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.299881  6819 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.300114  6880 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.291883  1557 pending_rounds.cc:85] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 10
I20260709 13:02:55.301200  1557 pending_rounds.cc:107] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:02:55.301431  1557 raft_consensus.cc:2889] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.302176  6106 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.273154  6706 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.196306  6861 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:55.329483  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74627.6us
I20260709 13:02:55.329854  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:02:55.251194  6350 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.332790  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.338521  1701 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:55.336532  1707 raft_consensus.cc:2376] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 15. Re-sending same reply.
I20260709 13:02:55.342842  5983 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.344003  6873 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:55.345278  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2621) took 1342 ms (client timeout 1644 ms). Trace:
I20260709 13:02:55.265494  1627 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:55.345708  1554 rpcz_store.cc:269] 0709 13:02:54.002885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.074850 (+ 71965us) service_pool.cc:224] Handling call
0709 13:02:55.345108 (+1270258us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b540036ced4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:55.345215 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650318208}
I20260709 13:02:55.348770  1635 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:55.331118  6706 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:55.322256  6877 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.327983  6763 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:55.352037  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.277529  6737 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.360374  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.362882  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:55.381433  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2592) took 3019 ms (client timeout 2814 ms). Trace:
I20260709 13:02:55.382982  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:55.384153  1628 rpcz_store.cc:269] 0709 13:02:52.361743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.361992 (+   249us) service_pool.cc:224] Handling call
0709 13:02:55.381213 (+3019221us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54002747d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:55.381343 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453660928}
I20260709 13:02:55.392207  1523 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:55.431025  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:55.432497  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 253641us
W20260709 13:02:55.434068  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71268.9us
W20260709 13:02:55.421295  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.435009  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.363745  6763 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:55.380967  6814 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.896s	user 0.015s	sys 0.010s
W20260709 13:02:55.388999  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 137674us
W20260709 13:02:55.375334  6840 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.745s	user 0.002s	sys 0.000s
I20260709 13:02:55.437719  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:55.365688  6886 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:55.363634  6830 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:55.365408  6854 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:55.444509  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:55.453071  2295 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:55.453125  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.366182  6070 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.451148  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:55.373308  6861 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:55.372313  6866 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.451426  1556 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:55.452157  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.373241  6519 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.460335  1552 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:55.359441  6106 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:02:55.385934  6692 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:02:55.461093  2297 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:02:55.434753  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:55.447438  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:55.457408  2288 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:55.430447  6535 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.365059  6883 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 13:02:55.430158  6513 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:55.428458  6693 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:55.450628  6720 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:55.450307  6798 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.470862  2296 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:55.488418  2297 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:55.446450  6830 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:55.457893  6350 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.376612  6793 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.452847  6854 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:55.504637  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:55.482156  1633 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:02:55.364022  6877 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:55.500294  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65613.4us
I20260709 13:02:55.507555  1633 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:55.513185  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.514262  1708 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:55.515038  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.513235  1633 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:02:55.482772  6693 consensus_queue.cc:1243] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:55.515888  1707 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:02:55.487953  6720 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.528682  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.528878  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:55.530675  2297 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:02:55.496871  6525 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:55.501536  6830 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.505520  6853 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:55.510167  6854 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:55.605898  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 158541us
I20260709 13:02:55.607882  1554 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:02:55.609331  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:55.510872  6705 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:55.500705  6869 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.476207  6535 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.531664  6720 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.673671  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:55.674453  1709 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:55.663319  6814 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:02:55.615638  6106 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.660647  6853 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:55.660410  6525 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:55.757346  1707 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:55.767746  6904 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.298s
I20260709 13:02:55.769204  6868 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:55.769729  6868 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:55.770117  6868 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:55.771936  6902 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:55.772405  6902 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.772714  6902 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.773329  6838 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.773407  1709 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:55.743716  6820 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.744292  6748 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:55.833534  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:55.834395  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.840605  1628 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:55.879673  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46231.1us
I20260709 13:02:55.879998  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.885739  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.774065  6838 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:55.924367  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 854) took 3325 ms. Trace:
W20260709 13:02:55.924093  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.753326  6869 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:55.929577  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48909.2us
I20260709 13:02:55.949307  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:55.950330  2288 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:02:55.950941  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.945173  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:55.950917  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:55.953399  1555 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:55.758512  6525 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:55.932245  1700 rpcz_store.cc:276] 0709 13:02:52.508803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.509004 (+   201us) service_pool.cc:224] Handling call
0709 13:02:52.509366 (+   362us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:52.704829 (+195463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.704862 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:52.776553 (+ 71691us) log.cc:844] Serialized 78 byte log entry
0709 13:02:52.991517 (+214964us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:53.056250 (+ 64733us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.056452 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.056525 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.897228 (+840703us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b18004a5360. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:55.380225 (+1482997us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:02:55.833525 (+453300us) raft_consensus.cc:1685] finished
0709 13:02:55.833553 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.833764 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19173,"prepare.queue_time_us":88094,"prepare.run_cpu_time_us":3763,"prepare.run_wall_time_us":33114,"raft.queue_time_us":82331,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":32030,"spinlock_wait_cycles":168945664,"thread_start_us":87364,"threads_started":1,"wal-append.queue_time_us":293100}
I20260709 13:02:55.969846  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:55.995923  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.926828  6803 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.007s
I20260709 13:02:56.010315  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.010749  1709 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:56.021962  1555 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:56.024358  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.033200  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:56.024905  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61797.5us
I20260709 13:02:55.928694  6906 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:55.942039  6621 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:55.758622  6106 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:55.964251  6525 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.961630  6693 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.057544  1700 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:55.963477  6880 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.011782  6355 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.076107  1700 raft_consensus.cc:1217] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[11.7-14.9]   Dedup: 11.7->[13.8-14.9]
I20260709 13:02:56.025739  6763 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.078707  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:56.076822  1700 pending_rounds.cc:85] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 7
I20260709 13:02:56.084553  1700 pending_rounds.cc:107] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:02:56.084926  1700 raft_consensus.cc:2889] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.085494  1628 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:56.045953  6912 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:02:56.085575  2293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.000s	sys 0.013s
W20260709 13:02:56.094239  2293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.000s	sys 0.014s
I20260709 13:02:56.094643  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:56.023453  6803 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:56.099103  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.004s	sys 0.010s
I20260709 13:02:55.962875  6869 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:02:56.109259  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.005s	sys 0.010s
I20260709 13:02:56.071079  6621 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:56.110984  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.088986  6513 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.004s	sys 0.009s
I20260709 13:02:56.073302  6693 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.079156  6355 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:56.199599  6513 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.005s	sys 0.009s
I20260709 13:02:56.058344  6906 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:56.216811  6876 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:02:56.150563  6877 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:56.227669  6903 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.558s	user 0.001s	sys 0.000s
W20260709 13:02:56.228739  6900 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.559s	user 0.000s	sys 0.001s
I20260709 13:02:56.227922  6820 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:56.448863  6860 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.108129  6912 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:56.404212  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 365515us
I20260709 13:02:56.464138  6862 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.117266  6763 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:56.464718  6763 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:56.121182  6901 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:56.405915  2292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.011s	sys 0.006s
W20260709 13:02:56.421970  6835 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.248s	user 0.012s	sys 0.006s
I20260709 13:02:56.742985  6835 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.427716  6853 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.744676  1706 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:56.744771  6853 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.745332  6835 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:56.746140  2292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.225s	user 0.012s	sys 0.012s
I20260709 13:02:56.746162  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 866) took 1560 ms. Trace:
I20260709 13:02:56.747351  1706 rpcz_store.cc:276] 0709 13:02:55.185653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.209088 (+ 23435us) service_pool.cc:224] Handling call
0709 13:02:56.745973 (+1536885us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003e1f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.746099 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3529927936}
I20260709 13:02:56.748840  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.750408  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.751317  6835 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:56.427852  6748 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.428146  6913 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:56.752215  6748 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.437933  6838 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.752739  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:56.753392  6835 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.754453  6748 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:56.438112  6803 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.438843  6917 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:56.755254  6917 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.756898  6894 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:56.757378  6894 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.757464  6917 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.757891  6894 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:56.759022  6835 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.439064  6648 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.439813  6621 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.760106  6648 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.762387  6648 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.440305  6598 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:56.765365  6598 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:56.442320  6869 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:56.430836  6737 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.434414  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.421439  6819 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.121s	user 0.013s	sys 0.013s
I20260709 13:02:56.783210  1701 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:02:56.783260  6819 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.784677  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.785655  6819 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.463800  6860 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.408823  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.421996  6848 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.123s	user 0.008s	sys 0.017s
I20260709 13:02:56.786540  1709 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:56.786597  6848 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.787437  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:56.789016  6848 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.787631  6860 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:56.456734  6559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.010s	sys 0.006s
W20260709 13:02:56.789767  6559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.010s	sys 0.007s
W20260709 13:02:56.484315  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.430486  6854 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.756s	user 0.010s	sys 0.005s
W20260709 13:02:56.790607  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.790426  6854 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.792207  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.792431  6854 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.433451  6915 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:56.792936  1706 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:56.793129  6617 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.793197  6915 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:56.794412  6915 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:56.797034  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2615) took 1736 ms (client timeout 1717 ms). Trace:
I20260709 13:02:56.433233  6693 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.435304  6652 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.465538  6912 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:56.465723  6901 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:56.798303  6693 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.798606  6901 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.798825  6693 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.801009  6693 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.432727  6525 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.476598  6820 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:56.805735  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:56.806672  1705 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 9: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:56.806443  6820 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.435607  6921 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
W20260709 13:02:56.540968  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.808797  1524 connection.cc:256] RPC call timeout handler was delayed by -1.4269195499999796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 13:02:56.809592  6598 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.810986  6598 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:56.811107  6734 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:56.542842  6355 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.812064  6355 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.812526  6355 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.812729  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.813410  1555 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:56.814301  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.814486  6355 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.814883  1555 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:02:56.815009  6355 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.507673  6924 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:56.602885  6899 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.306s	user 0.006s	sys 0.011s
I20260709 13:02:56.465536  6763 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.604177  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.606855  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2616) took 1539 ms. Trace:
I20260709 13:02:56.550534  6106 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:56.599306  6903 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.077s	user 0.000s	sys 0.006s
I20260709 13:02:56.592648  6877 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:56.665764  6804 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.430s	user 0.018s	sys 0.000s
I20260709 13:02:56.816057  6924 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.822949  6804 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.677292  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.678095  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:56.674890  6830 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.012s	user 0.007s	sys 0.010s
W20260709 13:02:56.675114  6868 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.900s	user 0.013s	sys 0.004s
W20260709 13:02:56.676035  6706 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.312s	user 0.005s	sys 0.012s
W20260709 13:02:56.675750  6720 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.926s	user 0.013s	sys 0.004s
I20260709 13:02:56.669421  6766 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:56.839118  6706 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.805765  6525 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.839315  6766 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.840637  6924 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.681808  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:56.841426  6706 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.815527  6355 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.842376  6763 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:56.842110  6720 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.843581  6355 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.844437  6720 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.841466  6766 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:56.848626  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2622) took 2845 ms (client timeout 1668 ms). Trace:
W20260709 13:02:56.849071  1559 rpcz_store.cc:269] 0709 13:02:54.003191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.109891 (+106700us) service_pool.cc:224] Handling call
0709 13:02:56.848388 (+2738497us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54002bacd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.848523 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802072960}
I20260709 13:02:56.850836  1634 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:02:56.852097  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 170203us
I20260709 13:02:56.853307  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.854123  1628 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:56.854871  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.855937  6854 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:56.857435  6854 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:56.860288  1628 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:56.861346  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:56.863884  6894 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:56.683969  6920 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:02:56.684231  6906 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.685673  6932 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:56.686829  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2656) took 3807 ms. Trace:
I20260709 13:02:56.675448  6927 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.865278  6932 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.865269  1704 rpcz_store.cc:276] 0709 13:02:52.879193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.952318 (+ 73125us) service_pool.cc:224] Handling call
0709 13:02:54.919058 (+1966740us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:02:54.919330 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.974070 (+ 54740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.686516 (+1712446us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.686748 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84308608}
I20260709 13:02:56.715747  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2603) took 4328 ms. Trace:
W20260709 13:02:56.865139  1551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2620) took 2862 ms (client timeout 1596 ms). Trace:
I20260709 13:02:56.866897  1557 rpcz_store.cc:276] 0709 13:02:52.387318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.832577 (+445259us) service_pool.cc:224] Handling call
0709 13:02:52.832934 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.302159 (+2469225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.715350 (+1413191us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b54000776d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.715615 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5627012992}
W20260709 13:02:56.867050  1551 rpcz_store.cc:269] 0709 13:02:54.002609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.029615 (+ 27006us) service_pool.cc:224] Handling call
0709 13:02:56.864944 (+2835329us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54002f9c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.865067 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173745024}
I20260709 13:02:56.717177  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2613) took 1674 ms. Trace:
I20260709 13:02:56.868362  1627 rpcz_store.cc:276] 0709 13:02:55.042707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.189807 (+147100us) service_pool.cc:224] Handling call
0709 13:02:55.190066 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.351630 (+161564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.716861 (+1365231us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b5400274cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.717085 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172907904}
I20260709 13:02:56.713889  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.718384  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2763) took 1661 ms (client timeout 2078 ms). Trace:
I20260709 13:02:56.720827  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.869400  1635 rpcz_store.cc:269] 0709 13:02:55.056667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.089713 (+ 33046us) service_pool.cc:224] Handling call
0709 13:02:56.718175 (+1628462us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002977d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.718311 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426721536}
I20260709 13:02:56.732550  6856 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.436676  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2617) took 1356 ms. Trace:
I20260709 13:02:56.868142  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:56.752272  6913 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.429384  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.782285  6878 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:56.778272  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2008) took 1328 ms. Trace:
I20260709 13:02:56.894917  1700 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:56.894588  1554 rpcz_store.cc:276] 0709 13:02:55.449579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.452912 (+  3333us) service_pool.cc:224] Handling call
0709 13:02:56.778091 (+1325179us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54002ba2d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.778208 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354316672}
I20260709 13:02:56.781688  6813 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.786027  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.797468  1631 rpcz_store.cc:269] 0709 13:02:55.060352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.066273 (+  5921us) service_pool.cc:224] Handling call
0709 13:02:56.796851 (+1730578us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:56.796959 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563558656}
I20260709 13:02:56.907909  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.908790  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:56.852115  6898 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.472s	user 0.012s	sys 0.005s
I20260709 13:02:56.816152  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.617767  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2623) took 1279 ms. Trace:
I20260709 13:02:56.930104  1633 rpcz_store.cc:276] 0709 13:02:55.338607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.524807 (+186200us) service_pool.cc:224] Handling call
0709 13:02:55.525158 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.607387 (+ 82229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:55.607427 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.615628 (+  8201us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.109786 (+494158us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:56.109845 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.109958 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.617503 (+507545us) raft_consensus.cc:1685] finished
0709 13:02:56.617532 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.617699 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4293,"raft.queue_time_us":357508,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17237,"thread_start_us":357284,"threads_started":1,"wal-append.queue_time_us":843811}
I20260709 13:02:56.932622  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:56.933564  1633 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:56.934665  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.935385  1633 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 10.
I20260709 13:02:56.815737  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.936828  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:02:56.819867  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:56.666004  6902 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.006s
I20260709 13:02:56.938124  6902 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.822938  6877 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.941239  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.941655  6877 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:56.669035  6849 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.374s	user 0.012s	sys 0.017s
I20260709 13:02:56.942353  6849 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.945068  6849 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.822157  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:56.824476  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:56.945971  1705 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:56.946120  1628 raft_consensus.cc:2368] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:56.947476  6902 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.816974  1632 rpcz_store.cc:276] 0709 13:02:55.079897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.196744 (+116847us) service_pool.cc:224] Handling call
0709 13:02:55.197034 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.238845 (+ 41811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.238875 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.238912 (+    37us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:55.238951 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.606591 (+1367640us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b54001419d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.606655 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.606791 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95738624}
I20260709 13:02:56.821501  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:56.817203  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.827666  6830 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.828756  6868 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.963148  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:56.963846  2289 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:56.829805  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.830577  6106 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:56.830884  1709 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:56.831425  6804 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:56.836501  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:02:56.965386  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 128954us
W20260709 13:02:56.965906  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 143872us
I20260709 13:02:56.864277  6350 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:56.865785  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2663) took 1814 ms. Trace:
W20260709 13:02:56.735095  6905 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.778s	user 0.000s	sys 0.001s
I20260709 13:02:56.736374  6489 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.855655  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:02:56.682145  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 229385us
I20260709 13:02:56.953112  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2767) took 1160 ms. Trace:
I20260709 13:02:56.881166  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:02:56.973444  2263 connection.cc:256] RPC call timeout handler was delayed by -2.9548374350000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 13:02:56.974179  1627 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
W20260709 13:02:56.974361  2263 connection.cc:256] RPC call timeout handler was delayed by -2.7091374350000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 13:02:56.886093  1675 connection.cc:256] RPC call timeout handler was delayed by -1.4481594599999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:02:56.974731  2263 connection.cc:256] RPC call timeout handler was delayed by -2.6230553290000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 13:02:56.974872  1675 connection.cc:256] RPC call timeout handler was delayed by -1.4049594599999864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 13:02:56.975203  1675 connection.cc:256] RPC call timeout handler was delayed by -1.3833594600000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:02:56.975260  2263 connection.cc:256] RPC call timeout handler was delayed by -2.5407132760000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
I20260709 13:02:56.890981  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.872690  1629 rpcz_store.cc:276] 0709 13:02:55.080724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.288844 (+208120us) service_pool.cc:224] Handling call
0709 13:02:55.289184 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.437187 (+148003us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:55.454253 (+ 17066us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.612120 (+157867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.612191 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:55.612259 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.436445 (+824186us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002986d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.436492 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.436620 (+   128us) inbound_call.cc:177] Queueing success response
0709 13:02:56.734510 (+297890us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
Metrics: {"mutex_wait_us":12085,"prepare.queue_time_us":12505,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":364274,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":15732,"spinlock_wait_cycles":356179584,"thread_start_us":77081,"threads_started":1,"wal-append.queue_time_us":301233}
I20260709 13:02:56.976167  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:56.898586  6873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.006s	sys 0.010s
I20260709 13:02:56.911659  6835 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:56.909578  1631 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:56.811730  6734 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.881664  6894 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.894353  6878 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:56.865643  6927 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.917621  1701 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:56.922223  6932 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.928737  6913 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:56.929203  6921 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.994971  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.972733  6877 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:56.951550  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:56.962257  1706 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:56.962239  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:56.829188  1707 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:02:57.009238  1707 pending_rounds.cc:85] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 7
I20260709 13:02:57.009567  1707 pending_rounds.cc:107] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:02:57.009867  1707 raft_consensus.cc:2889] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.967118  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:56.969259  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 147877us
I20260709 13:02:56.978761  6598 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:57.013460  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:56.996459  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.014633  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:56.968876  1703 rpcz_store.cc:276] 0709 13:02:55.051553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.052100 (+   547us) service_pool.cc:224] Handling call
0709 13:02:55.052437 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.274866 (+222429us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:55.274897 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.279689 (+  4792us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.113890 (+834201us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:56.113931 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.719342 (+605411us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54001333d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:56.719411 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.865510 (+146099us) raft_consensus.cc:1685] finished
0709 13:02:56.865528 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.865727 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3134,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10858,"spinlock_wait_cycles":510607232,"thread_start_us":807615,"threads_started":1,"wal-append.queue_time_us":1149161}
I20260709 13:02:56.972015  2293 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:56.880429  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:02:56.975623  2263 connection.cc:256] RPC call timeout handler was delayed by -2.1365747430000397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:02:57.016907  2263 connection.cc:256] RPC call timeout handler was delayed by -2.1363165089999825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:02:57.017355  2263 connection.cc:256] RPC call timeout handler was delayed by -1.8847374350000337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2965s for a response
I20260709 13:02:57.007634  1635 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:02:56.978924  6763 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:56.978297  6766 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:57.020857  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.021049  6766 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:56.986150  6835 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:57.021977  6835 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:57.022471  6535 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.023305  6766 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.021092  6848 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:57.025751  6927 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:56.989892  6513 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.026742  6513 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:56.989102  6849 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:56.981189  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:56.981732  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 126139us
I20260709 13:02:56.990373  1705 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
W20260709 13:02:56.978261  6873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.006s	sys 0.010s
I20260709 13:02:56.973238  6906 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:56.995639  1557 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:56.969044  6820 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.816867  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:56.968813  6350 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:57.000689  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2606) took 4612 ms. Trace:
I20260709 13:02:57.075821  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.080065  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:57.047277  1709 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:57.087010  6906 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:57.087230  6820 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.087565  6906 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.973428  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.087957  6846 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:57.088634  6848 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:57.088781  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115414us
I20260709 13:02:57.089131  6848 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:57.089632  6848 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:57.089718  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.090528  6734 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.091063  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:56.986932  6902 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.091887  6535 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:57.092338  6535 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.092027  6902 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.092763  6535 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.093106  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.094208  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.094316  6902 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.018275  6598 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.021510  6830 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.094966  6598 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:57.095842  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.096194  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.098347  6513 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.076536  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.075138  6748 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.103547  6748 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.105717  6748 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:57.089640  6820 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.002023  6863 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:57.087040  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.108433  6863 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.108500  6535 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.108839  1558 rpcz_store.cc:276] 0709 13:02:52.388152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.858542 (+470390us) service_pool.cc:224] Handling call
0709 13:02:52.858846 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.859038 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:53.349692 (+490654us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.096666 (+746974us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.637979 (+2541313us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18004cd3e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:56.774353 (+136374us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18004cd3e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:02:57.000366 (+226013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.000416 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:57.000466 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.000527 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.000628 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208575,"prepare.queue_time_us":760399,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":1655,"raft.queue_time_us":453780,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":141302,"spinlock_wait_cycles":49339392,"thread_start_us":1285027,"threads_started":3,"wal-append.queue_time_us":1570284,"wal-append.run_cpu_time_us":1519,"wal-append.run_wall_time_us":1006493}
I20260709 13:02:57.110785  6535 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.111307  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.083020  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:56.981439  6904 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:56.973572  1626 rpcz_store.cc:276] 0709 13:02:55.792740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.881459 (+ 88719us) service_pool.cc:224] Handling call
0709 13:02:55.881845 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.882132 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:55.924524 (+ 42392us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.697601 (+773077us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b18004b0dc0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:02:56.952645 (+255044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.952724 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:56.952796 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.952883 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.953028 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49826,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":17558,"raft.queue_time_us":59188,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5316,"spinlock_wait_cycles":2732928,"thread_start_us":204870,"threads_started":1,"wal-append.queue_time_us":978186}
I20260709 13:02:56.980669  1554 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:57.132339  1701 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:57.133153  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.133803  1554 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.012735  6877 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:57.028647  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.028934  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:57.029194  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.141208  1632 raft_consensus.cc:2368] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:02:57.030615  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:56.947649  6901 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.028158  6355 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:02:57.032678  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.144892  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:57.145905  1557 raft_consensus.cc:2410] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:57.084717  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:57.147091  2293 raft_consensus.cc:2368] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:57.112596  6911 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.112686  6763 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.113885  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.115093  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:02:57.017735  2263 connection.cc:256] RPC call timeout handler was delayed by -1.7338841579999666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
I20260709 13:02:57.159550  1627 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:57.159705  2263 connection.cc:256] RPC call timeout handler was delayed by -1.646584157999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:02:57.074780  6868 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:56.981561  6918 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:57.108685  1556 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:57.084337  6849 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:57.086233  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.106395  6848 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.125221  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.094791  2290 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
W20260709 13:02:57.130746  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47365.3us
I20260709 13:02:57.027829  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:57.142824  2294 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:02:57.144100  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.111822  6910 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:57.145444  6877 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:57.158054  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 129184us
I20260709 13:02:56.995174  6544 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:02:57.161458  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:56.901031  6855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.527s	user 0.000s	sys 0.025s
I20260709 13:02:57.162058  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:57.163034  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132629us
I20260709 13:02:57.015168  6921 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.116780  1558 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:57.166581  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.166761  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.197324  1557 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:57.198580  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.166220  6617 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:57.167198  6648 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:57.208349  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.210023  1552 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:57.154568  1554 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:57.213620  1627 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:57.173377  6910 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:57.184865  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 209076us
I20260709 13:02:57.191478  2293 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:02:57.163924  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48952.4us
I20260709 13:02:57.222128  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.222465  1560 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:57.194008  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:57.195102  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.195152  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.165524  6856 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:57.028054  6853 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.135s
I20260709 13:02:57.203636  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.012090  6813 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.180s
I20260709 13:02:57.207437  6617 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.198328  6598 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.208146  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.143508  6904 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:57.216902  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 226595us
W20260709 13:02:57.194991  6855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.817s	user 0.000s	sys 0.026s
I20260709 13:02:57.224185  1559 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.179157  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.257587  2294 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:57.258924  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.230823  6648 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:57.212829  6937 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:57.268698  6648 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:57.226137  6734 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:57.269305  6648 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.231132  6617 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:57.231067  6939 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.271406  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.198740  6932 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:57.273010  1554 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:57.274171  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45724.5us
I20260709 13:02:57.284312  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.277033  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.326200  2293 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:57.329986  1705 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:57.227229  6910 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.221005  6860 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:57.210544  6902 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
W20260709 13:02:57.272320  6930 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:02:57.333907  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 58000.1us
I20260709 13:02:57.344725  2288 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:57.345261  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.227234  6856 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:57.218536  6873 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.171116  6918 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.269055  6937 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.357265  6838 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:57.362128  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 130773us
I20260709 13:02:57.168488  6911 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.394781  6706 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:57.395501  6706 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.323021  6912 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:57.413594  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.416978  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:57.273416  6861 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:02:57.426791  2291 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:02:57.278898  6836 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:02:57.428856  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:57.436326  6945 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:02:57.437662  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 989) took 1917 ms. Trace:
I20260709 13:02:57.438141  2292 rpcz_store.cc:276] 0709 13:02:55.520402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.520574 (+   172us) service_pool.cc:224] Handling call
0709 13:02:55.521011 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.746617 (+1225606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.746647 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.747128 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.000124 (+252996us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:57.000166 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.000228 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.437420 (+437192us) raft_consensus.cc:1685] finished
0709 13:02:57.437440 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.437593 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":231391,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":39493,"thread_start_us":1477713,"threads_started":2,"wal-append.queue_time_us":390987}
I20260709 13:02:57.329252  6693 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:02:57.350817  6863 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.465641  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.452149  1673 leader_election.cc:400] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:57.466926  1673 leader_election.cc:403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.375480  6856 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:57.277699  6947 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.213s
I20260709 13:02:57.471184  1553 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:57.471436  6947 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:57.471979  6947 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:57.481849  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.384572  6838 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:57.525328  1627 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:57.483225  2297 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:57.377840  6860 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.397343  6873 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.550796  6901 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.407332  6350 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:57.568430  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 284223us
I20260709 13:02:57.568699  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.570181  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.571031  1708 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:57.349887  6849 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.569716  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:57.572806  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.577212  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.581110  1627 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:57.579856  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:57.582819  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.583420  1631 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:57.584089  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.573352  1556 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:57.585386  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.585644  6813 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:57.586140  6813 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.586362  6940 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:57.586542  6813 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:57.586772  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:57.587437  2292 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.588137  6489 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:57.586882  6940 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.565491  6763 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.607887  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.589766  1629 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:57.590502  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.590441  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.611196  1627 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:57.590384  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.594071  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.581607  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:57.613119  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:02:57.614220  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.596211  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.597754  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:57.591512  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 181687us
I20260709 13:02:57.617208  1702 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:57.617537  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.589259  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.601955  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.602651  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.619822  1703 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.579104  6688 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.621089  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.607559  6940 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.626515  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.627892  1557 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:57.628686  1628 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:57.629484  1525 leader_election.cc:400] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:02:57.630053  1525 leader_election.cc:403] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.631707  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.634231  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.636366  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.639012  6513 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 9. Current term is 10.
I20260709 13:02:57.633335  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:02:57.653944  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.589649  6350 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.588483  6911 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.602075  6956 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:57.625595  6688 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.642866  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.641531  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:57.652458  1708 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:57.647377  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.661993  1559 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:02:57.602785  6861 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:02:57.632251  6835 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.003s
I20260709 13:02:57.654534  6489 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:57.609395  6848 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.623664  6912 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.671968  6804 raft_consensus.cc:2764] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:57.672192  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.705452  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:57.672739  6873 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:57.677927  1703 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
I20260709 13:02:57.708089  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.709524  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.678929  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.711140  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.712352  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.670224  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:57.713624  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43508.2us
I20260709 13:02:57.685993  2292 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:02:57.687546  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46120.5us
I20260709 13:02:57.636653  6355 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:57.687831  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:57.629813  6853 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.689591  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 466618us
I20260709 13:02:57.656001  6855 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.691715  6544 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:57.589890  6838 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.694399  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.731041  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.753506  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.721087  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.706791  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.686825  6688 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.739852  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.756970  2296 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:57.689177  6901 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:57.740465  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.596s (SENT)
I20260709 13:02:57.758138  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.757691  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:57.738356  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.759105  1628 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.676153  6937 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.730518  1632 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:57.764010  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.742489  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.691826  6911 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:57.746052  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:57.768669  1559 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.748183  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:57.720010  6934 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:57.694808  6836 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:02:57.555608  6947 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.746045  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:57.736635  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48944.2us
I20260709 13:02:57.760237  1631 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:02:57.755369  1708 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.766429  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:02:57.767206  1558 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.570s	user 0.017s	sys 0.001s
I20260709 13:02:57.744939  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:57.798707  2292 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:57.799674  1558 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:57.690891  6598 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.691910  6861 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.738389  6918 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.808118  1708 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:57.809219  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:57.741791  6855 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:57.756038  6863 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.771088  6706 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.805635  1628 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:02:57.800963  1702 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:57.743238  6902 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:57.797531  6911 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:57.814595  6957 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:57.797159  6937 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:57.816408  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:57.863584  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 282180us
I20260709 13:02:57.802716  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.751724  6853 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.792395  6901 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:57.770154  6904 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:57.695178  6835 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.820711  1702 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:02:57.811165  1559 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:02:57.820923  6863 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.819594  6861 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.816047  6855 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.865432  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.858290  6947 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.866765  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.790813  6940 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.875294  1702 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:57.866446  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.882383  1631 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:02:57.885092  2292 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:57.885557  1702 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:57.885969  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.886540  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:02:57.886657  1702 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:57.887380  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.888182  6911 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:57.890187  6853 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.890870  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:02:57.891561  1628 raft_consensus.cc:606] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:57.894200  2292 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:57.856410  6902 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:57.896323  6902 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.897290  6863 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:57.899986  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.900177  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:57.900906  1633 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 13:02:57.888229  6959 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:02:57.865226  6873 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:57.901989  6940 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:57.902081  6873 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:57.902654  6873 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:57.904417  6902 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:57.906788  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:57.915185  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:57.883692  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:57.922564  1700 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:57.876006  6939 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:02:57.937702  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.876726  6860 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:57.935353  6748 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:57.963248  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 393550us
I20260709 13:02:57.964663  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:57.966985  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 385413us
I20260709 13:02:57.989828  1525 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:02:57.990352  1525 leader_election.cc:403] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.991159  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:58.070258  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 79184us
I20260709 13:02:58.079512  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 814) took 1269 ms. Trace:
I20260709 13:02:58.104250  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2618) took 4102 ms. Trace:
I20260709 13:02:58.073246  6734 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
W20260709 13:02:58.080860  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:02:58.093721  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.983995  6748 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:58.100792  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 354792us
I20260709 13:02:58.104784  1553 rpcz_store.cc:276] 0709 13:02:54.002032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.608289 (+606257us) service_pool.cc:224] Handling call
0709 13:02:54.608697 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.471691 (+2862994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.101044 (+629353us) spinlock_profiling.cc:243] Waited 2.86 s on lock 0x7b54000767d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:58.104183 (+  3139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6569193728}
I20260709 13:02:58.103974  1552 rpcz_store.cc:276] 0709 13:02:56.810550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.815870 (+  5320us) service_pool.cc:224] Handling call
0709 13:02:58.079236 (+1263366us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540036f1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:58.079416 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643485952}
I20260709 13:02:57.992635  6962 raft_consensus.cc:2764] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:58.081095  6489 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:58.073762  6598 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.118736  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 378267us
W20260709 13:02:58.142198  1675 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.668s (SENT)
I20260709 13:02:58.142274  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2684) took 1304 ms. Trace:
I20260709 13:02:58.142695  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.121258  1674 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:58.143405  1674 leader_election.cc:403] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:02:58.128842  1632 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:58.106402  6544 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:58.061522  6863 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:02:58.140224  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2026) took 1152 ms. Trace:
I20260709 13:02:58.151571  1554 rpcz_store.cc:276] 0709 13:02:56.987573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.233721 (+246148us) service_pool.cc:224] Handling call
0709 13:02:57.234090 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.280253 (+ 46163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.139879 (+859626us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b540036d3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:58.140131 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88860928}
I20260709 13:02:58.142915  1701 rpcz_store.cc:276] 0709 13:02:56.837719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.989526 (+151807us) service_pool.cc:224] Handling call
0709 13:02:57.132081 (+142555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.132964 (+   883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.141966 (+1009002us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000424d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:58.142187 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327123712}
I20260709 13:02:58.148072  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:02:58.149097  1675 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.730s (SENT)
W20260709 13:02:58.154311  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.747s (SENT)
I20260709 13:02:58.117713  6748 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.155045  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.164454  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.823s (SENT)
I20260709 13:02:58.164803  1674 leader_election.cc:400] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:02:58.165969  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 2.007s (SENT)
I20260709 13:02:58.166656  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.169317  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.05188e+06us
I20260709 13:02:58.171397  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.171787  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.172293  1552 raft_consensus.cc:2368] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10.
I20260709 13:02:58.173916  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.174237  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.171861  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.192350  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.194715  1553 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:02:58.117940  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44375.1us
I20260709 13:02:58.205824  1554 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:58.207526  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.061565  6901 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.208506  1554 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:02:58.211048  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.211637  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:58.213250  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87900.3us
I20260709 13:02:58.150190  6951 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:02:58.163553  6877 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.968s	user 0.008s	sys 0.017s
I20260709 13:02:58.211951  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.215356  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.148437  6489 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:58.219355  6973 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:58.174259  6846 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.148231  6598 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.209707  6934 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:58.257483  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2020) took 1275 ms. Trace:
I20260709 13:02:58.255478  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:02:58.258183  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65461.4us
I20260709 13:02:58.257911  1551 rpcz_store.cc:276] 0709 13:02:56.982130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.982251 (+   121us) service_pool.cc:224] Handling call
0709 13:02:56.982525 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.115230 (+132705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.115261 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.115801 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.277894 (+162093us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:57.277954 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.250252 (+972298us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54000e9554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:58.250333 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.250408 (+    75us) raft_consensus.cc:1685] finished
0709 13:02:58.250430 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.257407 (+  6977us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16565,"raft.queue_time_us":3218,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":4504,"spinlock_wait_cycles":304286336,"thread_start_us":161731,"threads_started":1,"wal-append.queue_time_us":523866}
W20260709 13:02:58.250804  6958 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
W20260709 13:02:58.271374  6617 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.000s	user 0.030s	sys 0.034s
I20260709 13:02:58.257632  6934 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:58.280177  6915 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:58.312292  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44275.2us
I20260709 13:02:58.288094  6544 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.223515  6974 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:58.307029  1552 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:58.397208  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 231251us
I20260709 13:02:58.398170  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.254156  6937 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:58.288573  6965 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:02:58.309160  6846 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:58.314852  6598 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:58.315629  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.316792  6860 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:58.316617  6855 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.261998  6835 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.312309  6915 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:58.318068  6350 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.003s
I20260709 13:02:58.319099  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.319367  6906 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.163s	user 0.009s	sys 0.007s
I20260709 13:02:58.312326  6617 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.312556  6934 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:58.441200  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87939.3us
W20260709 13:02:58.323644  6910 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.945s	user 0.013s	sys 0.008s
I20260709 13:02:58.324823  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.325610  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.446116  1709 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:02:58.325996  6964 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:02:58.326864  6963 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:02:58.329967  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.406980  6846 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:58.330963  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.331552  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.334507  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.335000  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.457960  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:58.290809  6945 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:58.399492  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:58.401472  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 185040us
I20260709 13:02:58.465997  1551 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:02:58.433316  1557 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:02:58.410673  6688 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.002s
I20260709 13:02:58.323462  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.466109  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.467999  1702 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:58.441982  1524 connection.cc:256] RPC call timeout handler was delayed by -1.645071540999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20850000000000002s for a response
I20260709 13:02:58.469962  1552 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:58.470180  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2018) took 1490 ms. Trace:
I20260709 13:02:58.470657  1555 rpcz_store.cc:276] 0709 13:02:56.979591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.979718 (+   127us) service_pool.cc:224] Handling call
0709 13:02:56.996943 (+ 17225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.001135 (+  4192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.001165 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.032402 (+ 31237us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.656142 (+623740us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.656201 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.721279 (+ 65078us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:57.721347 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.457562 (+736215us) raft_consensus.cc:1685] finished
0709 13:02:58.457583 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.470099 (+ 12516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9088256,"thread_start_us":490998,"threads_started":1,"wal-append.queue_time_us":667866}
I20260709 13:02:58.473922  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.474735  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:58.443233  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 124213us
I20260709 13:02:58.447182  1703 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:58.330675  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.453449  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.249902  6877 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.477176  1701 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:02:58.476263  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:58.478670  6835 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.478720  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.488519  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.480191  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.489657  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.490420  1552 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:02:58.491106  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.491201  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.492216  6835 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.481093  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.484340  6617 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.485107  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.478796  6937 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.495967  6693 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:58.495796  6937 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.496467  6693 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:58.496975  6693 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:58.497381  1709 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:02:58.498281  6937 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:58.486877  6856 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.000s	user 0.007s	sys 0.007s
I20260709 13:02:58.499302  6856 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.499698  6877 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.500118  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.501428  6856 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.503105  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.489553  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.442477  6906 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.505249  2289 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:58.505565  6878 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.506178  6906 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:58.506429  6911 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:58.506852  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 993) took 1474 ms. Trace:
I20260709 13:02:58.453095  6910 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.512708  6838 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.512032  6734 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:58.507385  6911 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:58.514281  6910 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.515194  1633 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:58.515570  6911 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:58.507279  2289 rpcz_store.cc:276] 0709 13:02:57.032216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.032431 (+   215us) service_pool.cc:224] Handling call
0709 13:02:58.506627 (+1474196us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54005e06d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:58.506769 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3127280384}
I20260709 13:02:58.478493  6688 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.516194  6911 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:58.510246  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.517431  1551 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:58.518033  6688 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.508076  2296 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:58.518123  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 119650us
I20260709 13:02:58.518738  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.517549  6911 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.520097  6688 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:58.520205  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.520603  6688 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.521067  6688 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:58.522454  1702 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:02:58.525156  6350 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:58.525607  6350 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:58.525988  6350 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:58.526682  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:02:58.527810  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.528611  6904 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:58.528997  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.529281  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 213718us
I20260709 13:02:58.529815  6933 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.506686  6878 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.529886  6918 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:58.530686  6918 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:58.531083  6918 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:58.532425  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.535046  6951 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.510663  6598 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:02:58.537209  6813 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.003s
I20260709 13:02:58.537680  6813 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.538075  6921 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.538885  1553 raft_consensus.cc:2368] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:02:58.539951  6846 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.541252  6846 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:58.541440  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.542179  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.542955  1558 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:58.451333  6863 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:58.423144  6489 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.403363  6971 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:02:58.548521  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.553862  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.544123  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.555646  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:58.561398  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.562265  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:58.562978  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.550601  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:58.553156  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.543738  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:58.565016  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.565196  1558 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.565586  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:58.566744  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:58.567899  1631 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:58.569685  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.569725  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.571009  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.571803  1555 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:58.572813  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:58.547101  6648 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.277s	user 0.037s	sys 0.025s
W20260709 13:02:58.551093  6956 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.000s
I20260709 13:02:58.549360  6863 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:02:58.495185  6873 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.000s
I20260709 13:02:58.569991  6598 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:58.465193  6748 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.573669  6836 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:58.529039  6904 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:58.580752  6933 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:58.608958  6971 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:58.572240  6906 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:58.571417  6820 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.589586  6849 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:58.607841  6863 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:58.545121  6932 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:02:58.573165  6968 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:58.568092  6106 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:02:58.589294  6967 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:58.571244  6972 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:58.618793  6947 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 13:02:58.611368  6956 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:58.568406  6877 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:02:58.602197  6648 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.619068  6904 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.612267  6873 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.647446  6855 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.670647  6873 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:58.643193  6813 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:58.648838  6877 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:58.676244  6934 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.678150  6918 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.625614  6820 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.641170  6968 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:58.668272  6976 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:02:58.668619  6849 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:58.676007  6951 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.676649  6877 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:58.549062  6915 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.678560  6934 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.622584  6971 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:58.585369  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:58.591394  1629 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:02:58.579046  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:58.567589  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.730321  1552 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:58.777070  1701 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:58.780303  6836 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:58.693460  6968 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.792342  6489 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:58.686874  6971 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.793191  6489 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:58.792661  6849 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.794986  6971 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.690061  6877 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.796774  1626 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:58.797936  1706 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:58.798079  6877 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.801528  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.732450  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.689033  6918 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:58.679592  6820 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.790657  6836 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.729008  1551 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
W20260709 13:02:58.798146  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 193894us
I20260709 13:02:58.803753  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.821885  1559 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:58.821358  6873 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:58.805506  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.787946  1704 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:58.787000  6973 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:58.823504  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.824977  1555 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
W20260709 13:02:58.791297  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 223585us
I20260709 13:02:58.833195  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:58.849088  6648 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.795082  6489 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.852545  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.856149  6873 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:58.837996  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.833525  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.839306  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.860128  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.861364  1627 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:02:58.861022  6956 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:58.861853  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2035) took 1114 ms. Trace:
I20260709 13:02:58.862457  1557 rpcz_store.cc:276] 0709 13:02:57.747446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.747873 (+   427us) service_pool.cc:224] Handling call
0709 13:02:58.847708 (+1099835us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400128ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:58.861600 (+ 13892us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524513536}
I20260709 13:02:58.866403  1557 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:58.866660  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 287658us
I20260709 13:02:58.868038  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:58.869169  1557 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:58.871361  1557 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.872509  6956 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:58.874876  6918 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:58.877169  6838 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:58.878430  1555 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:58.878887  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 272507us
I20260709 13:02:58.880975  6994 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:58.882197  1627 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:58.879642  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 871) took 2153 ms. Trace:
I20260709 13:02:58.884531  6748 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.896687  6918 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:58.900040  6918 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:58.900192  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 314843us
I20260709 13:02:58.901553  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.902622  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:58.903820  6544 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:58.904510  6544 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:58.904117  6956 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.884649  6873 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:58.905849  6544 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.907234  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 880) took 1219 ms. Trace:
I20260709 13:02:58.907608  1703 rpcz_store.cc:276] 0709 13:02:57.687539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.709252 (+ 21713us) service_pool.cc:224] Handling call
0709 13:02:58.906752 (+1197500us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b540010d6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:58.907022 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694427904}
I20260709 13:02:58.908459  6956 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.909458  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.890314  6902 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.909628  6863 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.924135  6873 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:58.927445  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.830559  6350 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.915902  1708 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.916422  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:58.905675  6974 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:58.836839  6934 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:02:58.916689  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:58.918969  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 855) took 1322 ms. Trace:
I20260709 13:02:58.919595  6838 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.898201  1700 rpcz_store.cc:276] 0709 13:02:56.725826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.961846 (+236020us) service_pool.cc:224] Handling call
0709 13:02:56.962154 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.935679 (+973525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.879047 (+943368us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400042754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:58.879344 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2200289152}
I20260709 13:02:58.886080  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.922374  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.931574  1555 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:58.940534  1703 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:58.929008  2297 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.938800  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:02:58.939142  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.955785  6838 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:58.956418  1558 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:02:58.941051  1626 rpcz_store.cc:276] 0709 13:02:57.596318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.598395 (+  2077us) service_pool.cc:224] Handling call
0709 13:02:57.598771 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.797464 (+1198693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.918640 (+121176us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003ded54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:58.918876 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705912192}
W20260709 13:02:58.931712  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 237598us
I20260709 13:02:58.824270  6693 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:58.957894  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:58.958495  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:58.959352  6693 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:58.928929  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.934345  1599 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 2.619s (SENT) [suppressed 25 similar messages]
I20260709 13:02:58.937536  6974 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:58.935940  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:58.960634  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:02:58.960955  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58477.3us
I20260709 13:02:58.961671  6648 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:58.961748  6904 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:58.962960  1559 raft_consensus.cc:2410] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:58.964219  6968 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.936571  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:58.930131  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:58.928102  6863 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:58.937655  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:58.931470  6350 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:58.994115  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:58.973819  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:02:59.005028  2296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88dfd27965e145dcb810423d7396b403"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:02:59.006515  2296 raft_consensus.cc:606] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:59.008680  6974 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.009322  6820 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.010653  6974 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:59.011416  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.013383  2296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:02:59.014612  2296 raft_consensus.cc:606] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:59.015259  2296 raft_consensus.cc:614] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:59.017573  1700 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:59.018088  6918 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.019317  2296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:02:59.020009  1704 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:59.020334  2296 raft_consensus.cc:606] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:59.021023  2296 raft_consensus.cc:614] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:59.024493  1558 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:59.028810  1628 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:59.031838  6904 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:59.033569  2289 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:59.035863  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.039987  1627 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.040920  6918 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.045615  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:59.047077  7000 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:59.047855  7000 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.048135  6902 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:59.049321  6902 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:59.049703  6902 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.050726  6902 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.052533  6902 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.053014  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:59.052116  7000 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.056833  6918 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.057220  6918 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:59.057632  6918 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.039474  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:02:59.068257  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55987.6us
I20260709 13:02:59.047991  6648 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.094332  1633 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.083422  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:02:59.083811  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
W20260709 13:02:59.088256  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.093931  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 208227us
I20260709 13:02:59.088311  1705 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:02:59.089594  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:59.104772  1628 raft_consensus.cc:606] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:02:59.106961  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2087) took 1503 ms. Trace:
I20260709 13:02:59.107561  1627 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:02:59.107165  1707 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 13:02:59.107465  1708 rpcz_store.cc:276] 0709 13:02:57.603899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.810228 (+206329us) service_pool.cc:224] Handling call
0709 13:02:58.915381 (+1105153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.928296 (+ 12915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.105437 (+177141us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540010d6d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:59.106881 (+  1444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540302080}
I20260709 13:02:59.080466  6820 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.114737  6856 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:59.091928  6904 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:02:59.121112  6904 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:59.120747  6912 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:59.121575  6904 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.122953  6912 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:59.132294  6912 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:02:59.084936  6688 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.007s
I20260709 13:02:59.133208  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.133589  6688 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.133845  6912 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.119753  6878 raft_consensus.cc:993] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:02:59.136216  6940 raft_consensus.cc:2764] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:59.136045  6994 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:59.137113  6994 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:59.135011  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.137832  6994 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:59.138262  6994 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.138970  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.139228  6912 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:59.139521  6763 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:59.087877  6853 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.086480  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.087268  6693 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:02:59.169811  7003 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.150s	user 0.004s	sys 0.000s
I20260709 13:02:59.172452  1558 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
W20260709 13:02:59.173717  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60548.4us
W20260709 13:02:59.173164  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 238996us
I20260709 13:02:59.187939  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.176327  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.123020  6904 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.124509  6648 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.197917  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.191927  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.206643  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.206954  6933 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:59.207188  1708 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.207993  6693 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.211745  6921 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:02:59.187175  6763 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.215993  6763 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.203109  6878 raft_consensus.cc:1081] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
W20260709 13:02:59.220512  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60555.5us
I20260709 13:02:59.221663  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:02:59.222119  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2083) took 1648 ms (client timeout 2150 ms). Trace:
I20260709 13:02:59.222741  6835 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:59.222607  1705 rpcz_store.cc:269] 0709 13:02:57.573575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.575185 (+  1610us) service_pool.cc:224] Handling call
0709 13:02:59.221899 (+1646714us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003e1f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:59.222059 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734175616}
I20260709 13:02:59.223693  6835 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:59.224238  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 134846us
W20260709 13:02:59.224668  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.226640  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.227285  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.227711  6933 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:59.228296  6933 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:59.228722  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:02:59.228945  6940 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:59.230151  6940 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:59.230980  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 685546us
I20260709 13:02:59.228718  6933 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:59.232535  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.179484  6853 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.233630  6939 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.236435  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.236886  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.237140  1708 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.238550  1555 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:59.139487  6994 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.241062  6994 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.231005  6940 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:59.242084  6940 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.243278  6940 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:59.244468  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56782us
I20260709 13:02:59.244735  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.227798   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:59.223417  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:02:59.247561  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.251273  1633 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:59.255523  1555 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:59.268219  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.239509  6918 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.276913  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.283385  6940 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.285099  6939 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:59.289831  6734 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:59.290266  6734 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:02:59.294111  6913 consensus_queue.cc:1175] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b2c3934407d422495728f84374c6b53" responder_term: 13 status { last_received { term: 13 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 13 index: 10 } }
I20260709 13:02:59.294622  1707 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.296720  6913 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.299899  7002 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.300302  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.302309  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2032) took 2302 ms. Trace:
I20260709 13:02:59.302685  1560 rpcz_store.cc:276] 0709 13:02:57.000025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.223370 (+223345us) service_pool.cc:224] Handling call
0709 13:02:57.223742 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.318496 (+ 94754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.318543 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.360010 (+ 41467us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.114095 (+1754085us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b180034e520. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:02:59.254206 (+140111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:59.254254 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.267681 (+ 13427us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.301793 (+ 34112us) raft_consensus.cc:1685] finished
0709 13:02:59.301822 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.302104 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88865,"raft.queue_time_us":99526,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9519,"spinlock_wait_cycles":31477120,"thread_start_us":63990,"threads_started":1,"wal-append.queue_time_us":1931524}
I20260709 13:02:59.304626  6904 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.309139  6878 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:02:59.309845  6918 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.309993  6878 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.310626  6918 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:59.310891  1703 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:59.311583  1704 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 14. Ops: []
W20260709 13:02:59.312744  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44612.7us
I20260709 13:02:59.268643  6688 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:59.324321  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49449.9us
I20260709 13:02:59.291200  6734 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:59.332969  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.333683  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:59.267750  6838 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:59.335904  6836 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:59.336206  6838 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.337198  6836 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:59.337746  1553 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.338302  6836 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.338722  6836 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.339653  7000 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:59.340126  7000 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.341060  7000 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.342334  7000 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:59.344262  6106 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:59.345721  6106 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.346133  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.346962  1554 raft_consensus.cc:2368] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12.
I20260709 13:02:59.348826  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.350417  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.352262  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 885) took 1026 ms. Trace:
I20260709 13:02:59.352707  1701 rpcz_store.cc:276] 0709 13:02:58.325835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.478417 (+152582us) service_pool.cc:224] Handling call
0709 13:02:59.352006 (+873589us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400399c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:59.352180 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632273280}
I20260709 13:02:59.353991  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.354688  1701 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:02:59.355252  1701 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:59.316140  6934 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:59.356003  1702 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.300446  7002 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.357722  7002 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:59.377650  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 123094us
I20260709 13:02:59.311728  6918 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:59.377718  1708 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:59.384560  6836 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.386541  6856 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:59.387305  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:02:59.321097  6921 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:59.335536  6910 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:59.389468  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.393217  1553 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:59.357942  6915 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:59.394693  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 70668.1us
I20260709 13:02:59.333689  6734 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.361246  6968 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.400182  1707 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:02:59.398181  1701 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:59.346175  6106 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.402890  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2640) took 1672 ms. Trace:
I20260709 13:02:59.387816  6918 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.401300  6921 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:59.409327  1702 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.312323  6913 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:59.411859  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 188592us
I20260709 13:02:59.408311  1630 rpcz_store.cc:276] 0709 13:02:57.720222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.721433 (+  1211us) service_pool.cc:224] Handling call
0709 13:02:59.387819 (+1666386us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540019c654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:02:59.392547 (+  4728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416578432}
W20260709 13:02:59.416219  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 85851.4us
I20260709 13:02:59.424633  1703 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:59.422370  6688 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:59.428869  6968 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:59.429082  6688 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:59.429648  6968 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.430128  6688 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.431908  6972 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.432447  6972 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:02:59.433050  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.434808  6972 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.434919  6968 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.428902  1702 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:59.418861  6974 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:02:59.438241  6974 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.442134  6974 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.411338  6910 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.442265  6921 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.445351  1707 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:59.399785  6933 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.460566  6913 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:59.462152  7005 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.466s	user 0.001s	sys 0.004s
I20260709 13:02:59.465968  6693 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.470247  6830 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:02:59.473320  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.474228  1703 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:02:59.455096  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:02:59.435417  6972 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:02:59.423334  6763 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:02:59.488289  6918 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.489657  1675 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:59.451974  6836 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.458576  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.504055  1704 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:02:59.511044  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52557.5us
I20260709 13:02:59.511240  6947 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:59.514763  6939 consensus_queue.cc:1243] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:02:59.516047  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 92262.5us
I20260709 13:02:59.516396  1707 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:59.517411  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.521533  6873 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:59.524911  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 97826us
I20260709 13:02:59.525903  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:59.526580  6910 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.528054  6849 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:59.532486  6983 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:59.519594  1675 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:02:59.547080  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.527591  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:59.549683  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:59.556646  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.557569  1704 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:02:59.535377  6918 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.509307  6878 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.553681  6835 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:59.571697  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.571925  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.557113  6902 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:59.597266  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.598091  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.575081  6693 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:59.573455  6947 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:59.604365  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.604851  6997 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:59.605147  2289 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:59.593451  6489 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:02:59.575665  6877 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:02:59.583773  6902 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.605962  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.568528  6849 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:02:59.607966  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.646746  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 119318us
I20260709 13:02:59.528074  6106 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:02:59.653920  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 134471us
I20260709 13:02:59.655320  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.622376  6693 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:02:59.653916  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.644621  6902 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:59.657441  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.606010  6933 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:59.659996  1707 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:59.659421  6688 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.658607  6853 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.687971  1704 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:59.690945  2297 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:02:59.696422  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.677918  7022 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.701376  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53904.2us
I20260709 13:02:59.702395  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:59.707165  2295 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:59.686847  6962 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.708201  6962 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.708515  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.709997  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:02:59.708666  7025 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:59.707361  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.711778  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.713807  2288 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:59.711998  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.712414  1708 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:59.712741  2295 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:02:59.715236  7014 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:02:59.715128  6489 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:59.715433  6693 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.716007  6489 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:02:59.718132  6489 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:02:59.719115  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.719904  2288 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:02:59.720775  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.721851  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.724726  6962 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.726305  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.705646  7034 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.727996  2295 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:59.732384  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2098) took 1168 ms. Trace:
I20260709 13:02:59.733755  2296 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:02:59.732769  1700 rpcz_store.cc:276] 0709 13:02:58.564305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.019704 (+455399us) service_pool.cc:224] Handling call
0709 13:02:59.076938 (+ 57234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.208449 (+131511us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:59.391246 (+182797us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.583860 (+192614us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.643320 (+ 59460us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.643473 (+   153us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:59.643511 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.643584 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.732184 (+ 88600us) raft_consensus.cc:1685] finished
0709 13:02:59.732202 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.732321 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124042,"prepare.queue_time_us":250825,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":133193,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":7625,"thread_start_us":344233,"threads_started":2,"wal-append.queue_time_us":244568}
W20260709 13:02:59.736647  7020 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:02:59.737417  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.737934  1552 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:02:59.738269  1552 pending_rounds.cc:85] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:02:59.738481  1552 pending_rounds.cc:107] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:02:59.738741  1552 raft_consensus.cc:2889] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:59.740069  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.704730  6994 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.687479  6688 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.747720  6688 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.749810  7022 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.751943  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.752427  7022 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:02:59.752619  7008 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.755s	user 0.003s	sys 0.000s
I20260709 13:02:59.754576  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.755650  2296 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:02:59.756138  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106115us
I20260709 13:02:59.731141  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:02:59.767913  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.774559  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.779911  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.779958  6856 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:59.781486  6953 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:59.767750  6688 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:59.783156  6688 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.785745  6688 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:02:59.786834  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 104293us
I20260709 13:02:59.788024  2295 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:59.789304  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.790712  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.791464  1560 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 11: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:02:59.792387  2295 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:59.793605  2295 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:59.794557  1551 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:59.782512  6953 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:59.795914  6953 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:59.800117  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.800803  1559 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:59.765707  6106 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.801906  6911 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:59.802074  1551 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:59.807857  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.808564  2296 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:02:59.809572  1554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:02:59.809758  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.810256  1554 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:59.810716  1554 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:59.811100  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.812932  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2092) took 1257 ms. Trace:
I20260709 13:02:59.813316  1706 rpcz_store.cc:276] 0709 13:02:58.555482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.999743 (+444261us) service_pool.cc:224] Handling call
0709 13:02:59.035325 (+ 35582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.207520 (+172195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.207550 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.278774 (+ 71224us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.496170 (+217396us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:59.496560 (+   390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.496631 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.812687 (+316056us) raft_consensus.cc:1685] finished
0709 13:02:59.812707 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.812865 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112991,"raft.queue_time_us":129680,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":162,"thread_start_us":128020,"threads_started":1,"wal-append.queue_time_us":228976}
I20260709 13:02:59.800179  6953 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.818557  1706 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:59.820012  6953 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:59.821550  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.810473  2289 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:02:59.827652  1551 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:59.829253  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2655) took 1264 ms. Trace:
I20260709 13:02:59.829663  1556 rpcz_store.cc:276] 0709 13:02:58.564618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.920329 (+355711us) service_pool.cc:224] Handling call
0709 13:02:58.921290 (+   961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:58.921839 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.921867 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:58.922421 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.067705 (+145284us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:59.102205 (+ 34500us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.102449 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.102524 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.208964 (+106440us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b1800300280. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:59.785309 (+576345us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b1800300280. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1
0709 13:02:59.786883 (+  1574us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b1800300280. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:02:59.828950 (+ 42067us) raft_consensus.cc:1685] finished
0709 13:02:59.828971 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.829079 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34322,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":1616,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":1538,"spinlock_wait_cycles":123822592,"thread_start_us":144858,"threads_started":1,"wal-append.queue_time_us":183078}
I20260709 13:02:59.832080  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.832805  1555 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:59.834090  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.835000  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.835306  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:02:59.835623  1560 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:59.836638  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.836823  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.837426  1560 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:59.837524  1558 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
W20260709 13:02:59.837745  7024 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:02:59.706666  6947 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.840157  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2653) took 1276 ms. Trace:
I20260709 13:02:59.840545  1552 rpcz_store.cc:276] 0709 13:02:58.564145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.920163 (+356018us) service_pool.cc:224] Handling call
0709 13:02:58.920570 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.739240 (+818670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.839925 (+100685us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54003fb454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:02:59.840102 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878821632}
I20260709 13:02:59.841568  6734 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:02:59.842271  6734 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.844197  6350 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.844833  6350 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.846640   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:59.847211  6350 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.846796  1457 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:59.848943  7016 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.340s	user 0.002s	sys 0.000s
I20260709 13:02:59.848733  1457 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 6 to 12, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 12 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:59.850611  2263 leader_election.cc:400] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12.
I20260709 13:02:59.851073  2263 leader_election.cc:403] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:59.851709  6933 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:59.853019  6830 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.853466  6830 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.854599  6830 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.854943  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:02:59.855783  7021 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:59.856345  7021 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.856534  1556 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:59.856654  6932 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:59.857131  6932 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.857298  7021 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.858018  6932 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.787981  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:02:59.859465  6932 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:59.888295  6912 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:59.888195  7002 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.888803  6912 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:02:59.889766  6912 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.890648  7002 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.891073  6912 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.875430  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:02:59.829777  6838 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:59.872877  7028 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
I20260709 13:02:59.859395  6877 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.893513  6877 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:02:59.896865  1557 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:02:59.862947  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.883723  6996 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.885640  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:02:59.899887  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 112085us
I20260709 13:02:59.915511  1559 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:02:59.900454  6983 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.901765  6734 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:02:59.908356  6838 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:59.909504  6617 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.912576  7022 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:59.913707  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.869733  1552 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:59.864794  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.861374  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:02:59.850704  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 10, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 10 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:02:59.930624  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.860785  6830 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.917277  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:02:59.920610  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 189594us
I20260709 13:02:59.930585  2295 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:59.935575  2297 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
I20260709 13:02:59.882689  1555 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:02:59.939436  1703 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:59.878194  1554 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:02:59.941414  6971 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:59.941524  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:02:59.941864  6877 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:02:59.942130  6971 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:02:59.942457  6939 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:59.942917  6939 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.942962  6763 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.943475  6939 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:59.943568  6763 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.944176  6763 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.951946  1551 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:02:59.920799  6996 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:02:59.847750  6350 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:02:59.921219  6983 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.934758  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:02:59.960527  1554 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:02:59.968650  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.969743  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.970502  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.986690  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:02:59.936280  6994 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:59.991531  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69840.6us
I20260709 13:02:59.959933  6918 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.962466  6933 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.996021  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.997972  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:02:59.965749  6983 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:00.002112  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:02:59.943269  6971 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.003780  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.006793  1705 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 10.
W20260709 13:03:00.008980  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.934031  6617 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.012647  1704 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:59.999651  2293 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:00.018256  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.001338  7002 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:00.023499  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 137986us
W20260709 13:03:00.041230  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.043088  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.051602  7032 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:03:00.056089  6996 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.057340  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.057678  6996 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.058188  6856 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:00.059798  6836 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.990558  6350 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.072088  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.072852  1707 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:00.073586  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 113520us
I20260709 13:03:00.074415  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.074450  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.076128  7022 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:00.076923  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.077566  1703 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:03:00.077908  6820 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.078374  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.078693  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.078424  6820 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.080108  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:00.080840  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.081332  6877 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.081597  7002 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.081856  6877 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.082372  6838 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.082842  6838 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.082652  7002 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.008908  6918 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.083921  6734 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:00.084059  6877 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.084354  6734 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.080766  6820 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.084786  6734 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.010412  7000 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:00.085731  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:00.076923  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.090193  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.083189  6838 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.102969  1707 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:00.088078  6902 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.108084  1634 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:00.108280  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:00.109898  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.110395  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.110798  6902 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.111080  1705 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:00.112632  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.120083  2294 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
W20260709 13:03:00.115993  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71558.2us
I20260709 13:03:00.085732  6918 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.131197  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.132941  1708 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:03:00.137447  6734 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.140086  6734 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:00.140610  6617 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.131356  6820 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.142047  1630 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:03:00.128571  6688 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:00.143613  6688 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.144047  6688 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.131058  6974 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:03:00.144989  6748 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:00.146274  6974 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:00.146644  6974 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:00.148516  6873 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:00.149070  6873 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:00.150143  6873 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.151497  7022 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.152081  7022 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.157626  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.158480  1708 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.160422  6933 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12.
W20260709 13:03:00.161362  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.162905  6350 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.160418  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.167208  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.168143  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:00.179453  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.184309  1632 raft_consensus.cc:2368] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:00.188465  7022 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:00.147496  6974 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.190579  6974 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.190950  2288 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:00.192113  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.122390  7035 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:00.198937  2290 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
W20260709 13:03:00.202323  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 127986us
I20260709 13:03:00.205279  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:00.206243  2292 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:00.224792  1708 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:00.233168  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:00.234860  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.235985  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 178749us
I20260709 13:03:00.238945  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.240587  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.242239  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.117228  6939 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:00.250072  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 293440us
I20260709 13:03:00.257953  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.252681  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.260179  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.244596  6934 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:00.268805  2297 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:00.270726  1632 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:00.271988  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.274262  1524 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:00.254716  6910 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.280792  1635 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:00.254349  7022 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:00.283612  1630 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:00.284020  1635 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 11: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:00.282950  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.282799  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:00.286454  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 128396us
I20260709 13:03:00.292498  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.292914  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:00.296069  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 161159us
I20260709 13:03:00.297518  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.304643  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.305742  1628 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:03:00.335616  1628 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 8
I20260709 13:03:00.306205  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.335976  1628 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:03:00.302136  7035 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.336232  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 70179.6us
I20260709 13:03:00.336585  7035 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.337283  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:00.338985  7000 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:00.339064  7035 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:00.339484  7000 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:00.336267  1628 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.339965  7000 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:00.314739  1557 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:03:00.341543  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.312857  1554 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:00.341964  7022 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.315245  6939 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.332129  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.334266  6873 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:00.344331  7022 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:00.345315  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.347436  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.331962  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.348147  1457 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 7 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:00.330497  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.349009  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.294976  6830 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.349891  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.350342  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.350072  1457 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 12 to 14, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.351982  6947 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:00.352181  1626 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:00.352595  6947 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:00.352751  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 48216.2us
I20260709 13:03:00.353494  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.353742  1675 leader_election.cc:400] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:03:00.354138  1675 leader_election.cc:403] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:00.353633  6947 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.355974  6350 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.356570  6350 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.357203  6350 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:00.347544  1454 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 290) took 38719 ms (client timeout 15000 ms). Trace:
W20260709 13:03:00.350381  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 94563.1us
I20260709 13:03:00.360186  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.360283  6953 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:03:00.360877  1633 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:00.361464  1633 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:00.358002  1454 rpcz_store.cc:269] 0709 13:02:21.627735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.634216 (+  6481us) service_pool.cc:224] Handling call
0709 13:02:21.880105 (+245889us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94e090 00007f70ce9ecf3a 00007f70d47a31cc 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc 00007f70d2d61392 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:00.347320 (+38467215us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:50.616528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:50.702238 (+ 85710us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:50.702289 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:02:50.702320 (+    31us) tablet.cc:662] Decoding operations
0709 13:02:50.703184 (+   864us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:50.703290 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:50.703328 (+    38us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:02:50.704021 (+   693us) tablet.cc:701] Row locks acquired
0709 13:02:50.704068 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:02:50.704238 (+   170us) write_op.cc:270] Start()
0709 13:02:50.704384 (+   146us) write_op.cc:276] Timestamp: P: 1783602170704172 usec, L: 0
0709 13:02:50.704415 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:02:50.705145 (+   730us) log.cc:844] Serialized 2477 byte log entry
0709 13:02:51.247179 (+542034us) op_driver.cc:464] REPLICATION: finished
0709 13:02:51.688132 (+440953us) write_op.cc:301] APPLY: starting
0709 13:02:51.688277 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:51.790570 (+102293us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:51.790698 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:52.006460 (+215762us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:52.006764 (+   304us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:02:52.006810 (+    46us) write_op.cc:312] APPLY: finished
0709 13:02:52.007178 (+   368us) log.cc:844] Serialized 65 byte log entry
0709 13:02:52.029200 (+ 22022us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:52.033770 (+  4570us) write_op.cc:454] Released schema lock
0709 13:02:52.044672 (+ 10902us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:52.054632 (+  9960us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:52.054669 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66725,"reported_tablets":48,"spinlock_wait_cycles":100549504,"child_traces":[["op",{"apply.queue_time_us":340488,"apply.run_cpu_time_us":13788,"apply.run_wall_time_us":366448,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29231,"num_ops":6,"prepare.queue_time_us":603689,"prepare.run_cpu_time_us":94021,"prepare.run_wall_time_us":282806,"raft.queue_time_us":165528,"raft.run_cpu_time_us":11258,"raft.run_wall_time_us":345652,"replication_time_us":479735,"spinlock_wait_cycles":5376,"thread_start_us":830336,"threads_started":4,"wal-append.queue_time_us":194482,"wal-append.run_cpu_time_us":16063,"wal-append.run_wall_time_us":2113147}]]}
I20260709 13:03:00.361750  6953 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:00.361951  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.363469  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.359808  1630 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:00.351449  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.353919  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.349097  1700 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.365360  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.356021  6971 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:00.371199  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.372709  6971 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.373359  6971 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:00.374354  6971 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.368906  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:00.362541  1633 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.6, Last appended by leader: 6, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.376965  1633 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:00.377279  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.370365  1634 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:00.327111  6983 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.375962  6836 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.385082  1628 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:00.387009  1457 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.391312  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.370769  7035 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.396025  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.398505  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.399382  1632 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:00.399693  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.400406  1631 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:00.413942  1708 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:00.414081  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:00.415009  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:00.386159  6868 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:00.416667  1701 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:00.386662  6902 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:00.416605  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 115126us
I20260709 13:03:00.429342  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.430740  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:00.431800  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67130.9us
I20260709 13:03:00.298296  6648 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:03:00.377029  6688 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.440944  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.441552  6688 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.403410  6617 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:00.442023  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.442521  6956 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:00.445801  7032 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:03:00.413309  6748 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:00.446805  6763 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.002s
I20260709 13:03:00.448504  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.420658  6820 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.453022  6902 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:00.449002  7032 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.454337  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.485080  1703 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.454237  1457 catalog_manager.cc:5697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 2 to 9, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.432029  6868 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.421173  6877 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:00.480489  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43512.3us
W20260709 13:03:00.482030  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47463.6us
I20260709 13:03:00.467718  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
W20260709 13:03:00.489522  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89882.9us
I20260709 13:03:00.490201  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.488382  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:00.491205  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64269us
I20260709 13:03:00.397260  6838 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.432454  6910 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:00.498167  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:00.500346  2262 leader_election.cc:400] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:00.500697  2262 leader_election.cc:403] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:00.500845  1633 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:00.494135  7002 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.502365  6873 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.478204  6748 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:00.504473  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.483597  6956 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:00.506304  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.509850  1708 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:00.525383  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.528470  1525 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.530541  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.526073  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.534905  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.524439  6873 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:00.537550  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47426.2us
I20260709 13:03:00.487623  6350 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.506035  6748 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.505797  7032 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.509524  6933 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:00.484041  6763 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.553575  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.562016  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.561758  1457 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 15, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 15 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:00.559880  1703 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:00.564961  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:00.536877  6489 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.539063  7002 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:00.503611  6910 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.522574  7035 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:03:00.575177  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 81292.9us
I20260709 13:03:00.546274  6878 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.551617  6962 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:00.550488  6947 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.596714  6947 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.541545  6693 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:00.547636  6877 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.582949  7035 consensus_queue.cc:1243] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:00.560844  6902 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.585911  6972 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:00.586632  6617 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:00.587425  7048 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:03:00.587988  6910 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:00.588574  6878 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:00.577692  6648 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:03:00.589833  1457 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 10, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.591233  6873 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.593330  6956 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.591230  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:00.601212  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 103152us
I20260709 13:03:00.569537  6918 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.565793  6734 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:03:00.636360  2263 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.750s (SENT)
W20260709 13:03:00.647203  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63023us
I20260709 13:03:00.650482  7032 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:00.651015  7032 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:00.652025  1457 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 11 to 12, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:00.654214  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 63058.1us
I20260709 13:03:00.605414  6933 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:00.633994  6617 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.656035  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:00.663642  2288 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:00.659324  2297 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:00.658489  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93585.1us
I20260709 13:03:00.661262  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.667794  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:00.669004  2290 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:00.671609  2292 raft_consensus.cc:2368] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:00.581084  6868 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:00.685446  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.690234  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 120873us
I20260709 13:03:00.700670  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.666308  7066 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:00.703956  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 79759.9us
I20260709 13:03:00.704890  6350 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.706555  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.692735  6489 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:00.715869  6878 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.720961  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:00.698210  6838 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:00.726105  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64914.1us
W20260709 13:03:00.726570  7059 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:03:00.727092  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.708732  7070 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:00.707823  1457 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 12 to 13, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 13 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:03:00.720081  7021 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:00.737982  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72209.2us
I20260709 13:03:00.739897  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:00.711064  6933 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:03:00.730804  7067 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:03:00.744652  6996 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:00.708213  7032 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.736358  6350 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:00.741834  1457 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:00.758072  6868 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:00.760560  7032 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.764873  6350 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:00.766063  6350 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:00.766467  6350 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.765866  1457 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:00.765193  6763 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:00.746223  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:00.741513  6617 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.774891  7077 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:00.779057  6877 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.749670  6910 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:00.783957  7076 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:00.782814  7077 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:03:00.788842  1457 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 11, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 11 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:03:00.783342  6910 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:00.796293  6910 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.798640  7000 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.799914  1626 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:00.800807  1457 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 12, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:00.805275  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.810314  7076 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:00.814390  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 252417us
I20260709 13:03:00.816493  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.794358  6868 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.781901  6996 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:00.821446  6648 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:00.822302  6996 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.822048  6648 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:00.824903  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 65395us
I20260709 13:03:00.820392  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:00.822822  6996 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:00.830538  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84403.3us
I20260709 13:03:00.837193  1457 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.821020  6617 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:00.832969  6877 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.861011  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:00.849007  7038 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.581s	user 0.002s	sys 0.005s
I20260709 13:03:00.845546  6956 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.864673  1558 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:03:00.863410  1557 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:03:00.866744  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.867178  1554 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:00.870581  6648 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:00.872052  1555 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:00.873245  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:00.874114  7068 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:00.833063  6350 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.874588  7068 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:00.850602  6748 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.874142  1552 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:00.871953  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.877877  6748 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:00.845356  6915 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:00.878222  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:00.878726  6915 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:00.879077  7069 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:00.879513  7069 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.879581  6915 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.880359  7069 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.881592  7069 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:00.883013  7084 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:03:00.883786  6956 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.884541  1708 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:00.885524  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:00.866778  6877 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:03:00.887315  6489 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:00.887615  1706 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:00.887880  6489 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:00.888989  1553 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:00.888783  6489 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.890283  6956 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:00.894634  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:00.867835   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:00.895922  6763 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:00.896512  6763 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:00.897398  6763 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.875582  7068 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.903340  1704 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:00.865729  1457 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 5 to 6, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.825577  6868 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:00.915618  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:00.934567  1457 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 9 to 11, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:03:00.937681  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.032361  1557 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:00.866088  7076 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:03:00.930754  7068 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.038514  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:01.040227  1457 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 12 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:01.039753  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 208297us
I20260709 13:03:01.035112  7067 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:00.883038  6910 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:00.935114  7032 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:00.935905  6996 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.034838  6912 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.031533  6617 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.031803  6956 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:01.054351  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:01.055917  1457 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:01.043015  1599 leader_election.cc:400] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:01.040534  7000 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.064427  1557 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:01.067301  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:01.062208  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.063197  1553 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:00.938340  7070 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:03:01.072175  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.073343  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.035616  6648 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:03:01.036199  7077 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:03:01.030900  7080 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.116s
I20260709 13:03:01.051077  7032 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:01.079895  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.059335  6996 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.091589  2288 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:01.091856  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.037310  7089 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:03:01.109500  1553 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:01.045578  7067 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.083374  7070 consensus_queue.cc:1243] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:01.060425  6617 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.129438  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2114) took 1141 ms. Trace:
I20260709 13:03:01.091341  7032 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.066282  6956 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:01.129220  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.148168  1557 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:01.070132  6912 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:01.135037  7087 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:03:01.149396  7090 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:01.149868  7070 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:01.171221  7070 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:01.172694  7090 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:01.141213  1701 rpcz_store.cc:276] 0709 13:02:59.988406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.993075 (+  4669us) service_pool.cc:224] Handling call
0709 13:03:00.012147 (+ 19072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.417182 (+405035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.128307 (+711125us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b5400166a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:01.129370 (+  1063us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929697024}
I20260709 13:03:01.138187  6971 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.174381  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:01.180275  1551 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:01.111591  6996 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.211689  7071 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:01.217726  7062 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.600s	user 0.000s	sys 0.003s
I20260709 13:03:01.217661  7071 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:01.218973  6106 raft_consensus.cc:993] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:03:01.219578  6106 raft_consensus.cc:1081] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:03:01.220033  7071 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:01.224946  7091 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:01.228367  6617 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.233564  6912 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.236035  7067 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.238533  6956 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:01.268064  7080 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.278154  6489 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:01.248647  7094 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:03:01.278437  7080 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:01.267223  7064 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.651s	user 0.002s	sys 0.000s
W20260709 13:03:01.280203  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.187325  6971 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.282298  6878 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.282943  6878 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.283457  6971 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.212162  6763 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:01.284492  6763 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.285609  1632 raft_consensus.cc:1217] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[13.11-17.13]   Dedup: 13.11->[15.12-17.13]
I20260709 13:03:01.286149  1632 pending_rounds.cc:85] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 11
I20260709 13:03:01.286360  1632 pending_rounds.cc:107] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 14.12
W20260709 13:03:01.286582  7088 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.426s	user 0.002s	sys 0.000s
I20260709 13:03:01.287456  6763 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.286623  1632 raft_consensus.cc:2889] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.275678  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.300153  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.300673  6962 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:01.301229  6962 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:01.301836  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:01.302318  1635 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:03:01.302238  6962 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.303623  7073 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:01.304198  7073 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:01.305070  7073 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:01.307108  6953 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.307870  6953 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:01.308102  6934 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:01.307114  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.309614  6934 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:01.272676  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:01.310035  6934 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:01.311036  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.310906  6934 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.312325  6934 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:01.313280  6693 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:01.288727  6996 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:01.317234  7055 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.751s	user 0.000s	sys 0.002s
I20260709 13:03:01.318060  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.317987  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.318810  6972 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:01.318961  1635 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:03:01.319283  6972 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:01.319542  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:01.320720  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.320256  6972 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.273229  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:01.321547  6972 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:01.321432  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:01.322242  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49104.4us
I20260709 13:03:01.322402  1627 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 11.
I20260709 13:03:01.324440  1627 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:01.324923  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.291002  7080 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.324968  6996 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:01.325730  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:01.326650  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.295500  6912 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.327172  1552 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:01.295958  7067 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.321349  6971 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:01.327816  7067 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.328264  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.328634  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.329239  6971 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:01.330173  7067 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.330741  7067 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:01.331117  7067 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.231170  7090 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:01.333317  7090 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:01.296784  6350 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.297528  6915 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:01.298677  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.278975  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:01.338040  1554 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.339175  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:01.290637  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.289902  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.346231  1707 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:03:01.347993  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.349586  1708 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:03:01.351898  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:01.352491  2289 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:01.352612  2294 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:01.356410  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.356616  2293 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:01.357095  2296 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:01.358265  6983 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:01.359203  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.359795  6932 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:01.360313  6932 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.360981  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.362159  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.362393  6932 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.327415  6912 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.364907  1709 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:01.365770  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.366290  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.366411  2293 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:01.366664  7070 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:01.367965  6971 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:01.368336  6912 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.369626  7070 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.369959  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:01.370272  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.370556  6939 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.370937  1705 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:01.337575  1556 pending_rounds.cc:85] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 5
I20260709 13:03:01.371529  6971 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:01.371836  1556 pending_rounds.cc:107] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:01.371985  6971 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.289152  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.372550  6971 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.373121  1559 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:01.371053  6939 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.373683  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:01.374253  2293 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.375432  6939 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:01.372433  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.375918  6912 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.371261  6983 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:03:01.331460  7067 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:01.327335  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:01.372368  1556 raft_consensus.cc:2889] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:01.377607  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50354.9us
I20260709 13:03:01.376816  6912 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.378873  6939 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:01.379261  6939 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:01.376588  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.380702  6939 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:01.382747  1703 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:01.395004  6878 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.395790  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.397922  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.325127  6763 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:01.256146  6868 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.407312  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.402287  1555 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:01.408120  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.408702  1633 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:01.463613  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.465085  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.465869  1631 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:01.408959  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:01.467561  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58697.5us
I20260709 13:03:01.467669  7022 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:01.468204  7022 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:01.468376  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.468611  7022 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:01.409140  6763 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.469691  6763 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.470263  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.471334  2297 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:01.474318  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.411898  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.480185  1700 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:01.416035  7071 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:01.417663  6947 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:03:01.420202  6911 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:03:01.409566  6962 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.461777  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.486193  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.380049  6912 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:01.479478  6763 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.426440  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:01.428381  2288 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:01.435609  1702 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:01.453142  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.455937  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.523986  1634 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:01.456048  1552 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:01.487876  1555 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:01.422698  1554 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
W20260709 13:03:01.487706  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.488226  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:01.488634  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.530011  1631 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:01.399708  1559 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:01.495126  1706 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:01.503813  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:01.512717  1700 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:01.488128  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:01.430537  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:01.433424  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:01.548085  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 114736us
W20260709 13:03:01.508014  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51869.8us
I20260709 13:03:01.557217  2289 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:01.600003  1633 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:01.600492  1557 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:01.511092  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.602063  2293 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:01.604024  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:01.611119  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:01.611974  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:01.612923  1555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:03:01.613554  1555 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:01.613951  1555 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:01.620739  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70287.3us
I20260709 13:03:01.624671  2288 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:01.626477  2288 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:01.626354  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:01.633894  7032 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.635319  1629 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.454078  6868 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:01.638154  6868 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.640758  6868 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:01.645653  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 141954us
I20260709 13:03:01.649055  1634 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:01.641960  1631 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:03:01.642763  1629 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
W20260709 13:03:01.645610  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 134630us
W20260709 13:03:01.646831  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 85416.2us
I20260709 13:03:01.490411  6912 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:01.670804  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 52546.7us
I20260709 13:03:01.653609  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:01.504879  6910 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:01.509758  7076 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:01.733387  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.752115  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.755390  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 83635.7us
W20260709 13:03:01.759506  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106007us
I20260709 13:03:01.677706  7098 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:01.680651  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:01.667598  7035 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:01.682806  2293 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:03:01.687518  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:01.688346  7089 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:01.432060  7070 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:01.504034  6933 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:01.755537  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 129293us
I20260709 13:03:01.673763  1634 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:01.507889  6878 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.207s
I20260709 13:03:01.419685  7002 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.296s
I20260709 13:03:01.430028  7067 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.433768  6971 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:01.720521  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61479.4us
I20260709 13:03:01.428088  6932 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.287s
W20260709 13:03:01.781255  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93875.6us
I20260709 13:03:01.854628  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:01.779687  2290 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:03:01.808930  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48365.5us
I20260709 13:03:01.857576  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:01.604146  6983 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:01.864665  1709 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:01.715786  6912 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:01.655488  6877 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:01.659406  6763 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.744956  7076 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:01.780476  6350 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:01.829294  7090 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:01.780649  6106 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:01.825083  6836 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.888881  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 445186us
I20260709 13:03:01.865583  6983 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.872668  6912 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:01.901690  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44217.5us
I20260709 13:03:01.903465  1701 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:01.830775  6748 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:01.841365  6915 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:03:01.908967  2292 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:01.909319  1551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.028s	sys 0.044s
W20260709 13:03:01.944545  1551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.028s	sys 0.044s
I20260709 13:03:01.772689  6693 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 13:03:01.615887  6939 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:01.917384  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.013s	sys 0.038s
W20260709 13:03:01.959966  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.013s	sys 0.038s
I20260709 13:03:01.932404  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:01.908180  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52322.6us
W20260709 13:03:01.956571  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.018s	sys 0.017s
I20260709 13:03:01.859884  6911 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:03:01.904695  6350 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.961210  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2666) took 1005 ms. Trace:
W20260709 13:03:01.962277  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.018s	sys 0.017s
W20260709 13:03:01.975926  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.026s	sys 0.033s
I20260709 13:03:01.967203  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:01.977218  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.026s	sys 0.033s
W20260709 13:03:01.967854  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.007s	sys 0.077s
W20260709 13:03:01.933224  7073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.009s	sys 0.040s
W20260709 13:03:01.977738  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.007s	sys 0.077s
W20260709 13:03:01.980579  1705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.045s
W20260709 13:03:01.980928  1705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.014s	sys 0.045s
I20260709 13:03:01.926255  6953 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.914786  7035 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.865758  7070 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.844408  6933 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.965210  1633 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:01.975495  1551 rpcz_store.cc:276] 0709 13:03:00.955465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.182789 (+227324us) service_pool.cc:224] Handling call
0709 13:03:01.212941 (+ 30152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.213150 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:01.256526 (+ 43376us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.960738 (+704212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.960813 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:01.960875 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.960965 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.961135 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15537,"prepare.queue_time_us":43177,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":21593,"thread_start_us":688150,"threads_started":1}
I20260709 13:03:01.940263  6983 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:01.987556  1627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.027s	sys 0.018s
W20260709 13:03:02.007094  1627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.027s	sys 0.018s
W20260709 13:03:02.002573  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.032s	sys 0.047s
I20260709 13:03:02.000613  7100 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.005719  2291 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:01.980490  7073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.009s	sys 0.040s
W20260709 13:03:02.008723  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.033s	sys 0.047s
W20260709 13:03:02.019959  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 551603us
I20260709 13:03:01.902822  6763 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:01.917708  6947 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:03:02.028652  1703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.011s	sys 0.036s
I20260709 13:03:01.980832  6350 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:02.025234  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.972764  7089 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:02.029130  1703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.012s	sys 0.036s
I20260709 13:03:01.983108  7032 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:01.988293  7069 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:01.934957  6836 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.046443  1556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.036s	sys 0.056s
W20260709 13:03:02.047715  1556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.036s	sys 0.056s
I20260709 13:03:01.988857  6868 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:03:02.032330  6972 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.003s
W20260709 13:03:01.986343  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54083us
I20260709 13:03:01.942449  6106 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.001379  6956 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.993700  6688 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:02.054636  1457 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.008s	sys 0.061s
W20260709 13:03:02.055109  1457 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.008s	sys 0.061s
W20260709 13:03:02.055680  2294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.001s	sys 0.048s
I20260709 13:03:02.002795  6953 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.058066  1634 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:03:02.092818  1634 pending_rounds.cc:85] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 7
I20260709 13:03:02.027662  7100 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.043380  7069 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.021963  6877 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:03:02.067714  6106 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:02.078034  1632 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.000s	sys 0.064s
W20260709 13:03:02.146782  1632 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.000s	sys 0.065s
W20260709 13:03:02.078800  2294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.001s	sys 0.048s
I20260709 13:03:01.990695  7076 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.093130  1634 pending_rounds.cc:107] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:02.050328  7090 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:02.060297  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:02.145485  1555 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:02.079864  7103 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.570s	user 0.000s	sys 0.001s
I20260709 13:03:02.156384  1552 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:02.151237  1634 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.157125  1634 pending_rounds.cc:107] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:03:02.157461  1634 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.158290  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:02.151566  1633 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:02.154347  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.033s	sys 0.086s
W20260709 13:03:02.165694  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.033s	sys 0.087s
I20260709 13:03:02.169529  1551 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:02.149513  7000 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.029s	sys 0.028s
I20260709 13:03:02.152506  7069 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.059736  6956 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.198760  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2664) took 1243 ms. Trace:
I20260709 13:03:02.201262  1553 rpcz_store.cc:276] 0709 13:03:00.954916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.113704 (+158788us) service_pool.cc:224] Handling call
0709 13:03:01.114141 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.163869 (+ 49728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.163920 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.164545 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.509759 (+345214us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:01.510120 (+   361us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.610104 (+ 99984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.610184 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.198508 (+588324us) raft_consensus.cc:1685] finished
0709 13:03:02.198530 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.198685 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179993,"prepare.queue_time_us":384,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":90,"raft.queue_time_us":115331,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":44852,"thread_start_us":295418,"threads_started":1,"wal-append.queue_time_us":324498}
W20260709 13:03:02.203179  2295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.021s	sys 0.049s
W20260709 13:03:02.211078  2295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.021s	sys 0.049s
W20260709 13:03:02.182022  7000 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.029s	sys 0.028s
I20260709 13:03:02.179019  6971 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.193961  7067 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.169225  7090 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.277823  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2072) took 1064 ms. Trace:
I20260709 13:03:02.278453  1560 rpcz_store.cc:276] 0709 13:03:01.213092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.415674 (+202582us) service_pool.cc:224] Handling call
0709 13:03:01.417565 (+  1891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.418794 (+  1229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.418839 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.419472 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.976553 (+557081us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:01.976593 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.976664 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.277576 (+300912us) raft_consensus.cc:1685] finished
0709 13:03:02.277595 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.277744 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":556611,"threads_started":1,"wal-append.queue_time_us":785896}
I20260709 13:03:02.298563  6912 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.244755  6939 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:02.304522  6836 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.315301  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:02.316272  1626 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:02.316627  1626 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:02.317421  1626 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:02.319077  1626 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.333346  1555 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.334005  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51916 (request call id 2073) took 1112 ms. Trace:
I20260709 13:03:02.351110  7076 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.337994  6688 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.369025  6688 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.338771  6648 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:02.371953  6648 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:02.373054  6648 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.374596  6648 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.339454  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2075) took 1066 ms. Trace:
I20260709 13:03:02.375928  1552 rpcz_store.cc:276] 0709 13:03:01.273191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.683738 (+410547us) service_pool.cc:224] Handling call
0709 13:03:01.726189 (+ 42451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.171917 (+445728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.339193 (+167276us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000e8154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:02.339377 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979674368}
I20260709 13:03:02.340026  6910 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:02.340961  1626 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:02.377259  6910 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:02.342149  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2667) took 1386 ms. Trace:
I20260709 13:03:02.378137  1556 rpcz_store.cc:276] 0709 13:03:00.955745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.337026 (+381281us) service_pool.cc:224] Handling call
0709 13:03:01.337376 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.378201 (+ 40825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.378232 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.378655 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.048187 (+669532us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:02.065855 (+ 17668us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.099429 (+ 33574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.099510 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.337763 (+238253us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b18003dc060. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:02.341831 (+  4068us) raft_consensus.cc:1685] finished
0709 13:03:02.341961 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.342088 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33268,"prepare.queue_time_us":19915,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":20092,"raft.queue_time_us":60992,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":11331,"spinlock_wait_cycles":33094912,"thread_start_us":669136,"threads_started":1,"wal-append.queue_time_us":961050}
I20260709 13:03:02.343647  6953 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.381652  6953 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:02.382336  6953 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.319737  6933 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:02.346370  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2693) took 1020 ms. Trace:
I20260709 13:03:02.383850  7067 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:02.384438  1627 rpcz_store.cc:276] 0709 13:03:01.325743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.325855 (+   112us) service_pool.cc:224] Handling call
0709 13:03:01.326191 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.326530 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.326559 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.327031 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.007742 (+680711us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:02.007842 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.007944 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.346090 (+338146us) raft_consensus.cc:1685] finished
0709 13:03:02.346120 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.346284 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":457,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":680281,"threads_started":1,"wal-append.queue_time_us":895325}
I20260709 13:03:02.394196  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2668) took 1356 ms. Trace:
I20260709 13:03:02.393990  6910 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.395980  6910 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:02.394764  1558 rpcz_store.cc:276] 0709 13:03:01.037546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.354224 (+316678us) service_pool.cc:224] Handling call
0709 13:03:01.354539 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.354865 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.354892 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.355332 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.980962 (+625630us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.005466 (+ 24504us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.010860 (+  5394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.010977 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.393922 (+382945us) raft_consensus.cc:1685] finished
0709 13:03:02.393949 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.394117 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4895,"prepare.queue_time_us":24554,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":26719,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":7145,"thread_start_us":625147,"threads_started":1,"wal-append.queue_time_us":885838}
I20260709 13:03:02.397805  6868 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.398499  6868 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.400764  7002 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:02.401186  6868 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:02.358404  7104 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:03:02.336701  6911 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:02.401564  7002 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.351601  1555 rpcz_store.cc:276] 0709 13:03:01.221516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.612781 (+391265us) service_pool.cc:224] Handling call
0709 13:03:02.333200 (+720419us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b4c00129d40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cccb0a20 00007f70cccc1a29 00007f70cccc213f 00007f70ccc862ab 00007f70cccb3f3c 00007f70cccb43fe 00007f70ccf4188b 00007f70ce940495 00007f70ce99130f 00007f70ce9dba55 00007f70d39abcd0
0709 13:03:02.333901 (+   701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6622464}
I20260709 13:03:02.241074  7073 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.363612  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2685) took 1076 ms. Trace:
I20260709 13:03:02.366950  6956 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.404150  7073 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.367844  7070 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.404088  1628 rpcz_store.cc:276] 0709 13:03:01.287231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.291056 (+  3825us) service_pool.cc:224] Handling call
0709 13:03:01.291401 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.291841 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.291881 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.292444 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.960496 (+668052us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:01.960545 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.960616 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.363338 (+402722us) raft_consensus.cc:1685] finished
0709 13:03:02.363361 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.363522 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5110,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":202,"thread_start_us":667611,"threads_started":1,"wal-append.queue_time_us":923616}
I20260709 13:03:02.406554  7073 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.406952  7002 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:02.404701  6953 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:02.419085  6962 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.935s	user 0.009s	sys 0.001s
I20260709 13:03:02.311172  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:02.481681  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 170610us
W20260709 13:03:02.429394  7022 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.000s
I20260709 13:03:02.326941  7100 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.430683  7070 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.445188  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.491626  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.449401  6688 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.453836  6910 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.456765  6912 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.460321  7002 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.488216  6693 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:02.483498  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:02.502143  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:02.504076  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.004s	sys 0.007s
W20260709 13:03:02.504556  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.004s	sys 0.007s
W20260709 13:03:02.509649  2289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.031s	sys 0.044s
W20260709 13:03:02.513566  2289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.031s	sys 0.044s
W20260709 13:03:02.511233  1708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.008s	sys 0.003s
W20260709 13:03:02.517140  1708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.003s
W20260709 13:03:02.511924  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.004s	sys 0.006s
W20260709 13:03:02.520522  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.005s	sys 0.006s
I20260709 13:03:02.480211  6971 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.502918  6947 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:02.512349  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.008s	sys 0.002s
I20260709 13:03:02.513576  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:02.527556  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.009s	sys 0.002s
W20260709 13:03:02.528641  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 771954us
I20260709 13:03:02.501256  6693 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:02.511616  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.009s	sys 0.002s
W20260709 13:03:02.663471  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.009s	sys 0.002s
I20260709 13:03:02.526927  1628 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
W20260709 13:03:02.536307  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.007s	sys 0.003s
I20260709 13:03:02.494796  7070 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.501056  6910 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.512537  6763 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:02.502640  7002 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.535343  6983 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
W20260709 13:03:02.552299  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.039s	sys 0.032s
W20260709 13:03:02.681341  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.039s	sys 0.032s
I20260709 13:03:02.682431  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2752) took 1207 ms. Trace:
I20260709 13:03:02.682896  1705 rpcz_store.cc:276] 0709 13:03:01.475639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.476206 (+   567us) service_pool.cc:224] Handling call
0709 13:03:01.476591 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.476833 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:01.981355 (+504522us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.681930 (+700575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.682011 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.682078 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.682170 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.682344 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39850,"prepare.queue_time_us":764900,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"thread_start_us":1204291,"threads_started":2}
I20260709 13:03:02.688316  7070 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.688897  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1247) took 1261 ms. Trace:
I20260709 13:03:02.689306  6962 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.689440  2295 rpcz_store.cc:276] 0709 13:03:01.427674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.427860 (+   186us) service_pool.cc:224] Handling call
0709 13:03:01.428208 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.428611 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.428637 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.429090 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.211613 (+782523us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:02.211713 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.211800 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.688572 (+476772us) raft_consensus.cc:1685] finished
0709 13:03:02.688603 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.688786 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":782117,"threads_started":1,"wal-append.queue_time_us":1218981}
I20260709 13:03:02.690721  6910 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.672178  6693 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:02.696437  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:02.699409  7002 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:02.701442  7129 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.702243  7129 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.639338  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.010s	sys 0.002s
W20260709 13:03:02.560717  7101 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.021s	sys 0.064s
I20260709 13:03:02.650385  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:02.653501  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.533335  6971 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.490867  7022 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.667019  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.008s	sys 0.004s
I20260709 13:03:02.690196  6962 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.640048  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.007s
W20260709 13:03:02.706892  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.011s	sys 0.002s
W20260709 13:03:02.732595  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.000s	sys 0.008s
W20260709 13:03:02.733212  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 878625us
I20260709 13:03:02.737082  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:02.774333  2261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.043s	sys 0.060s
W20260709 13:03:02.774588  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.776716  7129 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:02.778216  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.774823  2261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.043s	sys 0.060s
I20260709 13:03:02.689193  6912 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.773548  7101 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.030s	sys 0.064s
W20260709 13:03:02.785406  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48416.1us
I20260709 13:03:02.789624  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:02.793879  2296 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:02.682811  7090 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:02.686489  6763 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:02.714643  6868 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.792448  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.787137  2295 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:02.796602  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:02.797585  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 147294us
I20260709 13:03:02.773720  6910 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.784133  6934 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:02.784819  6953 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.811251  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:02.785087  7073 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:02.814802  2288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.041s	sys 0.096s
I20260709 13:03:02.814147  2292 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:02.791863  7097 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.721536  6983 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.819537  2288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.042s	sys 0.096s
I20260709 13:03:02.808244  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:02.795745  6836 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.822772  2291 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:02.803397  7100 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.799989  6868 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:02.828856  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:02.847004  1702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.090s	user 0.002s	sys 0.019s
W20260709 13:03:02.847401  1702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.002s	sys 0.019s
W20260709 13:03:02.847934  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.014s	sys 0.008s
I20260709 13:03:02.845774  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:02.818884  6933 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:02.849128  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.014s	sys 0.008s
W20260709 13:03:02.845928  2294 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.006s	sys 0.013s
W20260709 13:03:02.852728  2294 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.006s	sys 0.014s
I20260709 13:03:02.853994  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1245) took 1429 ms. Trace:
W20260709 13:03:02.846767  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.003s	sys 0.017s
I20260709 13:03:02.847693  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:02.857403  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:02.793040  6912 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.858433  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.004s	sys 0.017s
W20260709 13:03:02.846541  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.011s	sys 0.010s
W20260709 13:03:02.859213  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.011s	sys 0.010s
W20260709 13:03:02.859687  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 831373us
W20260709 13:03:02.846279  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.009s	sys 0.010s
W20260709 13:03:02.863526  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.009s	sys 0.010s
I20260709 13:03:02.865468  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:02.867134  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:02.869283  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:02.870244  1629 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:02.872620  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:02.873471  1629 raft_consensus.cc:2410] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:02.822341  7097 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.848569  2291 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:03:02.881496  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.010s	sys 0.010s
W20260709 13:03:02.881857  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.010s	sys 0.010s
I20260709 13:03:02.803223  6763 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.880123  6933 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:02.846153  6748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.004s	sys 0.016s
I20260709 13:03:02.835553  7123 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:02.856428  2294 rpcz_store.cc:276] 0709 13:03:01.424319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.424617 (+   298us) service_pool.cc:224] Handling call
0709 13:03:01.424951 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.425296 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.425322 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.425810 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.147733 (+721923us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.853259 (+705526us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.853572 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.853671 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.853726 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:02.853745 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.853916 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1397,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":1402896,"threads_started":2,"wal-append.queue_time_us":911801}
I20260709 13:03:02.812404  6910 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:02.900419  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.009s	sys 0.013s
I20260709 13:03:02.816617  6934 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:02.909329  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.009s	sys 0.013s
W20260709 13:03:02.903884  1635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.011s	sys 0.025s
I20260709 13:03:02.836194  7089 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:02.903570  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.001s	sys 0.034s
W20260709 13:03:02.914702  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.002s	sys 0.034s
I20260709 13:03:02.817672  6953 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:02.906730  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.011s	sys 0.025s
W20260709 13:03:02.915406  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.011s	sys 0.026s
W20260709 13:03:02.910905  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.011s	sys 0.025s
W20260709 13:03:02.843729  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.011s	sys 0.010s
W20260709 13:03:02.917138  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.012s	sys 0.010s
W20260709 13:03:02.917578  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 931585us
W20260709 13:03:02.888254  7000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.015s	sys 0.020s
I20260709 13:03:02.886719  7097 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:02.904186  6748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.005s	sys 0.016s
I20260709 13:03:02.926222  2294 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:02.919525  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:02.975849  6748 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:02.976924  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57484.6us
I20260709 13:03:02.832892  7073 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:02.908478  7069 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.004s
I20260709 13:03:02.904039  6933 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.835297  6836 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.983604  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:02.843746  7100 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:02.921247  7000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.015s	sys 0.020s
I20260709 13:03:02.921677  6911 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.978909  7115 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:03:02.912359  7070 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:02.971450  7126 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.993893  7131 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:03.011312  7131 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:03.012933  7131 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:03.014436  7131 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:03:03.016330  6648 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.641s	user 0.005s	sys 0.010s
I20260709 13:03:03.016815  6763 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.016912  6911 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.018647  6983 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.020002  7070 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.022055  7022 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.023013  7100 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:03.005733  7132 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:03:03.004747  6934 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:02.987900  7129 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:03.008031  6962 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.010267  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1032) took 1509 ms. Trace:
I20260709 13:03:03.025112  7090 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.007402  6910 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.023077  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.054018  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.029340  6971 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.029769  7127 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.989979  7123 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:03.010319  7126 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.060231  7126 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.061867  7141 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:03.062094  7127 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.057292  6956 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.064392  6956 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.005741  6953 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.038451  2288 rpcz_store.cc:276] 0709 13:03:01.501046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.627368 (+126322us) service_pool.cc:224] Handling call
0709 13:03:01.627708 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.628393 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.628421 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.628898 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.824744 (+1195846us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.824804 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.824898 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.009908 (+185010us) raft_consensus.cc:1685] finished
0709 13:03:03.009957 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.010180 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8557,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":1195358,"threads_started":1,"wal-append.queue_time_us":1354896}
W20260709 13:03:03.030315  7076 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:03:02.971724  7089 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.066644  6956 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.068192  7089 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.056337  2295 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:03.070343  6911 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.072837  7073 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.072999  6911 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.038338  7129 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.073859  7129 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.073565  6911 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:03.073968  6350 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.026s	user 0.009s	sys 0.005s
I20260709 13:03:03.074635  6350 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:02.993743  7069 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.076814  1554 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:03.077795  7069 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.053315  7128 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.053413  7131 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:03:03.059665  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2755) took 1546 ms. Trace:
I20260709 13:03:03.059248  7002 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:03.081458  7002 consensus_queue.cc:1243] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:03.081212  1707 rpcz_store.cc:276] 0709 13:03:01.513233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.513388 (+   155us) service_pool.cc:224] Handling call
0709 13:03:01.615799 (+102411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.655910 (+ 40111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.655961 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.656593 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.521003 (+864410us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.521045 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.521112 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.059412 (+538300us) raft_consensus.cc:1685] finished
0709 13:03:03.059434 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.059608 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1951,"raft.queue_time_us":21488,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":659788,"threads_started":1,"wal-append.queue_time_us":1009903}
I20260709 13:03:03.017105  6868 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.082533  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.018774  6836 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.083253  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.083521  2290 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:03:03.083933  2296 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:03.064637  7127 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.080626  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.080335  7128 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.056013  7090 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.089601  7128 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.090984  7090 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.092873  6911 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.093281  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1033) took 1591 ms. Trace:
I20260709 13:03:03.061861  7123 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:03.004787  6912 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.096801  7123 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.096136  2289 rpcz_store.cc:276] 0709 13:03:01.501403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.677311 (+175908us) service_pool.cc:224] Handling call
0709 13:03:01.677718 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.678155 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.678197 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.678824 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.514860 (+836036us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:02.514923 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.515023 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.092970 (+577947us) raft_consensus.cc:1685] finished
0709 13:03:03.093003 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.093196 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":353681,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5786,"thread_start_us":835540,"threads_started":1,"wal-append.queue_time_us":1323854}
W20260709 13:03:03.051169  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 512878us
I20260709 13:03:03.102038  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:03.090353  7126 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.038113  6763 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.104169  6763 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.100361  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.111793  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:03.116261  1554 raft_consensus.cc:2368] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:03.131757  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.132815  1558 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:03.136739  1555 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:03.142563  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:02.928303  7120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.009s	sys 0.007s
W20260709 13:03:03.106303  6947 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.002s	sys 0.034s
I20260709 13:03:03.146744  1556 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:03:03.145275  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.012s	sys 0.009s
W20260709 13:03:03.149133  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.012s	sys 0.009s
W20260709 13:03:03.208781  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.036s	sys 0.028s
W20260709 13:03:03.209333  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.036s	sys 0.028s
I20260709 13:03:03.209740  6748 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.210734  7156 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:03.151242  7120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.010s	sys 0.007s
W20260709 13:03:03.216509  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.006s	sys 0.021s
I20260709 13:03:03.138356  6911 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:03.217511  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:03.146145  6947 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.003s	sys 0.035s
I20260709 13:03:03.087560  6693 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:03.251557  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.006s	sys 0.021s
I20260709 13:03:03.254195  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.259914  1558 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:03.135826  7090 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:03:03.204661  7126 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.263226  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.268375  2288 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:03.266268  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.269934  1556 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:03:03.252403  7071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.020s	sys 0.048s
W20260709 13:03:03.281690  7071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.020s	sys 0.048s
I20260709 13:03:03.282336  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2683) took 1997 ms. Trace:
I20260709 13:03:03.283493  1632 rpcz_store.cc:276] 0709 13:03:01.284623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.284748 (+   125us) service_pool.cc:224] Handling call
0709 13:03:01.285320 (+   572us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:01.299137 (+ 13817us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:02.147250 (+848113us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.883859 (+736609us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.895258 (+ 11399us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.895484 (+   226us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:02.895864 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.909983 (+ 14119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.910081 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.282001 (+371920us) raft_consensus.cc:1685] finished
0709 13:03:03.282035 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.282249 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43277,"prepare.queue_time_us":1251832,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":3956,"raft.queue_time_us":271296,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":46119,"thread_start_us":1516230,"threads_started":2,"wal-append.queue_time_us":987361}
I20260709 13:03:03.211483  7156 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.251930  6868 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.305001  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.271698  6915 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.254519  6911 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.307331  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.307930  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.315163  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:03.254765  6688 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.252583  6947 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:03.312261  2289 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:03.305840  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.265771  7098 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.316354  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.327448  1554 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:03:03.318574  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 461205us
I20260709 13:03:03.323549  1601 leader_election.cc:400] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:03.302539  7002 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.330837  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:03.322759  1558 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:03.328285  1601 leader_election.cc:403] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:03:03.330813  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.327981  1556 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:03.338100  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.343025  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.316015  7126 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.319963  6911 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.338958  1628 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:03.348986  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.354230  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.320724  7076 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.356009  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.356181  1634 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:03.355016  1631 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
W20260709 13:03:03.319149  7101 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.007s	sys 0.011s
I20260709 13:03:03.327891  7147 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.360545  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:03.337779  7136 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:03.315310  6868 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:03.360703  1704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.001s	sys 0.021s
W20260709 13:03:03.357246  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 334816us
I20260709 13:03:03.327240  7090 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:03.370847  1704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.001s	sys 0.021s
W20260709 13:03:03.361333  1672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.013s
W20260709 13:03:03.372072  1672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.007s	sys 0.014s
W20260709 13:03:03.372454  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.374235  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.374578  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.357466  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 885) took 1903 ms. Trace:
I20260709 13:03:03.366331  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:03.335613  7071 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.321476  6915 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.371605  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2139) took 1934 ms. Trace:
I20260709 13:03:03.375229  1631 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 11.
I20260709 13:03:03.383132  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.348261  7131 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.384374  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.386950  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:03.356568  7140 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.378659  1704 rpcz_store.cc:276] 0709 13:03:01.437469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.467647 (+ 30178us) service_pool.cc:224] Handling call
0709 13:03:01.468031 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.468274 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:01.977614 (+509340us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.371250 (+1393636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.371306 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:03.371352 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.371426 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.371547 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":749357,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":200559,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":1902132,"threads_started":3,"wal-append.queue_time_us":1042613}
I20260709 13:03:03.391429  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.340550  7002 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.419679  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.367133  6648 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:03.367851  7068 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.390225  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.379423  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:03.412817  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:03.413676  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.420776  2289 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:03.392910  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.336437  7098 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.342746  7127 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:03:03.336109  6688 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.370375  6350 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.423528  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.423486  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.394160  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:03.421924  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.024s	sys 0.023s
I20260709 13:03:03.363852  6763 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.450050  1705 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:03.452370  1631 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:03.395594  6983 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.374303  7133 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.353996  7126 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.453181  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.479710  1706 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:03.481281  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.483193  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:03.483870  1631 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.486840  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:03.393158  6878 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.361524  7000 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:03.459229  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:03.417433  1703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.003s	sys 0.029s
I20260709 13:03:03.379655  6836 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.395329  6971 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.375512  7090 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.415824  6910 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.376925  1635 rpcz_store.cc:276] 0709 13:03:01.453852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.453969 (+   117us) service_pool.cc:224] Handling call
0709 13:03:01.454256 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.454609 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.454637 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.455125 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.167482 (+712357us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.916509 (+749027us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.354863 (+438354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.357132 (+  2269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.357198 (+    66us) raft_consensus.cc:1685] finished
0709 13:03:03.357216 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.357393 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":481123,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":120491,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"thread_start_us":1863730,"threads_started":3,"wal-append.queue_time_us":1220876}
I20260709 13:03:03.469002  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:03.471951  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.024s	sys 0.023s
I20260709 13:03:03.421593  7101 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.472404  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78341.9us
I20260709 13:03:03.414566  7153 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.370106  7147 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.507704  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:03.508312  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.493723  2291 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.015s	sys 0.049s
I20260709 13:03:03.454334  7157 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.482214  1632 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:03.394660  7071 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.412503  7022 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:03:03.459165  6956 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.501493  1704 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:03.457248  7068 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.495962  7069 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.517098  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 907) took 2117 ms. Trace:
I20260709 13:03:03.520640  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.524030  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.475127  7160 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:03.532212  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:03.388994  7136 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.536136  7162 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.537213  7148 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 13:03:03.538024  7146 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
W20260709 13:03:03.539060  7155 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:03.539222  6953 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:03.540077  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.362167  7076 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.543198  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:03.489516  6763 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.453734  7131 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.555284  2293 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:03.554739  1635 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:03.554344  7073 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.541085  6915 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:03.651530  7152 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:03:03.652076  6915 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.652746  6915 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.551723  6956 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.655133  7126 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.657328  6956 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.657445  7002 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.527626  6693 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.515548  7000 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.560153  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.617404  6912 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:03.500658  6350 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:03:03.516232  7090 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.530483  6910 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.522701  6971 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.561740  6648 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:03.530306  6836 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.562831  7177 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:03.530112  7101 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.529827  7153 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.526616  7157 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.554277  1703 rpcz_store.cc:276] 0709 13:03:01.384968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.385438 (+   470us) service_pool.cc:224] Handling call
0709 13:03:01.391592 (+  6154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.391855 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:02.036979 (+645124us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.501788 (+1464809us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.501843 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:03.501888 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.501967 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.502093 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":807380,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":150776,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":2109069,"threads_started":3,"wal-append.queue_time_us":1278981}
W20260709 13:03:03.565359  6933 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 13:03:03.491808  6983 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.542335  7158 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.561867  7069 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.426614  6947 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:03.569219  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.422896  7140 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:03.512125  2291 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.015s	sys 0.050s
I20260709 13:03:03.582062  1631 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
W20260709 13:03:03.589061  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 283900us
W20260709 13:03:03.551901  7150 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:03:03.570564  7154 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:03:03.568135  7147 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:03.572438  7128 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.573086  7160 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.639721  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.494297  7133 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.651002  7169 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.587168  7162 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.558091  6688 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.653323  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2703) took 1154 ms. Trace:
I20260709 13:03:03.660818  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.658406  7002 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.664176  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:03.663957  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:03.516428  6878 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.674738  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.832440  1553 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:03.697016  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.674332  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:03.833588  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.833801  1560 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:03.702931  2293 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
W20260709 13:03:03.618582  7149 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:03:03.698002  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:03.837905  1552 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.707989  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.708842  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.635212  7165 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:03.634357  6953 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.715868  7068 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.718020  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:03.718995  7177 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.705885  7155 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:03:03.626243  7156 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.694716  6693 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:03.699697  6912 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.655256  6915 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.738514  6836 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.739248  7098 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.913604  7177 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.975630  6912 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:03.713364  6877 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.977921  6877 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.746513  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.980950  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.982367  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:03.755789  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 268196us
I20260709 13:03:03.659500  6763 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.784802  1632 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:03.985167  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.985898  1632 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:03:03.986339  6763 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:03.986954  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:03.789415  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:03.797605  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 310799us
I20260709 13:03:03.988839  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:03.989786  6877 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.990716  7176 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:03.983608  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.996023  7163 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:03.996785  7163 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:04.000782  7166 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 13:03:04.001013  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.002422  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2757) took 2397 ms. Trace:
I20260709 13:03:04.004240  7192 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.004978  7192 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.005399  7192 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.008132  6763 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:04.008519  7151 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.706s	user 0.001s	sys 0.002s
I20260709 13:03:04.008765  6763 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.009204  6763 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:04.011950  7185 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:03:04.005630  6912 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.013931  7168 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.195s
I20260709 13:03:04.016067  6912 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:04.018556  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.809998  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:04.022729  1631 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.693907  7073 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.751536  6956 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.752391  7181 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.819739  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 436280us
I20260709 13:03:04.074577  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:03.817780  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 438397us
I20260709 13:03:03.821204  1626 rpcz_store.cc:276] 0709 13:03:02.499402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.499760 (+   358us) service_pool.cc:224] Handling call
0709 13:03:02.500244 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.503574 (+  3330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.503620 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.504298 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.969899 (+465601us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:02.969938 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.970009 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.653034 (+683025us) raft_consensus.cc:1685] finished
0709 13:03:03.653065 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.653246 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5352,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6825,"thread_start_us":465234,"threads_started":1,"wal-append.queue_time_us":802325}
I20260709 13:03:03.824470  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.079348  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:04.080591  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.829593  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.829810  1635 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:03.834218  1553 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:04.002920  1702 rpcz_store.cc:276] 0709 13:03:01.604783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.605539 (+   756us) service_pool.cc:224] Handling call
0709 13:03:01.628408 (+ 22869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.866445 (+1238037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.866485 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.867247 (+   762us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.361130 (+493883us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:03.361167 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.361241 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.002148 (+640907us) raft_consensus.cc:1685] finished
0709 13:03:04.002178 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.002350 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136211,"raft.queue_time_us":240042,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":19103,"thread_start_us":1584012,"threads_started":2,"wal-append.queue_time_us":702910}
I20260709 13:03:04.082592  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:04.083796  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.084444  1553 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:04.085271  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.790777  6868 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:03.900429  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:03.835844  7147 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.198s
I20260709 13:03:04.103948  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.104532  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2699) took 1792 ms. Trace:
I20260709 13:03:03.905949  1559 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:03.910297  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.756554  6933 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:03.913982  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:03.815614  7160 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:03.920233  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.961222  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.720077  7182 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:03.990310  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2756) took 2476 ms. Trace:
I20260709 13:03:04.027246  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:03.902117  7165 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.991254  7176 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.054953  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 220680us
W20260709 13:03:04.057189  1523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.677s (SENT) [suppressed 5 similar messages]
I20260709 13:03:03.819285  7128 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.092716  2293 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:03:03.902549  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 72767.7us
I20260709 13:03:04.096592  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2701) took 1783 ms. Trace:
I20260709 13:03:04.102363  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.911188  6688 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.175s
I20260709 13:03:04.021273  7071 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.116842  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:04.117743  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 328370us
I20260709 13:03:04.104914  1630 rpcz_store.cc:276] 0709 13:03:02.312275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.312463 (+   188us) service_pool.cc:224] Handling call
0709 13:03:02.316729 (+  4266us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.317113 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.317141 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.317797 (+   656us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.920401 (+602604us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:02.920448 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.920536 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.104302 (+1183766us) raft_consensus.cc:1685] finished
0709 13:03:04.104324 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.104471 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60,"raft.queue_time_us":1591,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":602154,"threads_started":1,"wal-append.queue_time_us":890469}
I20260709 13:03:04.119451  1631 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:04.111033  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.120903  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1221) took 1215 ms. Trace:
I20260709 13:03:04.122252  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.119904  1708 rpcz_store.cc:276] 0709 13:03:01.513581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.519565 (+  5984us) service_pool.cc:224] Handling call
0709 13:03:01.616763 (+ 97198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.524313 (+907550us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:02.550387 (+ 26074us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.179282 (+628895us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.255569 (+ 76287us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.255735 (+   166us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:03.255795 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.255865 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.990115 (+734250us) raft_consensus.cc:1685] finished
0709 13:03:03.990137 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.990256 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192180,"prepare.queue_time_us":13927,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":294268,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":62357,"thread_start_us":1213800,"threads_started":2,"wal-append.queue_time_us":808454}
I20260709 13:03:04.123943  7165 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.124639  7173 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:04.127789  7190 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.110843  1706 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:04.129645  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:04.130223  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.181679  1557 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 11: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:04.131831  1554 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:04.181855  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52333.3us
I20260709 13:03:04.132431  6953 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.135056  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:03.911531  7002 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.135581  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:04.137097  1628 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:04.195480  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.117095  6912 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:04.257149  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 200002us
I20260709 13:03:04.210904  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:04.229027  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93521.4us
I20260709 13:03:04.206892  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:04.231778  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48871.8us
I20260709 13:03:04.266463  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.133886  7126 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.207341  7128 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.278728  1554 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:04.259526  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64139us
I20260709 13:03:04.141541  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:03.790498  7158 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.148211  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:03:04.286964  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 138861us
I20260709 13:03:04.156999  1560 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:04.143778  1633 rpcz_store.cc:276] 0709 13:03:02.312980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.313394 (+   414us) service_pool.cc:224] Handling call
0709 13:03:02.313789 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.314178 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.314202 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.314700 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.915934 (+601234us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:02.916003 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.916095 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.096334 (+1180239us) raft_consensus.cc:1685] finished
0709 13:03:04.096362 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.096517 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4484,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":600728,"threads_started":1,"wal-append.queue_time_us":930344}
I20260709 13:03:04.289211  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.164408  7076 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.177709  2296 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:04.292490  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.293582  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.177992  1635 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:04.177560  1707 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:04.177037  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.180437  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.197299  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:04.302194  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 121847us
I20260709 13:03:04.303063  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:04.197821  2293 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:04.030730  6693 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.306116  1630 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:04.127955  7182 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:03.826975  7169 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.183822  7068 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:03:04.198438  7195 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.214998  6910 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.216267  1702 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:04.210569  7069 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.218617  6933 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.134572  7098 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:04.221493  7097 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.238s	user 0.009s	sys 0.003s
I20260709 13:03:04.213707  7131 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:04.223472  7129 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.149s	user 0.012s	sys 0.001s
I20260709 13:03:04.185962  7136 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.276728  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.083065  6956 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.358927  7159 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.818s	user 0.006s	sys 0.002s
I20260709 13:03:04.359058  7186 raft_consensus.cc:2764] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:04.140956  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:04.360229  1553 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:04.360008  7172 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:04.360500  1703 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
W20260709 13:03:04.361277  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 84662.5us
I20260709 13:03:04.355465  7097 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.362020  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.362850  7127 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:04.363263  7097 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.363793  7187 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:04.364238  7187 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:04.364707  7187 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.285697  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.366078  7184 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.366501  1704 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:04.366612  7184 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.366120  6956 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.367376  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.206425  7190 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.368026  1704 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:04.166893  2291 rpcz_store.cc:276] 0709 13:03:02.905397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.905544 (+   147us) service_pool.cc:224] Handling call
0709 13:03:02.905870 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.906146 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:03.787416 (+881270us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.120489 (+333073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.120545 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:04.120613 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.120696 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.120835 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17044,"prepare.queue_time_us":165937,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":17816,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":1125480,"threads_started":2}
I20260709 13:03:04.363349  7127 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:04.369668  7190 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:04.205976  7180 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:03:04.369925  7127 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.209836  7179 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.303s	user 0.000s	sys 0.000s
I20260709 13:03:04.314862  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.375813  1559 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:04.208964  7153 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.091426  7181 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.211012  7163 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:04.211789  7140 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.332907  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:04.334597  1551 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:04.339960  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:04.320627  6911 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.962s	user 0.007s	sys 0.004s
I20260709 13:03:04.223024  6983 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:04.286823  7000 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.213456  7176 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.322981  7185 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:04.186589  7173 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.309123  6912 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.325387  7147 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:04.179198  7177 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:04.358121  7178 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
I20260709 13:03:04.364379  1627 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:04.326936  7076 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.218426  7089 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:04.368906  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.311749  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.388156  1558 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:04.369818  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:04.348819  6910 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:04.376029  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.459517  1552 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 6.
I20260709 13:03:04.429572  6933 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:04.425858  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.462107  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.462441  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:04.428164  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.434697  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:04.400915  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78655.9us
I20260709 13:03:04.353920  7101 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.346092  7194 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.315431  6915 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:04.463990  1559 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:04.352001  7195 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:04.368870  7184 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.314064  7126 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.439604  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:04.472985  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:04.439888  7173 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:04.439952  7185 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.444658  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.385738  7098 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:04.469028  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.407936  6971 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:04.460886  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.462814  1554 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:04.377190  7068 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:04.441382  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.426952  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.466470  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:04.470860  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.482393  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.371508  7129 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.467816  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.469633  1626 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:03:04.454613  7133 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:04.530053  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59275.9us
I20260709 13:03:04.437060  6911 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.479100  1552 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:04.483278  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:04.468497  6910 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.451157  7076 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:04.455037  7141 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.393s	user 0.009s	sys 0.005s
I20260709 13:03:04.457211  7147 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:04.551051  2297 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:04.528825  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.551587  1559 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:04.485486  7184 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:04.483230  1707 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:04.557835  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:04.536134  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52962.6us
W20260709 13:03:04.458182  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164676us
W20260709 13:03:04.537137  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 68237.5us
I20260709 13:03:04.465946  6915 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:04.539661  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 361545us
I20260709 13:03:04.477267  7185 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.560797  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.459259  7157 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.541085  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.543715  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.564347  1708 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:04.456462  7089 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:04.564527  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:04.550884  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78046us
I20260709 13:03:04.528823  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.468492  7192 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.539510  7100 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.013s	sys 0.027s
I20260709 13:03:04.562602  1556 raft_consensus.cc:2368] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:04.563818  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.572772  1704 raft_consensus.cc:2410] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:04.467077  7195 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:04.563880  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:04.567065  1560 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:04.478068  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:04.567239  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.480147  7068 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:04.538703  7153 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:04.545604  6910 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.477082  7162 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:04.547673  7076 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:04.550812  6763 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.005s
I20260709 13:03:04.575883  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:04.547273  7133 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.577019  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.577181  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.548450  7174 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:04.549427  7194 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.579983  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.581071  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.555694  7141 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:04.476671  6933 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.581534  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:04.583889  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.555560  7184 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.586521  1707 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.589623  2289 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:04.562844  6915 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:04.581449  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.555195  7208 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:04.572140  7100 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.013s	sys 0.028s
I20260709 13:03:04.566219  7190 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.591557  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2708) took 1164 ms. Trace:
I20260709 13:03:04.622910  1630 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:04.626760  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.485632  7098 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.590811  7131 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.577265  7195 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:04.616649   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:04.627483  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43667.1us
I20260709 13:03:04.625676  1635 rpcz_store.cc:276] 0709 13:03:03.452725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.321106 (+868381us) service_pool.cc:224] Handling call
0709 13:03:04.321528 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.321783 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:04.345765 (+ 23982us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.590276 (+244511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.590347 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:04.590416 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.590520 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.590676 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19976,"prepare.queue_time_us":25972,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":20187,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":26280,"thread_start_us":180313,"threads_started":1}
I20260709 13:03:04.630260  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:04.623184  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:04.628058  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 148721us
I20260709 13:03:04.641937  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.646209  1635 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:04.646874  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.641139  1555 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:04.646023  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.622079  7089 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.651027  7207 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:04.651568  7207 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:04.651700  7144 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:04.652071  7207 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.652594  7144 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:04.653129  7144 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:04.654139  7144 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.656517  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.657174  1626 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:04.657868  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:04.658141  7216 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:03:04.658712  1553 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:04.658438  7068 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.659235  7189 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:03:04.659487  7068 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.660202  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2706) took 1237 ms. Trace:
W20260709 13:03:04.660450  7135 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.657s	user 0.008s	sys 0.008s
I20260709 13:03:04.614971  6878 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:04.658000  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.674253  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:04.595510  7153 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:04.659535  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.674867  1552 raft_consensus.cc:2376] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14. Re-sending same reply.
I20260709 13:03:04.660583  1634 rpcz_store.cc:276] 0709 13:03:03.422391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.422500 (+   109us) service_pool.cc:224] Handling call
0709 13:03:03.426471 (+  3971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.491076 (+ 64605us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:03.497318 (+  6242us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.830494 (+333176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.830541 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:03.830582 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.659978 (+829396us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b54001c34d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:04.660032 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.660147 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31735,"prepare.queue_time_us":6466,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":88309,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":120066,"spinlock_wait_cycles":195139200,"thread_start_us":325408,"threads_started":1,"wal-append.queue_time_us":686649}
W20260709 13:03:04.632218  7199 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:03:04.673373  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:04.615242  7002 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.593950  7174 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:04.676129  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:04.676153  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:04.625600  7184 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.673341  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:04.683719  1560 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:03:04.621450  6693 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.680052  1553 raft_consensus.cc:2376] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15. Re-sending same reply.
W20260709 13:03:04.681135  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87419.1us
I20260709 13:03:04.686625  1626 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:04.679857  1552 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:04.627221  7208 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:04.642154  7168 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:04.656569  7201 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:03:04.706831  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.592589  6763 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.689462  1554 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:04.717413  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:04.690898  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110973us
I20260709 13:03:04.595953  7133 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.719245  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:04.688969  7174 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:04.701783  7172 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:04.722913  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:04.632647  7190 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.754992  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.755752  1554 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:03:04.735080  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1254) took 1354 ms. Trace:
I20260709 13:03:04.736763  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.702256  7168 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:04.749574  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:04.672672  6983 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:04.755203  2291 raft_consensus.cc:2368] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:04.766396  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.757603  1560 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:04.770413  1634 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:04.757181  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:04.714326  6878 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.702216  7208 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:04.756872  2288 rpcz_store.cc:276] 0709 13:03:03.380955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.381068 (+   113us) service_pool.cc:224] Handling call
0709 13:03:03.384234 (+  3166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.384591 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.384619 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.419285 (+ 34666us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.816285 (+397000us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:03.816345 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.816444 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.734777 (+918333us) raft_consensus.cc:1685] finished
0709 13:03:04.734808 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.734996 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130955,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":64979,"thread_start_us":335719,"threads_started":1,"wal-append.queue_time_us":647381}
I20260709 13:03:04.760574  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2762) took 1020 ms. Trace:
I20260709 13:03:04.774166  1707 rpcz_store.cc:276] 0709 13:03:03.740136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.575484 (+835348us) service_pool.cc:224] Handling call
0709 13:03:04.760511 (+185027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.727276  7194 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:04.721517  7182 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.698086  7165 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.776614  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.742599  7172 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:04.789732  1554 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:04.755388  7002 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:04.786439  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.787631  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.787561  7131 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:04.795258  1674 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:04.795543  1707 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:04.796072  1674 leader_election.cc:403] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:04.796795  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:04.762341  7168 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.702441  7147 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:04.807081  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.788033  1552 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:04.797587  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1255) took 1416 ms. Trace:
I20260709 13:03:04.799726  2290 raft_consensus.cc:2368] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:04.807824  1554 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:04.774999  7228 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:04.776274  7177 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.811197  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:04.813264  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.814185  1552 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:04.810822  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.808218  2292 rpcz_store.cc:276] 0709 13:03:03.381330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.381450 (+   120us) service_pool.cc:224] Handling call
0709 13:03:03.422851 (+ 41401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.825657 (+402806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.825699 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.913209 (+ 87510us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.228870 (+315661us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:04.228937 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.229114 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.651310 (+422196us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b480024e1d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:04.793028 (+141718us) raft_consensus.cc:1685] finished
0709 13:03:04.793049 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.797520 (+  4471us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116633,"raft.run_cpu_time_us":13134,"raft.run_wall_time_us":1111610,"spinlock_wait_cycles":71274752,"thread_start_us":647780,"threads_started":2,"wal-append.queue_time_us":470127}
I20260709 13:03:04.792240  6911 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.816740  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.817664  1560 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:03:04.819281  7203 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:03:04.819701  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.820475  1552 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:04.815596  7226 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:04.795215  7211 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:04.800732  7131 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:04.805754  7134 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:04.828703  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:04.799970  7162 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:04.785729  6878 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:04.799276  7218 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:04.833431  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.832896  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.838130  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 917) took 1331 ms. Trace:
I20260709 13:03:04.845782  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:04.870352  7177 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.870878  7177 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.871222  7177 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.789474  7153 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.913187  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1257) took 1511 ms (client timeout 1677 ms). Trace:
I20260709 13:03:04.871490  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:04.846496  1704 rpcz_store.cc:276] 0709 13:03:03.506469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.623338 (+1116869us) service_pool.cc:224] Handling call
0709 13:03:04.623646 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.624024 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.624052 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.629676 (+  5624us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.786283 (+156607us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:04.786359 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.786482 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.837914 (+ 51432us) raft_consensus.cc:1685] finished
0709 13:03:04.837935 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.838068 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8858,"raft.queue_time_us":2776,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":13132,"thread_start_us":74844,"threads_started":1,"wal-append.queue_time_us":149049}
I20260709 13:03:05.042037  1554 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:05.043248  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.044054  1704 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:05.044863  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.021201  1552 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:05.045107  7229 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 13:03:05.045411  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:05.046330  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:04.900695  7213 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:05.048223  7213 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.048681  7213 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.021739  7134 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:05.051489  7134 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.874713  7195 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:04.825887  6912 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.925912  1559 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:03:05.054451  6912 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:04.914470  6933 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.899803  7185 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:05.041525  2296 rpcz_store.cc:269] 0709 13:03:03.382786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.382893 (+   107us) service_pool.cc:224] Handling call
0709 13:03:04.893750 (+1510857us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540050b754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:04.893915 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825651968}
I20260709 13:03:04.877674  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:05.057224  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 179652us
I20260709 13:03:05.058244  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.058771  7192 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:04.828104  7211 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.872570  7098 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.021802  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.064262  7213 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.063448  7211 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:05.063799  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 306331us
I20260709 13:03:05.047416  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.067185  2293 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:03:05.069309  7126 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.070298  7168 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:05.070622  2290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1038) took 1682 ms (client timeout 2011 ms). Trace:
I20260709 13:03:05.023253  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.074332  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:05.076886  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53713.2us
W20260709 13:03:05.074918  2290 rpcz_store.cc:269] 0709 13:03:03.388054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.637545 (+249491us) service_pool.cc:224] Handling call
0709 13:03:05.070411 (+1432866us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:05.070562 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222008576}
I20260709 13:03:05.077564  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:04.872020  7172 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:04.892742  7160 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.008s
I20260709 13:03:05.079082  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:04.924788  7226 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:05.086920  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.089975  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.090339  2296 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:05.090622  7226 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.091279  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:05.092571  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.093127  2296 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:05.093895  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.094466  2296 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:05.095257  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.096213  6878 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.085601  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:05.086872  7126 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.097198  7126 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.099069  2296 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:05.099145  7126 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.099917  7126 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:05.100457  7126 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:05.101475  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:05.101372  7126 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.102783  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:05.102612  7207 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.103650  1708 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
W20260709 13:03:05.104315  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.096340  7227 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:03:05.106063  7177 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.106508  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.107789  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.107843  6912 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:05.108537  1630 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:05.091324  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.110317  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.110383  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.111294  7177 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.111871  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.113276  1709 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:05.113966  6933 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:05.114228  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.114380  6915 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.072484  6763 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.123548  1560 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:05.115105  7187 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.750s	user 0.005s	sys 0.003s
I20260709 13:03:05.124620  7187 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.125309  7221 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:05.125948  7221 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.126312  7221 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.078506  7222 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:05.128180  7222 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.128564  7222 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:05.114988  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:05.108657  7181 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.106065  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:04.901353  7218 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.078696  7192 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.120340  7165 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.121310  7174 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.125399  7223 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:05.130553  6910 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 13:03:05.130100  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:05.132647  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.130887  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:05.121666  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:05.159420  2290 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:05.159844  1552 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:05.083046  7098 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:05.066540  7228 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:05.160830  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.161058  6912 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:05.161733  7228 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.162106  6912 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.131206  7172 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.115367  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:05.169682  2296 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:05.123543  7090 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.419s	user 0.011s	sys 0.001s
W20260709 13:03:05.165143  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 307605us
I20260709 13:03:05.149375  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.176110  7133 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.182749  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:05.089239  7127 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:03:05.089386  7160 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.189484  7185 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.190385  7207 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.191707  7208 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.193454  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:05.195502  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.218075  1704 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:05.199031  7140 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:05.021477  7131 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.188428  7187 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.164520  7192 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:05.167336  7098 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.157963  7223 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.222196  2296 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:05.223047  1635 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:05.210042  7173 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.004s
I20260709 13:03:05.231406  7173 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.202894  6915 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.167716  6910 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.181068  7090 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.231855  7218 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.229754  1707 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:05.233673  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.247215  1704 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:03:05.236826  7231 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:05.237841  7173 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.238669  7168 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.230317  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.223004  6933 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:05.227195  7207 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:05.228745  7098 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.243302  7131 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.232836  7223 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.258289  1629 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:05.223743  7140 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.242309  7162 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:05.262209  6910 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:05.262684  7076 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
I20260709 13:03:05.263983  7207 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.301213  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.302392  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.269099  7090 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.269294  7208 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.303002  1705 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:05.279702  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.304474  1630 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:05.274516  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2087) took 2117 ms. Trace:
I20260709 13:03:05.289500  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.292343  7222 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.295650  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.302474  7207 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.277113  7129 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.306309  2296 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:05.306087  1555 rpcz_store.cc:276] 0709 13:03:03.157452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.157607 (+   155us) service_pool.cc:224] Handling call
0709 13:03:03.158008 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.648964 (+1490956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.274253 (+625289us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002bbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:05.274427 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3410346880}
I20260709 13:03:05.312243  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.313165  1626 raft_consensus.cc:2368] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15.
I20260709 13:03:05.260375  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.314851  1704 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 6.
I20260709 13:03:05.310889  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.307169  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.310278  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.318681  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.318936  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.319337  1626 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:05.324925  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.324991  7211 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.325682  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
W20260709 13:03:05.325906  7188 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.092s	user 0.000s	sys 0.003s
I20260709 13:03:05.326550  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.329072  1555 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:03:05.329993  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.307723  7076 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.380182  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.371322  1626 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:05.336977  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:05.333966  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1223) took 2344 ms (client timeout 1998 ms). Trace:
W20260709 13:03:05.721376  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 384456us
I20260709 13:03:05.331112  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.366083  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.689388  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.723316  1630 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:05.723627  1700 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.724051  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2849) took 1058 ms. Trace:
I20260709 13:03:05.724505  1630 rpcz_store.cc:276] 0709 13:03:04.665815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.365734 (+699919us) service_pool.cc:224] Handling call
0709 13:03:05.723931 (+358197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.678241  1555 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:05.392182  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.725713  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.726310  1704 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.728575  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.730000  1630 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:05.721925  2294 rpcz_store.cc:269] 0709 13:03:02.977147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.981463 (+  4316us) service_pool.cc:224] Handling call
0709 13:03:05.321273 (+2339810us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:05.321440 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521110272}
W20260709 13:03:05.444855  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.734973  1557 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:03:05.730564  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2853) took 1047 ms. Trace:
W20260709 13:03:05.390134  7250 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:03:05.337222  7184 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.632s	user 0.008s	sys 0.003s
I20260709 13:03:05.743451  1557 pending_rounds.cc:85] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 9
I20260709 13:03:05.745054  1557 pending_rounds.cc:107] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 12.10
W20260709 13:03:05.688829  7126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.001s	sys 0.016s
I20260709 13:03:05.747736  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.730161  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1268) took 1051 ms. Trace:
I20260709 13:03:05.748904  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.745553  1557 raft_consensus.cc:2889] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:05.743723  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.749060  2296 rpcz_store.cc:276] 0709 13:03:04.678969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.338675 (+659706us) service_pool.cc:224] Handling call
0709 13:03:05.727284 (+388609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.749792  1633 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:05.750383  1633 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:05.750491  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:05.744423  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.380172  7226 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:05.743582  1626 rpcz_store.cc:276] 0709 13:03:04.683361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.728286 (+1044925us) service_pool.cc:224] Handling call
0709 13:03:05.730506 (+  2220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.432065  7090 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:05.349007  6933 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:05.751827  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1237) took 1051 ms. Trace:
I20260709 13:03:05.686018  7211 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.393486  7140 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.665135  7219 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.664409  6910 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:05.801992  6910 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.755445  7226 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:05.409981  7174 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:05.800783  7221 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.674s	user 0.010s	sys 0.003s
I20260709 13:03:05.727721  7156 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.752056  7184 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.759181  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:05.760461  7158 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.761713  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1045) took 1398 ms. Trace:
I20260709 13:03:05.763415  7195 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.764629  1704 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:05.823037  2293 rpcz_store.cc:276] 0709 13:03:04.363052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.365311 (+  2259us) service_pool.cc:224] Handling call
0709 13:03:04.365681 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.073764 (+708083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.761443 (+687679us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54002c8dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:05.761650 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612701056}
I20260709 13:03:05.764822  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.768741  7213 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:05.770861  7205 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.562s	user 0.007s	sys 0.002s
I20260709 13:03:05.771541  7073 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.772224  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.772682  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.773591  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:05.830016  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57888.9us
W20260709 13:03:05.830298  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 56820.3us
I20260709 13:03:05.830842  1674 leader_election.cc:400] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15.
I20260709 13:03:05.830899  2293 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:05.831249  1674 leader_election.cc:403] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:05.831298  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:05.831741  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1260) took 1393 ms. Trace:
I20260709 13:03:05.776144  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:05.777518  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.751685  7126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.001s	sys 0.021s
I20260709 13:03:05.784688  1700 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:05.789438  7090 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.791344  2294 rpcz_store.cc:276] 0709 13:03:04.700506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.746570 (+1046064us) service_pool.cc:224] Handling call
0709 13:03:05.751709 (+  5139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.688336  7223 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.795531  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:05.837936  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1259) took 1410 ms. Trace:
I20260709 13:03:05.838380  2296 rpcz_store.cc:276] 0709 13:03:04.428977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.755995 (+1327018us) service_pool.cc:224] Handling call
0709 13:03:05.837874 (+ 81879us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.809397  7211 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:05.805928  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.755956  1630 raft_consensus.cc:2368] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12.
I20260709 13:03:05.430572  7160 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:05.809655  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 896) took 2451 ms (client timeout 2215 ms). Trace:
I20260709 13:03:05.821331  1701 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:05.378623  7207 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:05.836089  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:05.840636  1629 rpcz_store.cc:269] 0709 13:03:03.354480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.355720 (+  1240us) service_pool.cc:224] Handling call
0709 13:03:05.805633 (+2449913us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54001419d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:05.805731 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4370097920}
I20260709 13:03:05.841815  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.841925  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.842597  1629 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:05.832193  2293 rpcz_store.cc:276] 0709 13:03:04.438384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.826585 (+1388201us) service_pool.cc:224] Handling call
0709 13:03:05.830729 (+  4144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.831443 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.831682 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.810973  7076 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.840886  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2168) took 1044 ms. Trace:
I20260709 13:03:05.854171  1701 rpcz_store.cc:276] 0709 13:03:04.796561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.758890 (+962329us) service_pool.cc:224] Handling call
0709 13:03:05.840823 (+ 81933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.843758  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 918) took 2337 ms. Trace:
I20260709 13:03:05.855268  1700 rpcz_store.cc:276] 0709 13:03:03.506693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.842820 (+2336127us) service_pool.cc:224] Handling call
0709 13:03:05.843148 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.843380 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.843416 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.843469 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:05.843509 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.843577 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.843692 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.857422  1674 leader_election.cc:400] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12.
I20260709 13:03:05.857820  1674 leader_election.cc:403] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:03:05.823825  7156 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:05.860416  2293 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:05.861091  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1238) took 1160 ms. Trace:
I20260709 13:03:05.861410  2293 rpcz_store.cc:276] 0709 13:03:04.700733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.858442 (+1157709us) service_pool.cc:224] Handling call
0709 13:03:05.860243 (+  1801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.860859 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.861041 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.806607  7219 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.842105  7073 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15.
I20260709 13:03:05.846107  7184 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:05.849359  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.850072  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1256) took 2468 ms. Trace:
I20260709 13:03:05.850281  7223 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.857748  7208 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:05.850603  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:05.837016  6910 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:05.871814  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:05.872650  1557 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:05.851104  7185 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:05.851919  7098 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
I20260709 13:03:05.852260  7134 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.840142  1631 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:05.857899  1700 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:05.883132  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 921) took 2364 ms. Trace:
I20260709 13:03:05.883638  1700 rpcz_store.cc:276] 0709 13:03:03.519099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.856480 (+2337381us) service_pool.cc:224] Handling call
0709 13:03:05.857513 (+  1033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.882762 (+ 25249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.883059 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.885270  1700 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:05.885969  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 923) took 2352 ms. Trace:
I20260709 13:03:05.886330  1700 rpcz_store.cc:276] 0709 13:03:03.533651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.884642 (+2350991us) service_pool.cc:224] Handling call
0709 13:03:05.885049 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.885690 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.885906 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.853812  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.863312  2293 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:05.844316  2294 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:05.836501  6934 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:05.836884  7158 raft_consensus.cc:2764] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:03:05.856315  7172 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.000s
I20260709 13:03:05.841317  6911 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:05.868374  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:03:05.850803  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1272) took 1148 ms (client timeout 1521 ms). Trace:
I20260709 13:03:05.868978  1634 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:05.866194  2295 rpcz_store.cc:276] 0709 13:03:03.381837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.381943 (+   106us) service_pool.cc:224] Handling call
0709 13:03:03.382276 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:03.784818 (+402542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.784856 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:03.836449 (+ 51593us) log.cc:844] Serialized 52 byte log entry
0709 13:03:04.160930 (+324481us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:04.160985 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.161100 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.769406 (+1608306us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b480024e1d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:05.770216 (+   810us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:05.849819 (+ 79603us) raft_consensus.cc:1685] finished
0709 13:03:05.849840 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.849989 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2476,"raft.queue_time_us":154010,"raft.run_cpu_time_us":31138,"raft.run_wall_time_us":2232868,"spinlock_wait_cycles":183116544,"thread_start_us":485671,"threads_started":2,"wal-append.queue_time_us":297197}
I20260709 13:03:05.823374  7221 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:05.859635  7131 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.002s
I20260709 13:03:05.874284  7223 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.864123  7130 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.863220  7156 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.868041  7208 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.888513  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1261) took 1446 ms. Trace:
I20260709 13:03:05.893608  6934 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:05.890668  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:05.894009  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1265) took 1320 ms. Trace:
I20260709 13:03:05.929888  2291 rpcz_store.cc:276] 0709 13:03:04.573622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.630078 (+ 56456us) service_pool.cc:224] Handling call
0709 13:03:05.893817 (+1263739us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:05.893961 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245057920}
I20260709 13:03:05.894357  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2156) took 2201 ms. Trace:
I20260709 13:03:05.901280  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:05.901019  2292 rpcz_store.cc:269] 0709 13:03:04.702340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.816147 (+113807us) service_pool.cc:224] Handling call
0709 13:03:05.842449 (+1026302us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54005186d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:05.850748 (+  8299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834292608}
I20260709 13:03:05.907145  1557 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:05.879343  7185 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:05.911715  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2730) took 1075 ms. Trace:
I20260709 13:03:05.934211  1634 rpcz_store.cc:276] 0709 13:03:04.836851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.849068 (+1012217us) service_pool.cc:224] Handling call
0709 13:03:05.911621 (+ 62553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.917263  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:05.900933  6911 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.888513  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1239) took 1187 ms. Trace:
I20260709 13:03:05.935459  2293 rpcz_store.cc:276] 0709 13:03:04.700994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.862045 (+1161051us) service_pool.cc:224] Handling call
0709 13:03:05.863090 (+  1045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.888217 (+ 25127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.888452 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.923125  2294 rpcz_store.cc:276] 0709 13:03:04.442197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.837494 (+1395297us) service_pool.cc:224] Handling call
0709 13:03:05.844068 (+  6574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.888289 (+ 44221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.888450 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.924220  7208 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:05.942305  7172 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.942932  7172 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.925382  7090 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.926687  7219 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.864177  7207 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.920213  7223 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.929270  1554 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:05.921345  7071 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:05.932107  1559 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:05.932701  1700 rpcz_store.cc:276] 0709 13:03:03.693031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.887187 (+2194156us) service_pool.cc:224] Handling call
0709 13:03:05.887562 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.893906 (+  6344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.893952 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.894018 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.894072 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.894157 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.894291 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.944796  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:05.946135  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:05.882807  7098 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:05.959915  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:05.919603  1631 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:05.944226  6911 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:05.962702  2292 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:05.964564  1555 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:05.963567  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1054) took 1227 ms. Trace:
I20260709 13:03:05.965587  1634 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:05.965950  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2758) took 2272 ms. Trace:
I20260709 13:03:05.965821  2292 rpcz_store.cc:276] 0709 13:03:04.736159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.944027 (+1207868us) service_pool.cc:224] Handling call
0709 13:03:05.958732 (+ 14705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.963248 (+  4516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.963483 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.863711  7226 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:05.966646  1704 rpcz_store.cc:276] 0709 13:03:03.694119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.946201 (+2252082us) service_pool.cc:224] Handling call
0709 13:03:05.965469 (+ 19268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.965655 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.965677 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.965718 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:05.965753 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.965800 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.965889 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.971112  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:05.934715  6934 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.977977  1626 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13.
I20260709 13:03:05.978206  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.979179  1634 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:05.944964  7185 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.982632  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2854) took 1281 ms. Trace:
I20260709 13:03:05.982625  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:05.981966  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:05.988859  7193 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.781s	user 0.004s	sys 0.004s
I20260709 13:03:05.989465  7098 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.991268  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:05.994277  1628 raft_consensus.cc:606] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:05.994676  1628 raft_consensus.cc:614] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:05.983613  1634 rpcz_store.cc:276] 0709 13:03:04.701726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.977914 (+1276188us) service_pool.cc:224] Handling call
0709 13:03:05.982566 (+  4652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.995744  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2760) took 2277 ms. Trace:
I20260709 13:03:05.997972  1704 rpcz_store.cc:276] 0709 13:03:03.717901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.970330 (+2252429us) service_pool.cc:224] Handling call
0709 13:03:05.994804 (+ 24474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.994998 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.995026 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.995071 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:05.995449 (+   378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.995520 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.995676 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.968758  7071 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:06.007918  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2163) took 1423 ms (client timeout 1539 ms). Trace:
I20260709 13:03:06.009613  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:06.010180  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45338 (request call id 24) took 1158 ms. Trace:
I20260709 13:03:06.010602  1628 rpcz_store.cc:276] 0709 13:03:04.836555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.991089 (+1154534us) service_pool.cc:224] Handling call
0709 13:03:05.995032 (+  3943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.012280  1631 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:06.009753  1708 rpcz_store.cc:269] 0709 13:03:04.584949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.594472 (+  9523us) service_pool.cc:224] Handling call
0709 13:03:06.007681 (+1413209us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54003e2454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.007838 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058167552}
I20260709 13:03:05.977365  7219 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:06.016309  1634 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:06.017160  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2837) took 2179 ms. Trace:
I20260709 13:03:06.017786  1634 rpcz_store.cc:276] 0709 13:03:03.837669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.014240 (+2176571us) service_pool.cc:224] Handling call
0709 13:03:06.014489 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.016869 (+  2380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.017090 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.046854  7223 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:05.982614  7226 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.049517  1704 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:06.050341  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2763) took 2309 ms. Trace:
I20260709 13:03:06.050704  1704 rpcz_store.cc:276] 0709 13:03:03.740509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.048625 (+2308116us) service_pool.cc:224] Handling call
0709 13:03:06.049075 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.050033 (+   958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.050273 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.008872  7192 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:03:06.015057  7126 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:03:06.058753  7126 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:06.059229  7126 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:06.060212  7071 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.014936  7167 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:06.062368  7167 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:06.063474  7167 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.063975  7223 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:06.064684  7167 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:06.065969  7219 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:06.067024  7219 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.084897  7172 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.092958  7207 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:06.183947  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.075321  7208 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.197861  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2153) took 2675 ms. Trace:
I20260709 13:03:06.198523  1701 rpcz_store.cc:276] 0709 13:03:03.522525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.858126 (+2335601us) service_pool.cc:224] Handling call
0709 13:03:05.859591 (+  1465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.859831 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:05.868273 (+  8442us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.197316 (+329043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.197381 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:06.197445 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.197564 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.197763 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":328594,"threads_started":1}
W20260709 13:03:06.178198  7153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.047s	sys 0.088s
I20260709 13:03:06.091439  6911 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:06.268826  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2152) took 2710 ms (client timeout 1893 ms). Trace:
W20260709 13:03:06.269690  1706 rpcz_store.cc:269] 0709 13:03:03.501219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.514937 (+ 13718us) service_pool.cc:224] Handling call
0709 13:03:06.206753 (+2691816us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.206893 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279337216}
I20260709 13:03:06.205459  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2161) took 1719 ms. Trace:
I20260709 13:03:06.275588  1701 rpcz_store.cc:276] 0709 13:03:04.485729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.204045 (+1718316us) service_pool.cc:224] Handling call
0709 13:03:06.204431 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.204636 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.204661 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.204708 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:06.204749 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.204820 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.205384 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.217442  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 466991us
I20260709 13:03:06.278458  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:06.206801  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1036) took 2820 ms (client timeout 1914 ms). Trace:
W20260709 13:03:06.279524  2297 rpcz_store.cc:269] 0709 13:03:03.386601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.386721 (+   120us) service_pool.cc:224] Handling call
0709 13:03:06.206566 (+2819845us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005e06d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.206721 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604542848}
W20260709 13:03:06.290870  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.293426  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.293249  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:06.216841  7153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.049s	sys 0.091s
I20260709 13:03:06.309456  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:06.343484  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:06.194105  7130 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12.
W20260709 13:03:06.353991  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2764) took 2613 ms (client timeout 1677 ms). Trace:
W20260709 13:03:06.354578  1703 rpcz_store.cc:269] 0709 13:03:03.740778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.983324 (+242546us) service_pool.cc:224] Handling call
0709 13:03:06.353745 (+2370421us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001cb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.353899 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837527936}
W20260709 13:03:06.356416  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47073.1us
I20260709 13:03:06.357194  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:06.352546  7174 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.301054  7090 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:06.360443  7090 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.362581  6911 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.362754  7090 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.199852  7126 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.363327  7218 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:06.364171  7218 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.366474  7218 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.367005  7218 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.368073  7169 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.367482  7218 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:06.369498  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2689) took 1983 ms (client timeout 1668 ms). Trace:
W20260709 13:03:06.370021  1556 rpcz_store.cc:269] 0709 13:03:04.385663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.412203 (+ 26540us) service_pool.cc:224] Handling call
0709 13:03:06.369212 (+1957009us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b540030a7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.369376 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181086080}
I20260709 13:03:06.373507  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 885) took 1179 ms. Trace:
I20260709 13:03:06.343421  7134 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:06.301064  7172 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:06.374852  7172 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:06.344255  6910 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.375310  7218 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:06.376493  6910 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.378167  2292 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:06.378695  6910 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.343888  7248 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.379338  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.379619  7248 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.380285  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.375334  7172 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:06.371428  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.383594  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.384262  1556 raft_consensus.cc:2368] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:06.375445  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:06.385262  1630 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:06.386428  1630 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:06.388974  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2169) took 1562 ms. Trace:
I20260709 13:03:06.389562  1703 rpcz_store.cc:276] 0709 13:03:04.826361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.358114 (+1531753us) service_pool.cc:224] Handling call
0709 13:03:06.388257 (+ 30143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.388506 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.388561 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.388622 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:06.388677 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.388750 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.388900 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.392266  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 863) took 2727 ms. Trace:
I20260709 13:03:06.392474  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.392977  1628 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:06.392791  1551 rpcz_store.cc:276] 0709 13:03:03.672695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.673604 (+   909us) service_pool.cc:224] Handling call
0709 13:03:03.708367 (+ 34763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.402553 (+694186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.392013 (+1989460us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54000776d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:06.392193 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440078976}
I20260709 13:03:06.394460  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1044) took 2133 ms. Trace:
I20260709 13:03:06.394820  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.392495  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2710) took 2927 ms. Trace:
I20260709 13:03:06.395500  1551 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:06.404611  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46986.7us
I20260709 13:03:06.395607  1633 rpcz_store.cc:276] 0709 13:03:03.465537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.314558 (+849021us) service_pool.cc:224] Handling call
0709 13:03:06.392179 (+2077621us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002972d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.392421 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157994752}
I20260709 13:03:06.373999  1554 rpcz_store.cc:276] 0709 13:03:05.194368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.194515 (+   147us) service_pool.cc:224] Handling call
0709 13:03:05.194899 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.958184 (+763285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.373227 (+415043us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54003fa054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:06.373441 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1688282880}
I20260709 13:03:06.394863  2289 rpcz_store.cc:276] 0709 13:03:04.261168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.276220 (+ 15052us) service_pool.cc:224] Handling call
0709 13:03:04.316858 (+ 40638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.622627 (+305769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.394220 (+1771593us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54007bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:06.394396 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626788736}
I20260709 13:03:06.435003  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.375440  6933 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:06.435345  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:06.436055  2293 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.436581  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.437191  1556 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:06.396170  7221 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.444694  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.445389  1554 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.446731  7126 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.374325  7134 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.447955  7185 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.450160  7185 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.413142  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.453791  1556 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:06.406114  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:06.412320  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2841) took 2384 ms. Trace:
W20260709 13:03:06.398351  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 905) took 1849 ms (client timeout 2057 ms). Trace:
I20260709 13:03:06.456172  1630 rpcz_store.cc:276] 0709 13:03:04.027945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.385853 (+2357908us) service_pool.cc:224] Handling call
0709 13:03:06.386235 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.411960 (+ 25725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.412238 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.456676  1635 rpcz_store.cc:269] 0709 13:03:04.549021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.673124 (+124103us) service_pool.cc:224] Handling call
0709 13:03:06.398136 (+1725012us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000cd0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.398282 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250105088}
I20260709 13:03:06.455402  1633 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:06.461205  1633 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:06.463203  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2846) took 2266 ms. Trace:
I20260709 13:03:06.463709  1633 rpcz_store.cc:276] 0709 13:03:04.196565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.460374 (+2263809us) service_pool.cc:224] Handling call
0709 13:03:06.460747 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.462877 (+  2130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.463136 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.455367  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.467864  7070 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.462324  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2845) took 2266 ms. Trace:
I20260709 13:03:06.469355  1635 rpcz_store.cc:276] 0709 13:03:04.196239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.459014 (+2262775us) service_pool.cc:224] Handling call
0709 13:03:06.459436 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.461839 (+  2403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.461881 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.461957 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:06.462003 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.462086 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.462244 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.471009  1635 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:06.473109  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 904) took 1924 ms. Trace:
I20260709 13:03:06.471716  1554 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:06.466496  1556 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.476055  6836 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:06.475970  7090 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:06.476645  6836 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:06.452621  7195 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:06.477169  6836 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.477406  7195 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.473857  1633 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.477825  7195 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.478138  6933 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:06.480299  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74739.7us
I20260709 13:03:06.451071  7076 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:06.481128  7076 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:06.481498  7076 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.482061  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:06.482880  7070 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:06.483304  7070 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.483675  7070 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.480468  6933 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.474287  1635 rpcz_store.cc:276] 0709 13:03:04.548781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.470501 (+1921720us) service_pool.cc:224] Handling call
0709 13:03:06.470813 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.472802 (+  1989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.473044 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.475574  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.487982  1554 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.485787  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 902) took 1940 ms. Trace:
I20260709 13:03:06.489154  1633 rpcz_store.cc:276] 0709 13:03:04.545881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.472150 (+1926269us) service_pool.cc:224] Handling call
0709 13:03:06.473658 (+  1508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.484825 (+ 11167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.485723 (+   898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.491396  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.492431  2297 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:06.493909  1551 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:06.495693  2294 raft_consensus.cc:2368] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:06.498875  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:06.500133  1633 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:06.501989  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.502686  2294 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:06.504459  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2857) took 1697 ms. Trace:
I20260709 13:03:06.504844  1633 rpcz_store.cc:276] 0709 13:03:04.807251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.499627 (+1692376us) service_pool.cc:224] Handling call
0709 13:03:06.499958 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.504156 (+  4198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.504392 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.506590  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2728) took 1670 ms. Trace:
I20260709 13:03:06.506930  1633 rpcz_store.cc:276] 0709 13:03:04.836109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.505783 (+1669674us) service_pool.cc:224] Handling call
0709 13:03:06.506139 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.506303 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.506329 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.506368 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:06.506396 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.506443 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.506538 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.476161  7248 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.529839  1556 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:03:06.501607  6934 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:03:06.536232  6934 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.537496  7134 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.538475  6934 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.538622  7208 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.539376  1706 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.549242  1554 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:06.551405  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:06.551891  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:06.552932  1551 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.557344  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:06.562939  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:06.571928  1599 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:06.568884  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:06.576979  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:06.579149  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:06.579412  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:06.570215  7156 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.000s
I20260709 13:03:06.561524  7097 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.564116  7254 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.601310  7254 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.587945  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:06.590903  7153 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.591996  1524 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:06.598234  7222 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.575709  7208 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:03:06.572499  7134 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.545239  7195 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:06.570871  7186 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:06.603926  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 934) took 1939 ms. Trace:
I20260709 13:03:06.613313  1706 rpcz_store.cc:276] 0709 13:03:04.664755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.292853 (+1628098us) service_pool.cc:224] Handling call
0709 13:03:06.313749 (+ 20896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.539829 (+226080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.603625 (+ 63796us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400132ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:06.603852 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518424960}
I20260709 13:03:06.604305  1632 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:06.603931  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.020s	sys 0.023s
I20260709 13:03:06.578085  7221 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:06.579929  6910 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:06.615963  2291 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.020s	sys 0.039s
W20260709 13:03:06.619287  2291 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.020s	sys 0.040s
W20260709 13:03:06.616863  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.021s	sys 0.023s
I20260709 13:03:06.629122  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.630759  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.631489  2293 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.611269  7097 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.651875  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.652275  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.652953  2297 raft_consensus.cc:2368] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:06.657938  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.603539  6836 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.659978  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.614888  7185 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.603993  7156 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.668851  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.686328  2289 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.661674  2297 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.687107  1599 leader_election.cc:400] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:06.618160  7186 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.669942  2294 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:06.687495  1599 leader_election.cc:403] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:06.689395  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1052) took 1964 ms. Trace:
I20260709 13:03:06.618871  6910 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:06.611835  7172 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.654891  7254 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.612349  7070 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:06.702346  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:06.721196  1553 raft_consensus.cc:2410] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:06.691924  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.722280  2297 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.656594  7221 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:06.626436  7067 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:03:06.619095  6688 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:03:06.692094  2295 rpcz_store.cc:276] 0709 13:03:04.725218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.901844 (+1176626us) service_pool.cc:224] Handling call
0709 13:03:05.902133 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.902358 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:06.208470 (+306112us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.689040 (+480570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.689115 (+    75us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:06.689169 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.689231 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.689342 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":464829,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":335,"thread_start_us":758525,"threads_started":2}
I20260709 13:03:06.718816  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:06.850219  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 131464us
W20260709 13:03:06.672652  7217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.014s	sys 0.007s
W20260709 13:03:06.759157  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.661185  7190 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:06.688793  7186 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:06.847329  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:06.694226  7222 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.853533  2293 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:06.855087  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.660375  7248 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:03:06.853574  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.868793  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.858693  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:06.869521  1706 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:06.716274  6910 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.852353  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:06.848522  6953 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:06.864776  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.865446  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.859696  7254 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:06.876601  1703 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:06.661327  6915 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.156s
I20260709 13:03:06.867570  2295 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:06.867990  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.885026  7073 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.888617  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2839) took 2976 ms. Trace:
I20260709 13:03:06.888602  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:06.866873  7217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.014s	sys 0.008s
I20260709 13:03:06.891588  2293 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:06.889279  1631 rpcz_store.cc:276] 0709 13:03:03.912444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.056092 (+2143648us) service_pool.cc:224] Handling call
0709 13:03:06.056574 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.057024 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.057064 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.057642 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.500552 (+442910us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:06.500613 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.500705 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.888302 (+387597us) raft_consensus.cc:1685] finished
0709 13:03:06.888332 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.888536 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2069,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":442445,"threads_started":1,"wal-append.queue_time_us":602414}
W20260709 13:03:06.866528  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.013s	sys 0.043s
I20260709 13:03:06.867815  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:06.873521  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.881361  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:06.896373  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.043s
I20260709 13:03:06.896956  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.897601  2289 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:06.902645  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:06.903044  7002 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:06.903632  7002 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.904369  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.907019  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:06.858808  7167 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:06.910471  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:06.912479  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:06.913142  1703 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:06.917240  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:06.917886  1631 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:06.919953  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:06.922411  2293 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:03:06.926630  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 920) took 3419 ms (client timeout 1936 ms). Trace:
W20260709 13:03:06.927100  1702 rpcz_store.cc:269] 0709 13:03:03.507197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.082468 (+575271us) service_pool.cc:224] Handling call
0709 13:03:06.926406 (+2843938us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54003c1254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.926546 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165259136}
W20260709 13:03:06.928580  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2828) took 4063 ms (client timeout 1914 ms). Trace:
W20260709 13:03:06.929205  1627 rpcz_store.cc:269] 0709 13:03:02.864982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.865141 (+   159us) service_pool.cc:224] Handling call
0709 13:03:06.928299 (+4063158us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001e8054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:06.928483 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3444744960}
I20260709 13:03:06.936985  2289 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
W20260709 13:03:06.938149  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 454450us
I20260709 13:03:06.884024  7254 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.883486  6953 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:06.944242  6953 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:06.952389  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.954380  1632 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:06.915189  7153 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:06.911674  7212 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:06.965102  1703 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:06.957134  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2733) took 1923 ms. Trace:
I20260709 13:03:06.968444  1632 rpcz_store.cc:276] 0709 13:03:05.034143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.956298 (+1922155us) service_pool.cc:224] Handling call
0709 13:03:06.956619 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.956809 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.956833 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.956872 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:06.956906 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.956968 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.957076 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.970252  1632 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:06.937896  7237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.817s	user 0.016s	sys 0.009s
I20260709 13:03:06.966305  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:06.987913  1706 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:06.966545  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:06.955217  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.017s	sys 0.047s
I20260709 13:03:06.971093  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2734) took 1866 ms. Trace:
I20260709 13:03:06.957914  1631 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
W20260709 13:03:06.989231  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.018s	sys 0.047s
I20260709 13:03:06.974673  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.948721  7254 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:06.948011  7167 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:06.958828  7195 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.003729  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2737) took 1228 ms. Trace:
W20260709 13:03:06.987287  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.989974  1703 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:06.989245  1632 rpcz_store.cc:276] 0709 13:03:05.104830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.969654 (+1864824us) service_pool.cc:224] Handling call
0709 13:03:06.970018 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.970811 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.971029 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.008008  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2738) took 1118 ms. Trace:
I20260709 13:03:07.008425  1632 rpcz_store.cc:276] 0709 13:03:05.889677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.006955 (+1117278us) service_pool.cc:224] Handling call
0709 13:03:07.007348 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.007571 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.007607 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.007663 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:07.007714 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.007804 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.007939 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.990079  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 632917us
W20260709 13:03:06.990916  7263 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:03:06.960260  7153 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:06.996217  6836 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.017656  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:06.975569  7212 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:07.023102  1703 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:07.024518  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:07.017128  7183 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.812s	user 0.032s	sys 0.017s
W20260709 13:03:06.989708  7237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.869s	user 0.017s	sys 0.009s
I20260709 13:03:07.004032  7167 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:06.932489  7221 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:07.019935  7219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.037s	sys 0.035s
I20260709 13:03:07.022601  7156 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.032326  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:07.065231  1634 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:07.104498  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2864) took 1097 ms. Trace:
W20260709 13:03:07.104667  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72433.4us
I20260709 13:03:07.065671  1554 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:07.066072  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.108069  1556 raft_consensus.cc:2368] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12.
I20260709 13:03:07.105859  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:07.066221  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.110525  1551 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 12: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:07.110380  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:07.067257  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.027946  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1053) took 2297 ms. Trace:
I20260709 13:03:07.112097  1557 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:03:07.069238  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2164) took 2447 ms (client timeout 2105 ms). Trace:
W20260709 13:03:07.062609  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2108) took 2247 ms (client timeout 1506 ms). Trace:
I20260709 13:03:07.091881  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2678) took 3359 ms. Trace:
W20260709 13:03:07.117069  1560 rpcz_store.cc:269] 0709 13:03:04.815119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.818404 (+  3285us) service_pool.cc:224] Handling call
0709 13:03:07.062242 (+2243838us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54000e8654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.062456 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674047232}
I20260709 13:03:07.105022  1634 rpcz_store.cc:276] 0709 13:03:06.007114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.064716 (+1057602us) service_pool.cc:224] Handling call
0709 13:03:07.065047 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.104134 (+ 39087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.104419 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.119732  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.120774  1634 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:07.092561  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.115797  1709 rpcz_store.cc:269] 0709 13:03:04.621473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.113957 (+492484us) service_pool.cc:224] Handling call
0709 13:03:07.069032 (+1955075us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b54003dc554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.069158 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18737920}
I20260709 13:03:07.117805  1558 rpcz_store.cc:276] 0709 13:03:03.732226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.106954 (+374728us) service_pool.cc:224] Handling call
0709 13:03:04.387915 (+280961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.459157 (+ 71242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.091566 (+2632409us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54003fb950. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.091795 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645421952}
I20260709 13:03:07.101034  1706 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:07.124224  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.136914  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:07.137424  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.137892  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.138769  1558 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:07.139070  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.140049  1556 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
W20260709 13:03:07.101184  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 239482us
I20260709 13:03:07.045490  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2776) took 2933 ms. Trace:
W20260709 13:03:07.056458  7238 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.948s	user 0.005s	sys 0.012s
I20260709 13:03:07.141790  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.043640  7070 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.178606  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.187597  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.188390  1557 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:07.188685  1554 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:07.189937  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.073133  1553 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:07.191633  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:07.181468  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47532.4us
I20260709 13:03:07.192900  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.193470  1553 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.194235  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:07.094550  7167 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.196223  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:07.196399  7167 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.127009  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.198940  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.191836  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.202587  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.203544  1634 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:07.190105  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:07.053706  6934 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:07.205416  6934 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:07.206302  6934 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.206951  1554 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:07.207481  6934 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:07.049450  7002 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.212029  7002 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:07.186473  1704 rpcz_store.cc:276] 0709 13:03:04.112507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.051759 (+1939252us) service_pool.cc:224] Handling call
0709 13:03:06.060207 (+  8448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.060590 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:06.060624 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.061123 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.540591 (+479468us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:06.540638 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.540704 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.045256 (+504552us) raft_consensus.cc:1685] finished
0709 13:03:07.045278 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.045422 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33083,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":479061,"threads_started":1,"wal-append.queue_time_us":782958}
W20260709 13:03:07.093214  7126 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.001s
W20260709 13:03:07.036466  7183 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.832s	user 0.033s	sys 0.017s
I20260709 13:03:07.041232  7172 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.218277  7172 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:07.218782  7172 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:07.120597  7190 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:07.047951  7153 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.114259  7195 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:07.136508  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.223359  2292 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:07.033830  7101 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.047989  7212 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:07.047462  7261 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:03:07.077080  7128 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:07.276629  7128 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.279304  7128 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:07.126822  6836 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:07.280932  6836 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:07.281906  6836 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:07.224445  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 931) took 2560 ms (client timeout 1556 ms). Trace:
W20260709 13:03:07.284049  1707 rpcz_store.cc:269] 0709 13:03:04.664234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.809261 (+145027us) service_pool.cc:224] Handling call
0709 13:03:07.224152 (+2414891us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.224354 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951026944}
I20260709 13:03:07.284682  1552 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:07.285547  1551 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:07.285938  1552 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:07.224702  1700 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:07.288523  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2759) took 3579 ms. Trace:
I20260709 13:03:07.288952  1700 rpcz_store.cc:276] 0709 13:03:03.708885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.960412 (+2251527us) service_pool.cc:224] Handling call
0709 13:03:07.224393 (+1263981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.287453 (+ 63060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.288279 (+   826us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001347d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.288441 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2905687424}
I20260709 13:03:07.289848  7002 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:03:07.226460  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:07.291255  7002 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:07.291517  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.068825  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2836) took 3236 ms. Trace:
I20260709 13:03:07.111932  2291 rpcz_store.cc:276] 0709 13:03:04.730464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.947236 (+1216772us) service_pool.cc:224] Handling call
0709 13:03:05.947611 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.948199 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:06.620608 (+672409us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.027463 (+406855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.027544 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:07.027610 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.027701 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.027872 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":428165,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":120,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":1064621,"threads_started":2}
W20260709 13:03:07.219743  7076 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
I20260709 13:03:07.265756  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:07.265789  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 80185.3us
I20260709 13:03:07.312965  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:07.295081  1628 rpcz_store.cc:276] 0709 13:03:03.832265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.015883 (+2183618us) service_pool.cc:224] Handling call
0709 13:03:06.392708 (+376825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.393452 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.068616 (+675164us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54002972d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.068760 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500897536}
I20260709 13:03:07.302471  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:07.308279  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.183068  7237 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:07.071187  7219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.038s	sys 0.035s
I20260709 13:03:07.311829  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:07.220744  7134 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:07.215474  1553 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:07.333628  1634 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:07.221835  7254 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.221323  7190 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:07.341766  2289 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:07.344076  2296 pending_rounds.cc:85] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 10
I20260709 13:03:07.346570  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:07.350349  2296 pending_rounds.cc:107] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:03:07.353572  2296 raft_consensus.cc:2889] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:07.323854  7153 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.291738  7002 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:07.328996  7222 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:07.360633  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.342398  7226 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:07.355171  6947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.021s	sys 0.047s
W20260709 13:03:07.351183  7097 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
I20260709 13:03:07.337575  1558 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:07.005074  1629 rpcz_store.cc:276] 0709 13:03:05.775188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.002612 (+1227424us) service_pool.cc:224] Handling call
0709 13:03:07.002965 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.003334 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.003370 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.003420 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:07.003464 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.003524 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.003657 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1371}
I20260709 13:03:07.386292  1629 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:07.363754  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.349692  7067 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:07.219985  7172 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:07.328723  7126 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:03:07.389417  7002 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:07.390518  7097 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.392457  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.352012  7134 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:07.329174  7076 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:07.404956  2289 raft_consensus.cc:2368] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:07.343170  7237 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:07.382155  6947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.021s	sys 0.047s
I20260709 13:03:07.415594  1628 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:07.415716  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.349409  7271 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:03:07.455741  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.456662  2297 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:03:07.363617  7192 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:03:07.453708  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2723) took 2897 ms. Trace:
I20260709 13:03:07.359213  7265 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
W20260709 13:03:07.475519  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.015s	sys 0.019s
I20260709 13:03:07.453485  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2729) took 2617 ms. Trace:
W20260709 13:03:07.452358  7267 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:03:07.492902  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.015s	sys 0.019s
I20260709 13:03:07.484297  1630 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:07.496119  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:07.498844  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:07.500146  2289 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 13:03:07.493439  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 618116us
I20260709 13:03:07.501626  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.503810  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:07.504758  1707 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:07.502848  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.518141  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.519052  1700 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:07.520361  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.502621  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.394454  7266 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:03:07.540407  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.392510  7226 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.546976  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.547842  1700 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:07.548949  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:07.549855  1700 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:07.541919  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.545019  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.553469  1707 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:07.554817  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:07.555102  1628 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:07.559644  7276 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:03:07.414732  7194 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:03:07.545921  7182 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.542430  6947 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.580412  7134 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:07.623569  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 427366us
W20260709 13:03:07.594714  1702 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.002s
W20260709 13:03:07.600157  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58312.5us
I20260709 13:03:07.626675  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:07.627038  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.627251  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:07.627956  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.628418  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.604046  1707 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:07.605422  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.605536  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:07.629309  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:07.541018  7186 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.003s
I20260709 13:03:07.614003  7270 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:07.606365  7219 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.619863  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:07.675480  2297 raft_consensus.cc:2368] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
W20260709 13:03:07.619812  1675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.009s
W20260709 13:03:07.676299  1675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.009s
W20260709 13:03:07.676668  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 600442us
W20260709 13:03:07.677189  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.459700  7190 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:07.630436  1707 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:07.617151  7195 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:07.635231  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 258588us
W20260709 13:03:07.638522  1631 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.042s	sys 0.058s
I20260709 13:03:07.742391  1707 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:07.742827  1631 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.042s	sys 0.058s
W20260709 13:03:07.615384  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 446655us
I20260709 13:03:07.633028  7222 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:07.677546  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.585363  7002 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:07.722116  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.626426  6947 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.633126  7141 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:07.745410  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.688830  7182 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:07.630836  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:07.766445  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 135671us
W20260709 13:03:07.686276  7147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.011s	sys 0.015s
W20260709 13:03:07.752110  1675 connection.cc:256] RPC call timeout handler was delayed by -2.305235907999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
W20260709 13:03:07.768448  1675 connection.cc:256] RPC call timeout handler was delayed by -2.2923540059999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 13:03:07.775233  1629 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:07.781024  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.781888  1629 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:07.677773  7186 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.793794  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.679680  7270 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:07.779490  7147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.011s	sys 0.016s
I20260709 13:03:07.765043  6947 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.764315  7097 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:07.806828  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.814862  1701 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:07.801131  7254 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:07.820792  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.821623  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:07.830363  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:07.841763  2297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:03:07.842545  2297 raft_consensus.cc:606] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:07.842962  2297 raft_consensus.cc:614] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:07.806589  7067 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:07.822460  7076 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:07.849489  7212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.012s
I20260709 13:03:07.832110  7237 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:07.870242  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 102394us
I20260709 13:03:07.831322  7295 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:07.839895  7266 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:07.879437  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2741) took 1898 ms. Trace:
W20260709 13:03:07.870033  7212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.009s	sys 0.013s
W20260709 13:03:07.872717  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 244344us
I20260709 13:03:07.886287  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.892395  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.884290  1632 rpcz_store.cc:276] 0709 13:03:05.980892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.009541 (+1028649us) service_pool.cc:224] Handling call
0709 13:03:07.009905 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.010169 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:07.490880 (+480711us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.879071 (+388191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.879130 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:07.879181 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.879252 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.879378 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83418,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":86539,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":2393,"spinlock_wait_cycles":7837312,"thread_start_us":852121,"threads_started":3,"wal-append.queue_time_us":358858}
I20260709 13:03:07.904316  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.906002  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.908128  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.910482  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.912698  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:07.889904  7182 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.876726  7266 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:07.840585  7195 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.920769  7156 raft_consensus.cc:2764] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:03:07.946333  7260 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.941s	user 0.005s	sys 0.000s
W20260709 13:03:07.947206  7302 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:07.948359  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2761) took 4217 ms. Trace:
I20260709 13:03:07.948882  7270 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:07.963708  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:07.964206  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51586us
I20260709 13:03:07.883149  7226 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:07.878778  7076 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:03:07.934554  7281 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:03:07.935479  7297 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:03:07.948881  1705 rpcz_store.cc:276] 0709 13:03:03.730857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.992882 (+2262025us) service_pool.cc:224] Handling call
0709 13:03:06.014476 (+ 21594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.014934 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.014980 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.046405 (+ 31425us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.621428 (+575023us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.621489 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.621597 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.945461 (+1323864us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:07.947822 (+  2361us) raft_consensus.cc:1685] finished
0709 13:03:07.947852 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.948274 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":610,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":156817,"spinlock_wait_cycles":46845312,"thread_start_us":574560,"threads_started":1,"wal-append.queue_time_us":900609}
W20260709 13:03:07.932463  7291 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:07.973904  7182 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.974520  7266 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.978091  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:07.978955  7089 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:07.979550  2290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1266) took 3387 ms (client timeout 2209 ms). Trace:
I20260709 13:03:07.980135  6933 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:07.980695  6934 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.773s	user 0.008s	sys 0.003s
I20260709 13:03:07.492964  1633 rpcz_store.cc:276] 0709 13:03:04.836435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.508834 (+1672399us) service_pool.cc:224] Handling call
0709 13:03:06.509218 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.509454 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:06.966644 (+457190us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.453079 (+486435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.453138 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:07.453193 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.453280 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.453419 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":583700,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":50134,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":24406,"thread_start_us":839972,"threads_started":2}
I20260709 13:03:08.038306  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.980966  7067 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:07.483487  1635 rpcz_store.cc:276] 0709 13:03:04.556372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.486734 (+1930362us) service_pool.cc:224] Handling call
0709 13:03:06.487060 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.487283 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:06.971247 (+483964us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.453209 (+481962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.453290 (+    81us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:07.453372 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.453471 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.453631 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44483,"prepare.queue_time_us":605643,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":75073,"thread_start_us":918422,"threads_started":2}
I20260709 13:03:08.006114  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:07.921006  7255 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:08.010500  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2861) took 2698 ms (client timeout 1673 ms). Trace:
W20260709 13:03:08.042567  1626 rpcz_store.cc:269] 0709 13:03:05.312133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.775518 (+463385us) service_pool.cc:224] Handling call
0709 13:03:08.010259 (+2234741us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400297cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:08.010422 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333856128}
I20260709 13:03:08.044221  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:08.015515  2290 rpcz_store.cc:269] 0709 13:03:04.592237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.160453 (+568216us) service_pool.cc:224] Handling call
0709 13:03:07.968583 (+2808130us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540073c3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:07.979470 (+ 10887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351944960}
W20260709 13:03:08.017788  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1061) took 1644 ms (client timeout 1556 ms). Trace:
W20260709 13:03:08.045845  2292 rpcz_store.cc:269] 0709 13:03:06.373014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.379076 (+  6062us) service_pool.cc:224] Handling call
0709 13:03:08.017558 (+1638482us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:08.017702 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939373440}
I20260709 13:03:08.024303  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:08.025202  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.025674  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:08.025935  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.048527  1556 raft_consensus.cc:2368] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:08.026665  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.049041  1633 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:08.049177  1560 raft_consensus.cc:2376] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20. Re-sending same reply.
I20260709 13:03:08.027441  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.027840  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:08.050503  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.050501  1555 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.050712  2293 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:08.051175  1626 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:08.051266  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:08.030903  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.052307  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.052539  1553 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:08.041638  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.053509  1705 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.053928  1558 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:08.053224  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2858) took 2869 ms. Trace:
I20260709 13:03:08.053542  2295 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.054652  1631 rpcz_store.cc:276] 0709 13:03:05.184280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.016365 (+1832085us) service_pool.cc:224] Handling call
0709 13:03:07.016727 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.016932 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:07.749276 (+732344us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.052626 (+303350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.052679 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:08.052728 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.053063 (+   335us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.053171 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35086,"prepare.queue_time_us":76857,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"thread_start_us":974057,"threads_started":2}
I20260709 13:03:08.057871  1707 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:08.052888  1635 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:08.060916  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.061787  1635 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:08.057883  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:08.065927  1626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:08.066738  1626 raft_consensus.cc:606] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:08.067323  1626 raft_consensus.cc:614] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:08.068186  1633 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:08.066391  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:08.097708  1555 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:08.099754  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:08.114602  1626 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:08.034289  6933 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:08.105270  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.110164  2295 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:08.057971  7266 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.132455  1632 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:08.064198  7097 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.099823  7186 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:08.156747  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 151617us
I20260709 13:03:08.163113  1705 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:08.164244  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2732) took 3326 ms. Trace:
I20260709 13:03:08.164929  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:08.105746  7221 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:03:08.165069  1627 rpcz_store.cc:276] 0709 13:03:04.837700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.933929 (+2096229us) service_pool.cc:224] Handling call
0709 13:03:06.934257 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.934453 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:07.458271 (+523818us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.134576 (+676305us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.163936 (+ 29360us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.163988 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:08.164033 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.164098 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.164189 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29028,"prepare.queue_time_us":166734,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":99680,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":5819,"thread_start_us":1165942,"threads_started":3,"wal-append.queue_time_us":415248}
I20260709 13:03:08.170977  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:08.179622  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:08.182103  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.185309  1560 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:08.186239  1559 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:08.187660  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:08.192111  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:08.189323  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:08.195067  7153 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.006s
I20260709 13:03:08.195616  7153 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.196199  7153 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.198272  7153 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:08.198304  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2842) took 4108 ms. Trace:
I20260709 13:03:08.198870  1630 rpcz_store.cc:276] 0709 13:03:04.089625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.468575 (+2378950us) service_pool.cc:224] Handling call
0709 13:03:06.469000 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.503199 (+1034199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.197996 (+694797us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000de3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:08.198228 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334457088}
I20260709 13:03:08.200263  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:08.202214  7303 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:08.203471  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.203812  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:08.204213  7310 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:03:08.204849  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2707) took 1138 ms. Trace:
I20260709 13:03:08.205235  1557 rpcz_store.cc:276] 0709 13:03:07.066832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.200664 (+133832us) service_pool.cc:224] Handling call
0709 13:03:07.201075 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.199268 (+998193us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:08.199300 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.199341 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:08.199371 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.204603 (+  5232us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54001379d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:08.204646 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.204789 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294922624}
W20260709 13:03:08.206524  7305 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:08.172472  7134 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:08.210127  6933 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.211285  6933 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.165484  7101 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:08.284050  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 61635.3us
I20260709 13:03:08.157642  7097 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:08.313822  2297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:03:08.316071  2297 raft_consensus.cc:606] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:08.316439  2297 raft_consensus.cc:614] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:08.319341  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 130100us
I20260709 13:03:08.287317  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:08.321640  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 121454us
W20260709 13:03:08.203488  7300 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:03:08.323597  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:08.321740  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 129721us
I20260709 13:03:08.327201  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:08.328651  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:08.330598  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:08.328398  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:08.330717  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:08.336179  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:08.339078  1524 leader_election.cc:400] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:08.339464  1524 leader_election.cc:403] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:08.339632  1630 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:08.340461  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:08.341306  1601 leader_election.cc:400] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14.
I20260709 13:03:08.343283  7322 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:08.344476  7071 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:08.344894  7071 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:08.345814  7071 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.349606  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:08.342694  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:08.327919  7070 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:08.359586  1524 leader_election.cc:400] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:08.332944  7098 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:08.331992  7237 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:08.366346  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:08.328267  7134 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:08.337989  7226 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:08.340018  7270 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:08.353201  7222 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:08.320662  6947 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.352275  7221 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:08.371415  7090 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.355986  7186 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:08.357848  7067 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:08.370853  7254 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:08.462551  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.471401  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2778) took 4287 ms. Trace:
I20260709 13:03:08.472002  1708 rpcz_store.cc:276] 0709 13:03:04.184202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.178881 (+1994679us) service_pool.cc:224] Handling call
0709 13:03:06.192173 (+ 13292us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:06.192679 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.192721 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:06.200908 (+  8187us) log.cc:844] Serialized 78 byte log entry
0709 13:03:06.897045 (+696137us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:07.355185 (+458140us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.374403 (+ 19218us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.374945 (+   542us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.449152 (+1074207us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b1800481ea0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:03:08.449980 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.471029 (+ 21049us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.471104 (+    75us) raft_consensus.cc:1685] finished
0709 13:03:08.471123 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.471326 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2131794,"prepare.queue_time_us":492404,"raft.queue_time_us":1086336,"raft.run_cpu_time_us":688,"raft.run_wall_time_us":45430,"spinlock_wait_cycles":21614464,"thread_start_us":1012576,"threads_started":2,"wal-append.queue_time_us":785193}
I20260709 13:03:08.377121  7237 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:08.456089  7300 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:03:08.388932  7333 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:08.489564  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 129040us
I20260709 13:03:08.485373  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:08.463856  7295 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 13:03:08.495071  1708 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:08.492416  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:08.455657  7098 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:08.502197  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.387428  7002 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
I20260709 13:03:08.462729  7222 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:08.462432  7186 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:08.480942  7237 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:08.509451  7306 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.194s
W20260709 13:03:08.509881  7325 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:03:08.488528  7221 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:08.493992  7254 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:08.499428  7295 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:08.521749  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 141298us
I20260709 13:03:08.470716  7090 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.510124  7308 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.189s
I20260709 13:03:08.506024  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:08.514906  6910 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.632s	user 0.015s	sys 0.001s
I20260709 13:03:08.506513  7182 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:08.517608  7186 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:08.541262  7315 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:08.542050  7315 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.543349  7337 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:08.544159  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:08.627004  7337 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:08.627616  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83538.6us
I20260709 13:03:08.628337  7337 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.629426  7337 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:08.630151  7298 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:08.630550  7298 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:08.543963  7098 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:08.631434  7298 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.634070  6947 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:08.544814  7332 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:08.635517  7192 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:08.636165  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:08.636261  7192 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:08.636783  2292 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.636911  1552 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:08.545207  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2781) took 3838 ms. Trace:
I20260709 13:03:08.637876  1701 rpcz_store.cc:276] 0709 13:03:04.706731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.291060 (+1584329us) service_pool.cc:224] Handling call
0709 13:03:06.301169 (+ 10109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.815298 (+1514129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.544921 (+729623us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:08.545139 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3480527232}
I20260709 13:03:08.639159  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.639792  1701 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:08.545818  7323 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:03:08.546649  7318 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:03:08.547375  7330 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:03:08.551977  7222 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:08.552770  7182 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.553972  6910 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.555996  7327 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.556550  2297 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:08.556919  7299 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.629s	user 0.004s	sys 0.004s
I20260709 13:03:08.557677  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:08.564677  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:08.571955  7317 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:03:08.573077  7313 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.577364  2293 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:03:08.580646  1629 raft_consensus.cc:2410] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:08.580843  7315 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:08.586622  7254 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.499382  7333 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.511073  7134 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:08.589898  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.702119  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.702556  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.591324  7126 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:03:08.593938  7268 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.610s	user 0.005s	sys 0.010s
I20260709 13:03:08.594684  7156 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:08.599892  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.735237  1708 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:08.697108  7312 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:03:08.602659  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.600657  7186 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.746953  2292 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:08.603973  1631 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:08.603672  7306 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:08.603682  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2744) took 1518 ms. Trace:
I20260709 13:03:08.751457  1628 rpcz_store.cc:276] 0709 13:03:07.085011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.349292 (+264281us) service_pool.cc:224] Handling call
0709 13:03:08.603509 (+1254217us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b54002986d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:08.603624 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65481472}
I20260709 13:03:08.605821  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.606796  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.753172  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2186) took 1006 ms. Trace:
I20260709 13:03:08.753516  1707 rpcz_store.cc:276] 0709 13:03:07.746845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.212923 (+466078us) service_pool.cc:224] Handling call
0709 13:03:08.606660 (+393737us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:03:08.753102 (+146442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16966016}
I20260709 13:03:08.754704  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.755246  1707 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:08.615507  6836 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.615727  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:08.762741  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 147091us
I20260709 13:03:08.616134  7212 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.618633  6648 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:08.735000  7156 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:08.620709  1627 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:08.623521  1633 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:08.632804  7298 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:08.647892  2288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
W20260709 13:03:08.695590  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.699874  1634 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:08.704196  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:08.708380  2295 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:08.709851  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:08.713266  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.797068  1628 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:08.798331  2288 raft_consensus.cc:606] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:08.798729  2288 raft_consensus.cc:614] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:08.731482  7190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.001s	sys 0.025s
W20260709 13:03:08.732529  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67492.2us
I20260709 13:03:08.601572  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:08.698307  7097 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:08.744556  2289 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:08.746210  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.746018  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:08.759236  7270 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:08.812153  1708 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:08.816370  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:08.816733  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2188) took 1069 ms. Trace:
I20260709 13:03:08.817126  1708 rpcz_store.cc:276] 0709 13:03:07.747392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.745957 (+998565us) service_pool.cc:224] Handling call
0709 13:03:08.816668 (+ 70711us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.818172  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.814767  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.819612  1705 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.711913  7126 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:08.533694  7002 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:08.766191  7332 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:08.822574   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:08.822902  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2196) took 1058 ms. Trace:
I20260709 13:03:08.823307  1708 rpcz_store.cc:276] 0709 13:03:07.764129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.817861 (+1053732us) service_pool.cc:224] Handling call
0709 13:03:08.822827 (+  4966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.824330  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.799562  2294 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
W20260709 13:03:08.807667  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 206163us
I20260709 13:03:08.826282  1707 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:08.827339  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2199) took 1062 ms. Trace:
I20260709 13:03:08.827508  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:08.827807  1707 rpcz_store.cc:276] 0709 13:03:07.764905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.754454 (+989549us) service_pool.cc:224] Handling call
0709 13:03:08.827263 (+ 72809us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.689376  6933 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:03:08.812578  1632 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:08.829099  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:08.836019  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:08.829787  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.767297  6836 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.776599  6648 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:08.856326  1705 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:03:08.819165  7190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.001s	sys 0.026s
I20260709 13:03:08.838014  1707 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.838384  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1263) took 4359 ms. Trace:
I20260709 13:03:08.839622  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.848093  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:08.844262  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:08.865845  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2198) took 1101 ms. Trace:
I20260709 13:03:08.877979  2289 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:08.878046  1705 rpcz_store.cc:276] 0709 13:03:07.764654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.780377 (+1015723us) service_pool.cc:224] Handling call
0709 13:03:08.865766 (+ 85389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.871055  1558 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:08.811810  7324 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:08.879011  1708 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:03:08.875902  2296 rpcz_store.cc:276] 0709 13:03:04.479303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.839198 (+1359895us) service_pool.cc:224] Handling call
0709 13:03:05.839654 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.375504 (+1535850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.375551 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.388787 (+ 13236us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.754691 (+365904us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:07.754775 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.034351 (+279576us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54006c02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:08.034423 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.556156 (+521733us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:08.838168 (+282012us) raft_consensus.cc:1685] finished
0709 13:03:08.838189 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.838325 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16665,"raft.queue_time_us":17682,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14820,"spinlock_wait_cycles":3467937408,"thread_start_us":363592,"threads_started":1,"wal-append.queue_time_us":465505}
I20260709 13:03:08.778299  7212 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.811252  7295 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:03:08.841024  7153 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:08.852268  7315 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:08.928578  1707 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:08.941242  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2191) took 1193 ms. Trace:
I20260709 13:03:08.941777  1707 rpcz_store.cc:276] 0709 13:03:07.748287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.829453 (+1081166us) service_pool.cc:224] Handling call
0709 13:03:08.941158 (+111705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.843585  7070 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:08.842330  6836 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:08.945544  6836 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:08.946077  6836 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:08.946611  6836 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.933703  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2193) took 1172 ms. Trace:
I20260709 13:03:08.935089  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.948748  1708 rpcz_store.cc:276] 0709 13:03:07.761582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.839344 (+1077762us) service_pool.cc:224] Handling call
0709 13:03:08.933599 (+ 94255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.949635  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:08.950233  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2189) took 1202 ms. Trace:
I20260709 13:03:08.950628  1708 rpcz_store.cc:276] 0709 13:03:07.747697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.949420 (+1201723us) service_pool.cc:224] Handling call
0709 13:03:08.949749 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.949932 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.949954 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.949997 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:08.950034 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.950085 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.950181 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21120}
I20260709 13:03:08.951227  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2195) took 1187 ms. Trace:
I20260709 13:03:08.800374  7339 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:03:08.862576  7332 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:08.936736  1701 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:08.836453  7089 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:08.944293  7311 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.944352  7346 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:08.920037  7156 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:08.951594  1705 rpcz_store.cc:276] 0709 13:03:07.763837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.950457 (+1186620us) service_pool.cc:224] Handling call
0709 13:03:08.950762 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.950929 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.950954 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.951005 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.951042 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.951089 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.951180 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.952883  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:08.964354  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2200) took 1199 ms. Trace:
I20260709 13:03:08.974622  1707 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:08.974325  1701 rpcz_store.cc:276] 0709 13:03:07.765187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.638869 (+873682us) service_pool.cc:224] Handling call
0709 13:03:08.964279 (+325410us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:08.970433  7340 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:03:08.984612  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2187) took 1237 ms. Trace:
I20260709 13:03:08.990778  1707 rpcz_store.cc:276] 0709 13:03:07.747101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.952202 (+1205101us) service_pool.cc:224] Handling call
0709 13:03:08.984509 (+ 32307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884992}
I20260709 13:03:08.882349  7265 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:08.844309  6648 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:08.962738  7070 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.963699  7153 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:08.843358  7320 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.125s
I20260709 13:03:08.968200  7089 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:09.013401  7345 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:03:09.013566  7153 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.005874  6648 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.004339  7070 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.015853  7332 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.018834  7333 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.020689  7186 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.022794  6910 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.024221  7070 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.004321  7265 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.050300  7265 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.056100  7212 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.001678  7311 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:09.028642  7342 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:03:09.060801  7355 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:03:09.067482  7311 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:09.067574  7322 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.000s
I20260709 13:03:09.070178  7322 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:09.109503  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 254653us
W20260709 13:03:09.110070  1523 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.958s (SENT)
W20260709 13:03:09.111436  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 267096us
I20260709 13:03:08.971853  1554 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:09.115842  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:09.119131  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:09.122056  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:09.133143  2260 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.516s (SENT) [suppressed 46 similar messages]
W20260709 13:03:09.135032  2260 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.556s (SENT)
I20260709 13:03:09.008751  7089 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.149294  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 926) took 1166 ms. Trace:
I20260709 13:03:09.152545  1631 rpcz_store.cc:276] 0709 13:03:07.982390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.140856 (+158466us) service_pool.cc:224] Handling call
0709 13:03:08.603802 (+462946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.748736 (+144934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.148502 (+399766us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54002986d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:09.148949 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063655168}
I20260709 13:03:09.137573  7070 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:09.136144  6648 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:09.135736  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:09.168697  6648 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:09.179594  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:09.181068  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:09.181900  7311 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:09.175385  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:09.166651  7070 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:09.136137  7186 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.172708  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:09.201409  7337 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.202358  7337 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.205062  7090 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.177416  6648 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.229735  2295 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:09.231712  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106924us
I20260709 13:03:09.222344  7194 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:09.236034  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:09.236793  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:09.200573  7070 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:09.253010  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:09.254070  2290 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:09.257020  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 909) took 1480 ms. Trace:
I20260709 13:03:09.257608  1634 rpcz_store.cc:276] 0709 13:03:07.776715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.977775 (+201060us) service_pool.cc:224] Handling call
0709 13:03:09.256458 (+1278683us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:09.256725 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579251456}
I20260709 13:03:09.258842  1553 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:09.252974  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:09.257110  7153 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:03:09.261627  2297 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:09.261965  7182 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.265221  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:09.266537  7271 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:09.268416  1675 connection.cc:256] RPC call timeout handler was delayed by -2.219454006000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:03:09.269351  1675 connection.cc:256] RPC call timeout handler was delayed by -2.0268540060000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 13:03:09.269771  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.7681e+06us
I20260709 13:03:09.271188  1673 leader_election.cc:400] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:09.271284  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 944) took 1764 ms. Trace:
I20260709 13:03:09.273095  7271 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:09.273582  7271 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.271921  1702 rpcz_store.cc:276] 0709 13:03:07.506815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.629166 (+122351us) service_pool.cc:224] Handling call
0709 13:03:07.686624 (+ 57458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.686885 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:07.750041 (+ 63156us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.270250 (+1520209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.270403 (+   153us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:09.270562 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.270725 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.270920 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1392481,"prepare.queue_time_us":78698,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":6550,"thread_start_us":127064,"threads_started":1,"wal-append.queue_time_us":226860,"wal-append.run_cpu_time_us":1031,"wal-append.run_wall_time_us":1007570}
W20260709 13:03:09.269800  1675 connection.cc:256] RPC call timeout handler was delayed by -2.0205540059999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 13:03:09.274705  1675 connection.cc:256] RPC call timeout handler was delayed by -1.9687581920000525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 13:03:09.275120  1675 connection.cc:256] RPC call timeout handler was delayed by -1.9020618440000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:03:09.275475  1675 connection.cc:256] RPC call timeout handler was delayed by -1.8207540060000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:03:09.275660  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 1.76609e+06us
I20260709 13:03:09.271554  1673 leader_election.cc:403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:09.273941  7271 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:09.295511  1675 connection.cc:256] RPC call timeout handler was delayed by -1.8207540060000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:03:09.296087  1675 connection.cc:256] RPC call timeout handler was delayed by -1.5957154080000464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 13:03:09.271634  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 910) took 1494 ms (client timeout 1640 ms). Trace:
W20260709 13:03:09.302351  1675 connection.cc:256] RPC call timeout handler was delayed by -1.4445213070000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260709 13:03:09.307287  1675 connection.cc:256] RPC call timeout handler was delayed by -1.3390425810000579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:03:09.308022  1675 connection.cc:256] RPC call timeout handler was delayed by -1.2934367830000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 13:03:09.310292  1675 connection.cc:256] RPC call timeout handler was delayed by -1.1897271889999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 13:03:09.300487  1629 rpcz_store.cc:269] 0709 13:03:07.776961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.793558 (+ 16597us) service_pool.cc:224] Handling call
0709 13:03:09.271247 (+1477689us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400274cd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:09.271473 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784673664}
W20260709 13:03:09.313585  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62362.7us
W20260709 13:03:09.334023  7351 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
W20260709 13:03:09.336848  7347 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
W20260709 13:03:09.338402  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.340521  6983 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.341462  7073 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:09.340862  7126 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.312274  7337 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.340521  7147 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:03:09.365097  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:09.366565  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.368095  6983 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:09.369598  7361 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
W20260709 13:03:09.377381  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 242463us
I20260709 13:03:09.380626  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2798) took 2516 ms. Trace:
W20260709 13:03:09.381629  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 110704us
I20260709 13:03:09.383219  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:09.387895  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.388155  7167 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
W20260709 13:03:09.388372  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74750.5us
I20260709 13:03:09.389421  7147 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:09.390367  7344 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.421s	user 0.006s	sys 0.000s
I20260709 13:03:09.391881  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:09.398902  1553 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:09.351684  7073 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:09.418191  2295 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
W20260709 13:03:09.424589  1523 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.221s (SENT)
I20260709 13:03:09.381191  1704 rpcz_store.cc:276] 0709 13:03:06.864225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.215675 (+351450us) service_pool.cc:224] Handling call
0709 13:03:07.216153 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.252071 (+ 35918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.252113 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:07.329412 (+ 77299us) log.cc:844] Serialized 26 byte log entry
0709 13:03:07.624733 (+295321us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:07.681266 (+ 56533us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.379903 (+1698637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.380113 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.380193 (+    80us) raft_consensus.cc:1685] finished
0709 13:03:09.380210 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.380411 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1621703,"prepare.queue_time_us":56952,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":1734128,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":9703,"thread_start_us":293543,"threads_started":2,"wal-append.queue_time_us":448516,"wal-append.run_cpu_time_us":7346,"wal-append.run_wall_time_us":1411911}
I20260709 13:03:09.425293  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:09.425845  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:09.420552  7073 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:09.433627  7298 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.010s
I20260709 13:03:09.360527  7335 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:09.444144  7335 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:09.445425  7335 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.447172  7335 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:09.448302  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 65230.7us
W20260709 13:03:09.452217  6836 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.010s
I20260709 13:03:09.452775  6836 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.455164  6836 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.459899  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:09.460127  7365 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.360772  7153 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:03:09.464352  7167 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:09.464835  7167 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.465256  7167 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:09.465672  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 109506us
I20260709 13:03:09.466598  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:09.468221  7337 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:09.469301  7337 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:09.469686  7337 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:09.469731  7182 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:09.470229  7337 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.470306  7182 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:09.472437  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110272us
I20260709 13:03:09.473435  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:09.464936  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.371757  6983 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:09.478018  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86362.3us
I20260709 13:03:09.398294  7147 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:09.473569  7071 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:09.480870  7248 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:03:09.507304  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:09.516909  7141 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.518538  7147 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.520437  7147 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:09.526260  1555 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:09.526525  7364 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:03:09.527894  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:09.528188  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 947) took 2020 ms. Trace:
I20260709 13:03:09.528846  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:09.530030  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:09.530197  7098 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:09.530617  7098 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:09.531016  7098 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:09.531253  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 178203us
I20260709 13:03:09.532192  7295 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:09.532676  7295 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.533090  7295 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:09.536970  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 112475us
I20260709 13:03:09.528210  7279 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:09.540886  7290 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:09.523535  7365 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.544310  7365 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.544757  7365 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.551314  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:09.554369  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:09.555168  1556 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:09.556572  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:09.556680  1558 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:09.557375  1552 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.560225  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:09.556582  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:09.508258  7128 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:09.556680  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48792.5us
I20260709 13:03:09.562330  7128 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.563812  7248 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:09.560684  1555 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:03:09.573357  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.471571  7182 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:09.493662  7271 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.531016  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:09.584757  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 102383us
I20260709 13:03:09.535265  7073 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.572571  6983 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:09.593699  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:09.594424  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:09.592065  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 118878us
I20260709 13:03:09.528648  1700 rpcz_store.cc:276] 0709 13:03:07.507586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.634046 (+126460us) service_pool.cc:224] Handling call
0709 13:03:07.682509 (+ 48463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.268856 (+1586347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.268908 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.269762 (+   854us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.313804 (+ 44042us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:09.313845 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.320076 (+  6231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.527732 (+207656us) raft_consensus.cc:1685] finished
0709 13:03:09.527752 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.527985 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1585566,"raft.queue_time_us":1659205,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":204,"thread_start_us":43540,"threads_started":1,"wal-append.queue_time_us":46447}
W20260709 13:03:09.588061  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57145.1us
I20260709 13:03:09.602301  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2784) took 2634 ms. Trace:
I20260709 13:03:09.603133  1634 pending_rounds.cc:85] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 9
I20260709 13:03:09.603401  1555 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:09.603593  1634 pending_rounds.cc:107] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:09.604014  1634 raft_consensus.cc:2889] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.605530  1626 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:09.605964  7290 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:09.602799  1703 rpcz_store.cc:276] 0709 13:03:06.861850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.026899 (+165049us) service_pool.cc:224] Handling call
0709 13:03:07.034323 (+  7424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.034609 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:07.476245 (+441636us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.358082 (+1881837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.399931 (+ 41849us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:09.449802 (+ 49871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.481917 (+ 32115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.495745 (+ 13828us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1399899,"prepare.queue_time_us":508341,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":1485767,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"thread_start_us":876229,"threads_started":3,"wal-append.queue_time_us":637943,"wal-append.run_cpu_time_us":1488,"wal-append.run_wall_time_us":1075546}
W20260709 13:03:09.609797  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.610188  7366 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:09.605759  7248 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:09.609948  7254 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:09.607676  1556 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:09.560240  7371 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:09.588792  6983 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:09.609036  7070 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:09.624356  7070 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.624264  7002 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:09.625025  7070 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:09.628218  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.626633  1630 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:09.632808  1626 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:09.613009  7366 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:09.630501  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:09.636685  7279 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:09.626995  6983 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.667192  7182 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.668308  2289 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:09.672915  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:09.673243  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:09.674535  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:09.680822  7237 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.142s	user 0.010s	sys 0.015s
I20260709 13:03:09.681339  7237 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.684315  7371 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:09.685140  7371 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:09.685879  7313 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.686419  7313 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.688697  7366 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.692863  7073 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.696893  7271 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:09.704231  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74028us
I20260709 13:03:09.705087  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:09.633737  7090 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:03:09.708173  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45353.3us
I20260709 13:03:09.721815  2288 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:09.703606  7089 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.000s
I20260709 13:03:09.683575  7237 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.749264  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:09.735576  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 141292us
W20260709 13:03:09.725760  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 125398us
I20260709 13:03:09.666016  7279 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.755015  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2785) took 2721 ms. Trace:
I20260709 13:03:09.753055  2293 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:09.763306  1552 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:09.742343  7071 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:03:09.778898  7221 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 13:03:09.779660  7375 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:03:09.776563  7254 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.780606  7254 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.781016  7254 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.779937  7316 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.781682  7316 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:09.749481  7153 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.733597  7337 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.745010  7248 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:09.760113  1706 rpcz_store.cc:276] 0709 13:03:06.862009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.136968 (+274959us) service_pool.cc:224] Handling call
0709 13:03:07.137295 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.137496 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:07.606470 (+468974us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.422427 (+1815957us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.478480 (+ 56053us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.501433 (+ 22953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.505121 (+  3688us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.582969 (+ 77848us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1250810,"prepare.queue_time_us":434516,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":1354550,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":944845,"threads_started":3,"wal-append.queue_time_us":464282,"wal-append.run_cpu_time_us":1736,"wal-append.run_wall_time_us":1095753}
I20260709 13:03:09.740754  7089 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.783103  7316 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.801862  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:09.809156  1631 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:09.811338  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.635921  7377 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:09.815244  7377 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:09.815632  7377 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:09.669397  7101 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:03:09.816839  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.818059  7101 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.816829  7377 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.818787  7101 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.819752  7377 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.820833  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:09.822184  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 101129us
I20260709 13:03:09.798923  7002 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.813043  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.811561  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:09.820438  7221 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.810500  7308 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:09.795706  7073 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:03:09.838937  1630 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:09.849280  1626 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:03:09.852527  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.805009  7153 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.831202  7221 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:09.870482  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164288us
I20260709 13:03:09.838685  7365 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.841214  7295 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.874471  1702 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:03:09.875617  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67628.8us
I20260709 13:03:09.843626  7002 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.878610  7378 raft_consensus.cc:2764] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:09.882014  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:09.883404  7383 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:09.885671  7383 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:09.886066  7383 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:09.886406  7383 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.888978  7153 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.889813  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.905926  7271 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.906280  7101 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.906601  7271 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:09.910279  7002 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.910583  7128 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.912801  7221 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.922022  1705 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:09.922941  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.923352  1706 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:09.923623  1707 raft_consensus.cc:2376] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 16. Re-sending same reply.
I20260709 13:03:09.926656  7126 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.929492  7365 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:09.933091  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.935288  7295 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.936271  7248 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:09.937572  7343 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.084s	user 0.005s	sys 0.000s
W20260709 13:03:09.945235  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.873353  7337 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:09.955132  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.955927  1707 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:09.956768  1708 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:09.958137  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.958832  1705 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.961871  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2793) took 3098 ms. Trace:
I20260709 13:03:09.964340  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 930) took 1363 ms. Trace:
I20260709 13:03:09.964735  1635 rpcz_store.cc:276] 0709 13:03:08.601241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.601356 (+   115us) service_pool.cc:224] Handling call
0709 13:03:08.601708 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.602423 (+   715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.602457 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.602932 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.722453 (+119521us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:08.722517 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.722622 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.936710 (+1214088us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:09.939953 (+  3243us) raft_consensus.cc:1685] finished
0709 13:03:09.939979 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.964165 (+ 24186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":6281600,"thread_start_us":119079,"threads_started":1,"wal-append.queue_time_us":164208}
I20260709 13:03:09.966774  7089 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.969630  7381 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:09.970112  7381 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:09.976465  1708 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:09.976670  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.977401  1700 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:09.977626  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.978238  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.978832  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.979343  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:09.971079  7381 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:09.962343  1709 rpcz_store.cc:276] 0709 13:03:06.863213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.183144 (+319931us) service_pool.cc:224] Handling call
0709 13:03:07.265226 (+ 82082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.280101 (+ 14875us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:07.604001 (+323900us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.419818 (+1815817us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.419934 (+   116us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:09.420068 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.961470 (+541402us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54001342d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:09.961516 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.961663 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1435460,"prepare.queue_time_us":292165,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":1531683,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"spinlock_wait_cycles":33591168,"thread_start_us":681438,"threads_started":3,"wal-append.queue_time_us":324469,"wal-append.run_cpu_time_us":2153,"wal-append.run_wall_time_us":1254803}
I20260709 13:03:09.982154  7271 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:09.984319  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:09.986240  7098 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:09.983579  7335 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.006s
W20260709 13:03:09.996198  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46755.1us
W20260709 13:03:09.999929  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 118022us
I20260709 13:03:09.947278  7221 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.003963  1706 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:09.975406  7070 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.016012  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.017340  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.018663  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.023334  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.034787  1705 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:10.038839  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.039738  1633 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:10.035876  7270 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.041080  7270 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.041958  1700 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:10.043151  1700 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:10.043467  1700 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:10.044461  1700 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.045653  1700 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:10.047252  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.054591  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:10.034478  1709 raft_consensus.cc:2376] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 16. Re-sending same reply.
I20260709 13:03:10.055860  7128 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.056990  7167 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
I20260709 13:03:10.056453  7128 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.057446  7167 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.059417  1703 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:10.063198  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.064284  1629 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:10.064743  1703 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:10.061815  7128 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:10.065963  7333 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:10.067154  7255 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:10.067529  7255 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:10.068375  7255 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.069710  7255 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.069525  7221 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.073076  7098 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:10.073849  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.076758  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1258) took 1373 ms. Trace:
I20260709 13:03:10.077178  2293 rpcz_store.cc:276] 0709 13:03:08.703657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.703840 (+   183us) service_pool.cc:224] Handling call
0709 13:03:10.076524 (+1372684us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:10.076680 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889016704}
I20260709 13:03:10.080068  7369 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:10.080513  7369 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:10.081571  7369 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.082875  7369 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:10.084365  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.085238  1706 raft_consensus.cc:2410] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:10.021674  7147 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.086149  7147 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.088348  7147 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:10.088584  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.089179  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:10.090152  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.090801  1626 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:10.091087  6836 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:10.091642  6836 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:10.093061  6836 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.094939  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.095801  2294 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:10.096343  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.096923  2296 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.096618  7279 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.097891  7156 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:10.098902  2293 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:10.099009  7279 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:10.095185  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.101343  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:10.102069  7194 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:10.102672  7194 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.105132  7194 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:10.015883  7071 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:10.107519  7237 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.107995  7071 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:10.108381  7237 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:10.108940  7071 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.109539  7313 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.110183  7071 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:10.110634  7316 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.107684  7271 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.071722  7221 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:10.113071  7316 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.118937  2294 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:10.118829  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:10.120126  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 104195us
I20260709 13:03:10.115566  7271 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.120970  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:10.117901  7254 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.131075  1705 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:10.126396  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:10.132808  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:10.132766  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.133190  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.025525  7381 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.140584  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:10.164384  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.043581  7270 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:10.139526  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.139832  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.166242  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:10.170303  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:10.176079  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.171820  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.165535  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.176059  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.176548  2292 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:03:10.173841  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.180192  1629 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:10.180349  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.180383  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54086.4us
I20260709 13:03:10.184150  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:10.177389  2292 pending_rounds.cc:85] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 9
I20260709 13:03:10.188576  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.189059  2292 pending_rounds.cc:107] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:03:10.189406  2292 raft_consensus.cc:2889] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.189666  1703 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:10.192327  2288 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:03:10.192744  2288 pending_rounds.cc:85] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 8
I20260709 13:03:10.193049  2288 pending_rounds.cc:107] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:03:10.193332  2288 raft_consensus.cc:2889] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.129954  7279 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.195641  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74811.2us
I20260709 13:03:10.139516  7378 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:10.195641  2293 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:10.202397   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:10.202633  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.151134  7167 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.160730  7333 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.204464  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:10.165604  7147 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.037842  6983 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.204041  1702 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:10.202997  7271 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:10.264415  1630 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:10.242522  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.243017  1628 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:10.165457  7371 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:10.241637  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.254633  1705 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:10.230958  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.255484  1555 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:10.258555  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.271974  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:10.137989  7098 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:03:10.246438  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:10.188813  7313 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:10.229532  7221 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.228708  7153 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:10.281160  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:10.283088  1634 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:10.285225  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:10.286337  1704 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:10.285264  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:10.296581  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50242.6us
W20260709 13:03:10.294178  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.290349  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.300030  1706 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.301119  2289 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:10.306166  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.309113  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:10.311149  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51454us
I20260709 13:03:10.316071  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.317442  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.290181  7153 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.320061  1705 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:10.321754  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.321947  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:10.322633  1560 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.315245  1626 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:10.263859  7279 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.320291  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:10.327919  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.284381  7147 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.309698  7290 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:10.332135  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.337491  1709 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:10.337189  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.338860  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.277486  7308 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:10.345546  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:10.328395  7153 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:10.354022  1626 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:10.358309  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:10.362116  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 95060.3us
I20260709 13:03:10.330273  7248 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:10.368988  2294 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:10.370240  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.372493  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:10.373934  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.390579  1554 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:03:10.315021  7221 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.318104  7313 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:10.393862  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52474.5us
I20260709 13:03:10.371142  1626 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:10.308689  7333 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.392571  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.390301  7182 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:10.406147  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:10.431530  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:10.432898  2288 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:10.434617  1705 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:10.402966  1558 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:10.434882  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:10.435475  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 157701us
I20260709 13:03:10.442431  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:10.441008  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:10.366043  7070 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.369127  7248 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:10.407156  7383 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.000s
W20260709 13:03:10.456636  7395 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:03:10.466228  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:10.441555  7344 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:10.463074  1700 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:10.463872  1628 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:10.401960  7335 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:10.464994  1702 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:10.457191  7404 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:10.359619  7271 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:03:10.393585  7290 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:10.471246  1700 pending_rounds.cc:85] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 9
I20260709 13:03:10.532766  1700 pending_rounds.cc:107] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:03:10.533210  1700 raft_consensus.cc:2889] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:10.534957  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 93496.7us
I20260709 13:03:10.467864  7248 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:10.443302  7254 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.524178  2296 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
W20260709 13:03:10.536829  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.517263  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:10.476503  7397 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:03:10.470332  7126 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.545694  7372 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 13:03:10.547436  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.592833  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.589273  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 65182.8us
I20260709 13:03:10.593173  1705 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:10.525952  7335 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:10.544456  7329 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:10.477171  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 119331us
I20260709 13:03:10.600634  1628 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:10.605387  1560 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 13.
I20260709 13:03:10.607494  1700 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:10.608494  1709 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:10.612267  7335 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:10.615965  6934 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:03:10.470892  7383 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.616508  7306 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.536808  7237 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 13:03:10.642668  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:10.644402  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 194950us
W20260709 13:03:10.647354  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45737.9us
W20260709 13:03:10.629436  7399 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:03:10.648068  1628 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:10.617290  6983 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:10.680814  7398 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.326s	user 0.003s	sys 0.006s
I20260709 13:03:10.686791  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:10.692438  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 257605us
I20260709 13:03:10.600745  7126 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.676492  7329 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:10.698292  1559 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
W20260709 13:03:10.698400  7405 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:03:10.699141  7388 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:10.704996  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:10.708788  1559 pending_rounds.cc:85] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 6
I20260709 13:03:10.715371  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:10.717191  1559 pending_rounds.cc:107] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:03:10.717604  1559 raft_consensus.cc:2889] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.699985  7290 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:10.729835  2288 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:10.681902  7070 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:03:10.693563  7237 consensus_queue.cc:1243] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:10.731317  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:10.734196  1628 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.704571  7271 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.739578  1629 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.001s
I20260709 13:03:10.741393  1629 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:10.714325  7279 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:10.756147  7329 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:10.757896  7406 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 13:03:10.758774  7071 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.002s
W20260709 13:03:10.760114  7376 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:03:10.761113  7413 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.761724  7413 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:10.788450  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 165800us
I20260709 13:03:10.734499  7147 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:10.794028  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:10.782171  7403 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:03:10.782644  7413 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:10.803813  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:10.753543  7271 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:10.958194  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 155507us
W20260709 13:03:10.968950  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 329144us
I20260709 13:03:10.802012  7221 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:10.970192  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 327589us
I20260709 13:03:11.012378  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:11.014165  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:10.804257  7417 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.805878  7333 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:10.977010  7369 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.887s	user 0.015s	sys 0.000s
I20260709 13:03:11.020155  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:11.022512  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 294002us
I20260709 13:03:11.023170  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:11.013068  7403 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:03:11.011441  7418 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:11.011298  7098 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:11.036504  2291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.022s	sys 0.060s
I20260709 13:03:11.062431  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:11.063810  2291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.023s	sys 0.060s
I20260709 13:03:11.036818  7101 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:11.065820  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 914) took 1507 ms. Trace:
I20260709 13:03:11.066133  7194 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.066880  7194 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.066325  1557 rpcz_store.cc:276] 0709 13:03:09.286866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.315006 (+ 28140us) service_pool.cc:224] Handling call
0709 13:03:09.328477 (+ 13471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.361020 (+ 32543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.361074 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.988043 (+626969us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b2c00000f48. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:10.007253 (+ 19210us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.250840 (+243587us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:10.250898 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.250989 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.791354 (+540365us) raft_consensus.cc:1685] finished
0709 13:03:10.791385 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.794279 (+  2894us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128948,"raft.queue_time_us":27936,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":1408,"spinlock_wait_cycles":13395328,"thread_start_us":204565,"threads_started":1,"wal-append.queue_time_us":297516}
I20260709 13:03:11.069386  7194 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.033255  7388 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:11.079346  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 285533us
I20260709 13:03:11.023936  7420 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:11.039062  7156 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.941s	user 0.009s	sys 0.006s
I20260709 13:03:11.099171  7098 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:11.022379  7417 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.070786  7383 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.084810  7237 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:11.105247  7237 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.105764  7237 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.083151  7420 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.107688  7420 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:11.111861  7337 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.112522  7337 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:11.113996  7407 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.418s	user 0.007s	sys 0.004s
I20260709 13:03:11.115854  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:11.116392  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1305) took 1010 ms. Trace:
I20260709 13:03:11.116879  2292 rpcz_store.cc:276] 0709 13:03:10.106229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.120004 (+ 13775us) service_pool.cc:224] Handling call
0709 13:03:10.120345 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.241053 (+120708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.116092 (+875039us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:11.116304 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45381888}
I20260709 13:03:11.117698  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:11.119987  7392 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.120180  7424 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.120954  7392 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:11.120815  7424 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:11.121256  7194 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.121606  7413 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:11.122069  7413 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.121802  7194 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.122467  7413 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:11.123189  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81679.1us
I20260709 13:03:11.123888  7412 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:11.124182  7194 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.041169  7324 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:03:11.124500  7412 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:11.125340  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.063676  7333 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:11.126724  7412 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.127490  7306 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.127944  7195 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:11.128049  7306 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.130452  7306 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.091918  7419 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:11.131053  7419 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.131150  7392 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:11.093182  7416 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:11.132220  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.131547  7419 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.133113  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2775) took 1048 ms. Trace:
I20260709 13:03:11.133495  1633 rpcz_store.cc:276] 0709 13:03:10.084922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.135230 (+ 50308us) service_pool.cc:224] Handling call
0709 13:03:11.132058 (+996828us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b58001f6898. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:03:11.133045 (+   987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8722304}
I20260709 13:03:11.123207  7424 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.105623  7417 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.124806  7194 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.136582  7194 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.138715  7194 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:11.090011  7381 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.925s	user 0.009s	sys 0.003s
I20260709 13:03:11.085150  7415 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:11.096995  7153 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.000s
W20260709 13:03:11.130597  7255 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.060s	user 0.013s	sys 0.000s
I20260709 13:03:11.168494  1708 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
W20260709 13:03:11.172531  7248 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.003s
W20260709 13:03:11.182731  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 163989us
I20260709 13:03:11.191241  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:11.192395  7248 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.210491  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1074) took 1073 ms. Trace:
I20260709 13:03:11.170655  7388 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:03:11.197242  7402 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.870s	user 0.000s	sys 0.002s
W20260709 13:03:11.214686  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 180519us
I20260709 13:03:11.170285  7153 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.196171  7394 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:11.195726  7248 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.212376  2291 rpcz_store.cc:276] 0709 13:03:10.136864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.228970 (+ 92106us) service_pool.cc:224] Handling call
0709 13:03:10.291093 (+ 62123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.318718 (+ 27625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.318739 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.353413 (+ 34674us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.065225 (+711812us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:11.065286 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.065637 (+   351us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.207649 (+142012us) raft_consensus.cc:1685] finished
0709 13:03:11.207674 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.210402 (+  2728us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":595686,"threads_started":1,"wal-append.queue_time_us":768003}
I20260709 13:03:11.252170  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:11.253448  1555 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:11.253609  1557 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 13: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:11.254132  7392 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.256232  1628 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:11.262298  1634 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:03:11.263089  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.263890  1626 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:11.228093  7002 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.009s
I20260709 13:03:11.211071  7337 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.213099  7335 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:11.268419  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.228679  7156 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:11.207831  7070 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:03:11.266700  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.001s	sys 0.017s
I20260709 13:03:11.268558  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.257229  7429 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.272495  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.001s	sys 0.017s
W20260709 13:03:11.274511  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 587793us
W20260709 13:03:11.267884  7002 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.005s	sys 0.009s
I20260709 13:03:11.282364  7438 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.282897  7438 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.285027  7438 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.266484  7126 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:11.287312  2260 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.516s (SENT)
I20260709 13:03:11.266124  7101 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:11.266999  7422 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:11.292960  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.288297  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.292016  1554 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:11.287679  7255 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:03:11.299973  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:03:11.300904  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72410.8us
I20260709 13:03:11.305019  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.266392  7420 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.299845  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.308027  1631 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:11.313290  1557 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:03:11.300899  7425 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:03:11.294230  7418 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:11.265977  7392 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.332445  7101 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.336544  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.337487  1557 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 7.
I20260709 13:03:11.289942  7156 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:11.340441  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64540.4us
I20260709 13:03:11.341776  7156 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.344192  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.344398  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.345263  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.345589  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.347645  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.348326  1634 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:11.350841  7290 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:11.351454  7290 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.353578  7290 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.354055  7290 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.354533  7290 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.356734  7153 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.357470  7420 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.321453  7422 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.360513  7422 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.296236  7429 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.365919  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.366074  7419 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.314093  7126 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:11.277573  7413 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.368788  1558 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:03:11.370440  7413 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.374240  6836 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:11.376179  7421 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:11.394394  7429 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.403820  7419 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.422152  7290 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.431071  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2809) took 1472 ms. Trace:
I20260709 13:03:11.432606  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.433598  7237 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.434300  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:11.434945  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.435307  7430 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.435956  7126 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.443137  6983 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.443717  6983 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.444586  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.446286  6983 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.447086  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.447489  1553 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:11.447741  1700 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:11.449106  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2168) took 1201 ms. Trace:
I20260709 13:03:11.437223  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:11.438997  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 749649us
I20260709 13:03:11.438483  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.437695  7366 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:11.439162  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.09353e+06us
I20260709 13:03:11.461465  1703 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:11.438719  7279 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.460039  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:11.457432  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.471402  1705 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:11.459014  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2804) took 1669 ms (client timeout 2216 ms). Trace:
W20260709 13:03:11.471918  1704 rpcz_store.cc:269] 0709 13:03:09.781511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.793272 (+ 11761us) service_pool.cc:224] Handling call
0709 13:03:11.450029 (+1656757us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54001b7454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:11.450361 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089698560}
I20260709 13:03:11.435789  7377 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.474020  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.473570  7377 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.475741  1703 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:03:11.477694  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.480758  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.481037  7313 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.481690  7430 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.482389  7381 raft_consensus.cc:2764] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:11.483282  7381 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:11.464761  7126 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:11.483322  7366 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:11.484283  7279 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:11.484957  7366 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:11.485453  7366 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:11.485599  7279 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:11.486068  7279 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:11.486536  7366 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.469602  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.488011  7366 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:11.489238  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.490268  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.491048  1709 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:11.492614  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.479503  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.449569  1553 rpcz_store.cc:276] 0709 13:03:10.247271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.247455 (+   184us) service_pool.cc:224] Handling call
0709 13:03:11.448890 (+1201435us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54001ee1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:11.449004 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048175872}
W20260709 13:03:11.495556  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.496330  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.496840  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.432025  1707 rpcz_store.cc:276] 0709 13:03:09.958487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.980997 (+ 22510us) service_pool.cc:224] Handling call
0709 13:03:09.981294 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.981573 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:09.982185 (+   612us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.322849 (+340664us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.323440 (+   591us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.126751 (+803311us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b18003e16a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7557 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:11.304173 (+177422us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18003e16a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:11.430355 (+126182us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b18003e16a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:03:11.430554 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.430616 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:11.430676 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.430838 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.430996 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1411470,"prepare.queue_time_us":981486,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":1121731,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":13600,"spinlock_wait_cycles":28636416,"thread_start_us":328322,"threads_started":1,"wal-append.queue_time_us":608078}
I20260709 13:03:11.502794  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.461328  7394 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:11.523170  1632 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 7.
I20260709 13:03:11.487119  7279 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.524670  7279 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.522150  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:11.533627  1703 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:11.534541  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:11.537660  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.539429  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.532636  7378 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:11.556610  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2820) took 1595 ms (client timeout 1775 ms). Trace:
I20260709 13:03:11.531908  7248 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:11.553937  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.558199  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.559104  1553 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:11.536571  7381 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:11.557262  1708 rpcz_store.cc:269] 0709 13:03:09.961657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.978554 (+ 16897us) service_pool.cc:224] Handling call
0709 13:03:11.556354 (+1577800us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400167954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:11.556521 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734290304}
I20260709 13:03:11.548110  7369 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:11.564349  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.563071  1554 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:11.569767  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.572438  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.557309  6983 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.561498  7248 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:11.580408  1554 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:11.560743  7381 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:11.569736  7369 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.590122  1553 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 18 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
W20260709 13:03:11.589964  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 110598us
I20260709 13:03:11.574611  7378 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:11.598210  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 107106us
I20260709 13:03:11.598397  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:11.586952  7377 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.591033  7381 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.602197  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:11.593724  7369 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:11.586952  7279 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:03:11.603051  2292 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.595398  7237 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.609193  2288 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:11.605436  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:11.609081  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51178.2us
W20260709 13:03:11.608246  7410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.007s	sys 0.033s
I20260709 13:03:11.586405  6983 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.614285  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.614166  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:11.617697  2294 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:11.618095  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.620710  2295 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:11.621059  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.621886  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:11.623916  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.623983  7248 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.625303  7248 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:11.625792  6983 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.627686  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:11.628237  7377 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.628731  7377 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.629182  7237 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:11.630721  7377 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.630602  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:11.631301  7377 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:11.631590  7430 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:11.632121  7430 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.632503  7430 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.623911  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.633069  7271 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.633364  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:11.634497  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:11.635594  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.635907  7377 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:11.636394  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.637017  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.637450  7377 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.640897  7324 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:11.641206  7254 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:11.642592  7254 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:11.643882  1635 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:11.644102  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:11.644598  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2872) took 1508 ms. Trace:
I20260709 13:03:11.644956  1635 rpcz_store.cc:276] 0709 13:03:10.136638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.643225 (+1506587us) service_pool.cc:224] Handling call
0709 13:03:11.643601 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.644314 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.644540 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.646699  7366 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.647320  7366 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.648095  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:11.650394  7271 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.651674  7279 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.651979  7098 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.652262  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2165) took 1445 ms. Trace:
I20260709 13:03:11.596479  7394 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.649653  7366 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.652642  7098 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.654783  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:11.655503  1706 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.657228  7383 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:11.657745  7383 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:11.609987  7101 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:11.658684  7383 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.659317  7101 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:11.616564  7410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.007s	sys 0.033s
I20260709 13:03:11.664208  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.664567  7438 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:11.666236  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:11.660593  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.688762  2288 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:11.667029  1708 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:11.652712  1552 rpcz_store.cc:276] 0709 13:03:10.207082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.207227 (+   145us) service_pool.cc:224] Handling call
0709 13:03:10.257368 (+ 50141us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.469251 (+211883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.469284 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.556888 (+ 87604us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.034446 (+477558us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:11.034503 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.651897 (+617394us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003fb454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:11.651947 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.651991 (+    44us) raft_consensus.cc:1685] finished
0709 13:03:11.652004 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.652205 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49603,"raft.queue_time_us":239,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":63985,"spinlock_wait_cycles":486256896,"thread_start_us":357568,"threads_started":1,"wal-append.queue_time_us":562546}
I20260709 13:03:11.659400  1675 leader_election.cc:400] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:11.690145  1675 leader_election.cc:403] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:11.674542  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 395340us
I20260709 13:03:11.688023  2292 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:11.669762  7392 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.693011  1707 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:11.695901  1630 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:11.678864  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48349.4us
I20260709 13:03:11.675284  7335 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:11.635663  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:11.614982  7316 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:11.686599  2293 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:11.687003  2294 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:11.665508  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2879) took 1300 ms. Trace:
I20260709 13:03:11.698071  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:11.663990  7383 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.711395  1707 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.692595  1701 raft_consensus.cc:2376] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17. Re-sending same reply.
I20260709 13:03:11.712087  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:11.687649  7414 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:11.658605  7098 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.710842  1628 rpcz_store.cc:276] 0709 13:03:10.365586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.735139 (+369553us) service_pool.cc:224] Handling call
0709 13:03:10.757472 (+ 22333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.264780 (+507308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.665259 (+400479us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54002968d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:11.665447 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146547456}
W20260709 13:03:11.725317  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66069.1us
I20260709 13:03:11.719964  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.688165  7438 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:11.736761  1705 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:03:11.695222  7254 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:11.740520  1700 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:11.652278  7279 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.731632  1630 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:11.720069  7366 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.730643  7414 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.754999  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:11.755267  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:11.746366  7430 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.747614  7335 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.719439  6934 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:11.749275  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.751592  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:11.754606  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 423564us
I20260709 13:03:11.759785  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:11.724557  7378 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.723810  7316 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:11.761372  7366 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.760870  7254 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:11.803455  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 107171us
I20260709 13:03:11.787813  1675 leader_election.cc:400] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12.
I20260709 13:03:11.792444  1701 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:11.792910  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:11.807655  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.799446  2295 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:11.814062  1700 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:11.784029  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:11.800173  2293 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:11.816143  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:11.809094  1709 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:03:11.817641  1709 pending_rounds.cc:85] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 11
I20260709 13:03:11.818017  1709 pending_rounds.cc:107] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:03:11.818377  1709 raft_consensus.cc:2889] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:11.811049  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:11.777871  7430 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.772171  7414 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.833652  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:11.796829  7366 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:11.786645  7279 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:11.892925  1706 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:11.784763  7444 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:03:11.894519  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 101390us
I20260709 13:03:11.784899  7290 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:03:11.826929  7392 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:11.898319  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:11.770131  7335 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.764235  7436 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:03:11.793848  7378 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:11.907294  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 238910us
I20260709 13:03:11.813246  7369 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:11.906652  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57405.1us
I20260709 13:03:11.894392  7195 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:11.912935  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 128982us
I20260709 13:03:11.914626  7381 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.914270  7335 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.918234  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:11.890358  6934 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.794700  7316 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.844591  7270 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:03:11.937039  1705 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:11.910490  7396 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:11.937876  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 245604us
W20260709 13:03:11.938506  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 150924us
I20260709 13:03:11.942322  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:11.944193  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:11.834966  7101 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.927886  6836 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.920253  7126 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:11.922303  7456 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.929906  7333 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:11.956220  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:11.923684  7392 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:11.954514  7182 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:11.931257  7071 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:11.909850  7430 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:03:11.922227  7156 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.921701  7388 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 13:03:11.968927  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 164350us
I20260709 13:03:11.908176  7279 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:11.961823  7186 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:11.974968  7457 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:11.945459  6934 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.899308  7438 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:11.941933  7002 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.976701  1701 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:11.975345  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:11.962661  7335 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:11.937388  7324 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:11.971827  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:12.093915  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.987614  7444 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:11.959079  7396 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:12.165285  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.968901  7182 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:12.193094  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.212355  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.153842  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.027913  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:12.186928  7459 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 13:03:12.029762  7333 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:11.968811  7126 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.220053  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.188979  7371 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.192427  7335 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:12.226369  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.234752  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:12.225847  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.242584  7396 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:11.974902  6836 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:11.969317  7456 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.234656  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:12.249118  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 291310us
I20260709 13:03:12.249620  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.250233  1552 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:12.192148  7153 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:12.218544  7422 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.260751  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:12.217249  7444 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:12.192471  7098 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:03:12.251984  7335 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.206069  7290 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:03:12.251494  7254 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.250813  7377 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.263441  7153 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.310407  1675 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:03:12.311848  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:12.299549  7383 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.004s
W20260709 13:03:12.284504  7463 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:03:12.254029  7413 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:12.285872  7333 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.286878  7473 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:03:12.288882  6836 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:12.290845  7290 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:12.309144  7153 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.250958  7392 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:12.294991  7458 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
W20260709 13:03:12.296518  7460 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:12.252694  7371 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.208298  7002 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.302248  1708 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:12.302994  7431 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:12.313678  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:12.314136  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45443.6us
W20260709 13:03:12.306784  1707 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.000s
I20260709 13:03:12.029599  7186 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.215673  7195 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:12.316609  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 331938us
I20260709 13:03:11.476315   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:12.352507  7465 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:12.308239  7464 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 13:03:12.320884  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 405340us
I20260709 13:03:12.321293  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:12.332432  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:12.432547  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100215us
I20260709 13:03:12.308629  7237 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:12.291564  1703 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:12.335073  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:12.436688  1628 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:12.437002  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:12.356029  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:12.356045  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 328262us
I20260709 13:03:12.362668  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.367666  1635 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:12.368543  1707 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:12.273953  7422 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.442749  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:12.335898  7413 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.376248  1552 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:12.341822  7290 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:12.380038  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:12.320510  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.022s	sys 0.039s
I20260709 13:03:12.429600  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:12.428993  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 57695.3us
I20260709 13:03:12.439034  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.447640  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:12.371924  7186 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:12.367245  7431 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:12.320050  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 406725us
W20260709 13:03:12.454425  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 479123us
I20260709 13:03:12.370209  1455 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 12, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:03:12.450949  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.022s	sys 0.039s
I20260709 13:03:12.459025  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:12.455516  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 736734us
I20260709 13:03:12.379119  7465 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:12.435647  7464 consensus_queue.cc:1243] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:12.471662  1632 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:12.465224  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:12.435266  7237 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.447899  7279 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:12.471717  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:12.572819  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.356616  7444 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.461771  7431 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:12.642134  7002 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.460583  6836 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:12.643996  6836 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:12.644449  6836 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:12.479918  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:12.648526  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 168688us
I20260709 13:03:12.649411  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.650302  7366 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:12.650859  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.651144  7366 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:12.651633  7366 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:12.652040  7366 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:12.652031  7248 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:12.652808  7248 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:12.653273  7248 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.486539  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 919) took 1221 ms. Trace:
I20260709 13:03:12.659940  1556 rpcz_store.cc:276] 0709 13:03:11.265641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.306818 (+ 41177us) service_pool.cc:224] Handling call
0709 13:03:11.307158 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.359450 (+ 52292us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:11.603624 (+244174us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.908436 (+304812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.908514 (+    78us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:11.908563 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.294466 (+385903us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:12.486284 (+191818us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b54002ba7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:12.486337 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.486471 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85970,"prepare.queue_time_us":5545,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":86120,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":18027,"spinlock_wait_cycles":136019328,"thread_start_us":457556,"threads_started":2,"wal-append.queue_time_us":306017}
I20260709 13:03:12.528548  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.665220  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.666637  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.668197  1559 raft_consensus.cc:1217] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Deduplicated request from leader. Original: 12.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:03:12.668773  1559 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:03:12.670183  7475 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:03:12.672030  7381 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:12.526654  7290 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.672863  7290 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.593328  7465 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.676419  7308 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:12.676820  7308 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:12.677728  7308 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.676832  7465 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.684625  7422 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:12.686820  1554 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:12.688061  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.688407  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.688723  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.689306  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.689677  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.690196  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.690595  1558 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:12.524068  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:03:12.691533  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 167536us
I20260709 13:03:12.691659  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.692346  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:12.692786  1558 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:12.472683  1632 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:12.579068  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.693869  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.695196  7414 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:12.695601  1558 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:12.695668  7414 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:12.696534  7414 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:12.697477  1629 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:12.527565  7442 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:12.699092  7442 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.494820  7481 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.491410  7126 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:12.500633  7378 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:12.537261  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2184) took 1131 ms. Trace:
I20260709 13:03:12.572791  7383 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:12.493736  7413 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.510318  7237 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.450388  7388 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:03:12.527659  7456 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.502722  7279 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.596903  1628 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:12.604508  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.604423  1627 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:12.607791  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 136118us
I20260709 13:03:12.480240  7195 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.636096  7371 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:12.478416  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:12.753295  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 274957us
I20260709 13:03:12.754812  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:12.668412  7478 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:03:12.689561  1552 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:12.690496  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.494668  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.478838  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:12.532830  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.003s	sys 0.011s
W20260709 13:03:12.765596  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.003s	sys 0.011s
I20260709 13:03:12.756924  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:12.711808  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.700212  7473 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:12.714785  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.709728  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:12.716570  1635 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:12.717974  1630 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:12.615664  7464 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:12.723460  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.727293  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.774677  1556 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:12.701104  6934 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:12.734050  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:12.752333  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.710824  1551 rpcz_store.cc:276] 0709 13:03:11.312586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.328543 (+ 15957us) service_pool.cc:224] Handling call
0709 13:03:11.328976 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.675355 (+346379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.675385 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.675949 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.890495 (+214546us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:11.890537 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.890605 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.443572 (+552967us) raft_consensus.cc:1685] finished
0709 13:03:12.443591 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.443721 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":437462,"raft.queue_time_us":358467,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":20303,"thread_start_us":174505,"threads_started":1,"wal-append.queue_time_us":262245}
I20260709 13:03:12.768554  1554 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 7: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:12.770310  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60680.4us
I20260709 13:03:12.787266  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.771634  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.772471  1626 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:12.772807  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.775919  2293 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:12.785983  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.728698  7413 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:12.702734  7366 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.738299  7279 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.826156  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.798383  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.827473  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:12.798641  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.792593  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2234) took 1114 ms. Trace:
I20260709 13:03:12.830386  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2884) took 1207 ms. Trace:
I20260709 13:03:12.830814  1627 rpcz_store.cc:276] 0709 13:03:11.622600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.622776 (+   176us) service_pool.cc:224] Handling call
0709 13:03:11.625630 (+  2854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.738237 (+1112607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.830162 (+ 91925us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b5400343fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:12.830327 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246338944}
I20260709 13:03:12.834932  1706 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:03:12.768020  7126 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:12.837052  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.801973  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.839452  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.830664  1704 rpcz_store.cc:276] 0709 13:03:11.677748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.677878 (+   130us) service_pool.cc:224] Handling call
0709 13:03:11.692556 (+ 14678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.724290 (+ 31734us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:11.861728 (+137438us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.957888 (+ 96160us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.988875 (+ 30987us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.989074 (+   199us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:12.296529 (+307455us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18000a04e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70d28b858f 00007f70cb8f7571 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd
0709 13:03:12.766093 (+469564us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.769993 (+  3900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.770076 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.792278 (+ 22202us) raft_consensus.cc:1685] finished
0709 13:03:12.792305 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.792517 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46646,"prepare.queue_time_us":201903,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":3612,"raft.queue_time_us":30587,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":35526,"spinlock_wait_cycles":25556352,"thread_start_us":946537,"threads_started":3,"wal-append.queue_time_us":340368}
I20260709 13:03:12.767573  7473 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.723992  7335 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.793051  7371 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.729724  7383 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:12.790918  7484 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:12.896173  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.896401  7474 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:12.901135  7431 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:12.902499  7415 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:12.773000  7101 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:12.905913  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.964958  1552 raft_consensus.cc:2410] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:03:12.800264  7071 raft_consensus.cc:2764] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 13:03:12.912204  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.968981  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:12.936156  7438 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:12.912983  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 465397us
I20260709 13:03:12.951651  7473 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:12.688405  7248 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:12.910754  7472 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:03:13.001852  7494 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:03:12.925400  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:13.050014  1558 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:12.950546  1556 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:12.951668  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:12.841117  7413 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.054788  1551 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:12.966076  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:12.966651  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:12.967558  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2243) took 1192 ms. Trace:
W20260709 13:03:12.912670  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 418029us
I20260709 13:03:12.972642  1553 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:13.056773  1700 rpcz_store.cc:276] 0709 13:03:11.775388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.824346 (+ 48958us) service_pool.cc:224] Handling call
0709 13:03:11.824817 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.891705 (+ 66888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.891751 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.917101 (+ 25350us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.373395 (+456294us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:12.373447 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.375782 (+  2335us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.967246 (+591464us) raft_consensus.cc:1685] finished
0709 13:03:12.967278 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.967481 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44130,"raft.queue_time_us":44301,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6851,"thread_start_us":419924,"threads_started":1,"wal-append.queue_time_us":529310}
I20260709 13:03:12.925009  7477 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:12.942219  7126 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.859347  6934 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:12.978716  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:12.979775  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 211677us
I20260709 13:03:13.072062  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.950132  7371 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.769111  7479 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.968108  7270 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.020630  2293 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:13.022840  1704 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:13.024873  2291 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:13.026836  7248 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.028445  7442 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.028817  1559 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:13.032480  6836 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.034664  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:12.960181  7484 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.015913  7485 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:13.052942  1558 pending_rounds.cc:85] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 10
I20260709 13:03:13.087992  1558 pending_rounds.cc:107] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:13.088357  1558 raft_consensus.cc:2889] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:12.963730  7415 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:13.063055  1554 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:13.092348  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.091318  1455 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 14, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 14 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:13.064349  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.093598  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:13.065387  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99377.4us
W20260709 13:03:13.064697  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 293156us
W20260709 13:03:13.069140  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.078696  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 100061us
I20260709 13:03:12.960963  7455 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:03:13.095528  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.082288  7371 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.071667  7002 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:13.099543  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:13.104413  1632 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:13.096069  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:12.978684  7101 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.099364  1629 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:03:13.072306  7126 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:13.086565  7456 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.102998  7455 consensus_queue.cc:1243] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:13.088819  7070 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:13.087615  7484 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:13.107621  7378 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:13.097226  7270 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.097536  6983 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.101375  7333 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.100991  7381 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.108733  7071 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:13.075389  7414 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:13.126814  7429 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.016s	sys 0.021s
I20260709 13:03:13.133565  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.144308  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.145267  2292 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:13.149338  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.118583  7477 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:13.181058  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.181584  7495 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:03:13.181900  7254 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:13.257956  7254 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:13.258373  7254 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:13.169775  7429 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.017s	sys 0.021s
I20260709 13:03:13.182547  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:13.123697  7070 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:13.146754  7455 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:03:13.188216  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1312) took 1573 ms. Trace:
I20260709 13:03:13.189540  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.197526  7414 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.199596  7431 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.204188  7383 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.124902  7271 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:13.205310  7335 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.213022  6836 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.216962  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2190) took 1009 ms. Trace:
I20260709 13:03:13.124557  7456 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.131201  7415 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.142342  7444 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:03:13.221730  7466 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.224252  7270 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.108544  7442 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:03:13.227185  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:13.124408  7248 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.170833  7377 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.169478  7366 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.130481  6934 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:03:13.104266  7279 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.234733  7422 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:13.251451  7496 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:03:13.093227  7485 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.256016  7333 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:13.261592  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 133512us
I20260709 13:03:13.260305  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.307622  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:13.365095  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.259500  7254 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.304009  7070 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.315876  1560 rpcz_store.cc:276] 0709 13:03:12.207410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.208618 (+  1208us) service_pool.cc:224] Handling call
0709 13:03:12.235687 (+ 27069us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.258580 (+ 22893us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:12.539211 (+280631us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.918424 (+379213us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.957897 (+ 39473us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.958044 (+   147us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:12.958080 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.958144 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.216598 (+258454us) raft_consensus.cc:1685] finished
0709 13:03:13.216637 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.216861 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31033,"prepare.queue_time_us":228902,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":15156,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":16723,"thread_start_us":591933,"threads_started":2,"wal-append.queue_time_us":522233}
I20260709 13:03:13.350863  7415 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:13.356348  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.365329  1704 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:13.371881  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.396991  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.313408  2288 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:13.354900  7186 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.001s
I20260709 13:03:13.266824  7378 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.315156  7271 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.408799  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.413089  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.435489  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.435809  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.296386  7237 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.442376  2295 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:03:13.388343  7366 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:13.347597  7466 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.392249  7381 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:13.348800  7383 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:13.439893  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.094s	sys 0.162s
W20260709 13:03:13.443847  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.095s	sys 0.162s
I20260709 13:03:13.444072  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.397235  7248 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:13.416282  7455 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.442159  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:03:13.420051  7505 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:03:13.443032  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.457152  2288 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:13.423436  6983 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.460680  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.460805  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:13.462723  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 356814us
I20260709 13:03:13.419639  7311 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.313663  2290 rpcz_store.cc:276] 0709 13:03:11.614775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.692968 (+ 78193us) service_pool.cc:224] Handling call
0709 13:03:11.718006 (+ 25038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.938795 (+220789us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:12.348303 (+409508us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.594653 (+246350us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.604353 (+  9700us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.604518 (+   165us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:12.605559 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.605637 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.187996 (+582359us) raft_consensus.cc:1685] finished
0709 13:03:13.188015 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.188135 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14094,"prepare.queue_time_us":15633,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":310317,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":165786,"thread_start_us":832240,"threads_started":3,"wal-append.queue_time_us":415162}
I20260709 13:03:13.452137  2291 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:13.429988  7101 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:13.449803  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 343278us
I20260709 13:03:13.440064  7377 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:13.464071  7442 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:13.440249  7483 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.439985  7476 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.448690  7270 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:13.460467  7498 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.414s	user 0.001s	sys 0.003s
I20260709 13:03:13.476100  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2188) took 1269 ms. Trace:
I20260709 13:03:13.418184  7186 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:13.350170  7479 consensus_queue.cc:1175] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 18 status { last_received { term: 17 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:13.460256  7508 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:13.460059  6934 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.468482  7290 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:13.482887  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 298623us
I20260709 13:03:13.422016  7378 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.485661  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.486918  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.455601  7383 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:13.498307  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 421687us
W20260709 13:03:13.518329  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 358179us
I20260709 13:03:13.462352  7414 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.490635  7481 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.003s
I20260709 13:03:13.652030  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.542524  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:13.726547  1706 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:13.651885  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 957767us
I20260709 13:03:13.542745  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:13.667843  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.728561  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.530090  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.491046  7002 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:13.727900  1709 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:13.737605  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:13.740182  1558 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:13.517378  7071 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:13.727563  1557 rpcz_store.cc:276] 0709 13:03:12.206800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.208510 (+  1710us) service_pool.cc:224] Handling call
0709 13:03:12.236473 (+ 27963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.252427 (+ 15954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.252468 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.482153 (+229685us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b2c00000f48. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:12.482603 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.904276 (+421673us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:12.904315 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.904396 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.469206 (+564810us) raft_consensus.cc:1685] finished
0709 13:03:13.469235 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.475991 (+  6756us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30648,"raft.queue_time_us":9892,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3674,"spinlock_wait_cycles":7816960,"thread_start_us":421245,"threads_started":1,"wal-append.queue_time_us":519603}
I20260709 13:03:13.728639  2291 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:13.747176  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:13.735347  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:13.738195  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:13.724124  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 197562us
I20260709 13:03:13.714375  7518 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.532902  7502 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:13.741654  2288 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:03:13.773403  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:13.773751  1558 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:13.774389  7381 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:13.776880  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.738822  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.778918  7381 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.779184  7532 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:13.781618  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:13.783032  7533 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:13.734745  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.788841  1707 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:13.747654  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:13.656982  7513 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:03:13.790388  1703 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:13.771911  7377 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:13.792531  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.793388  1709 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:13.795531  7070 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.797299  1634 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:13.725023  7414 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.769192  7335 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:13.543426  7483 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.740036  6983 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:13.750485  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:13.751080  1701 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:13.783679  7430 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:13.751303  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:13.751880  1631 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:13.751420  7519 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.649389  7271 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.532564  7237 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.798192  7533 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.753093  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:13.753535  7516 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.649757  7507 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:13.562531  7456 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.599753  7279 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.527220  7311 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.754554  1708 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:03:13.754681  7371 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.007s
I20260709 13:03:13.734083  7508 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.713066  7369 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.755247  2291 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 12: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:13.755865  7517 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:13.759796  7477 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:13.760828  7466 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.767948  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.857014  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:13.768627  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:13.750488  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.857846  1626 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.768842  7431 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.594854  7308 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.598337  7413 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.666841  7485 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.543681  7442 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:13.734524  7481 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.783157  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.639693  7182 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:13.783589  7484 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:03:13.821377  1554 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:13.788403  7515 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.742798  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 212772us
I20260709 13:03:13.740784  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.901337  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:13.752431  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:13.902051  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 161334us
I20260709 13:03:13.734225  7476 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.823127  1706 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:13.903959  7369 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:13.904799  7070 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.905020  7476 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:13.828227  7464 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.002s	sys 0.009s
W20260709 13:03:13.905669  7464 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.002s	sys 0.010s
I20260709 13:03:13.906009  7464 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:13.812392  7335 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:13.840255  2294 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:13.814518  7377 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:13.908003  7335 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.908864  7377 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.842254  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:13.845342  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.845568  7371 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.915695  2291 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:13.916292  7371 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.846282  7378 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.847891  7071 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.849409  7508 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.917213  7147 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:13.917755  7147 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:13.918761  7147 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:13.922636  7487 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.019s	user 0.005s	sys 0.000s
W20260709 13:03:13.850477  7497 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.830s	user 0.000s	sys 0.002s
I20260709 13:03:13.841902  7507 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.838639  7516 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.834420  7533 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:13.928485  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1296) took 1543 ms. Trace:
I20260709 13:03:13.929853  7516 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.928928  2296 rpcz_store.cc:276] 0709 13:03:12.384851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.385077 (+   226us) service_pool.cc:224] Handling call
0709 13:03:12.442404 (+ 57327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.442760 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.442817 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.451704 (+  8887us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.671323 (+219619us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b480024e1d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:12.718942 (+ 47619us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:12.718983 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.719047 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.832892 (+1113845us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:13.928201 (+ 95309us) raft_consensus.cc:1685] finished
0709 13:03:13.928223 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.928382 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":613,"raft.run_cpu_time_us":2118,"raft.run_wall_time_us":223312,"spinlock_wait_cycles":68686464,"thread_start_us":223770,"threads_started":1,"wal-append.queue_time_us":481051}
I20260709 13:03:13.930745  1552 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:13.823961  7002 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.799000  7518 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.886487  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.946007  1708 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:13.884835  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:13.889285  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:13.892143  1560 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:03:13.832165  7430 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.896136  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2897) took 1265 ms. Trace:
I20260709 13:03:13.947118  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:13.895927  7413 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:13.897848  1559 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:13.898097  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:13.898612  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115542us
I20260709 13:03:13.900425  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.851496  7186 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:13.933157  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:13.903600  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.933075  1630 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:13.949590  1633 rpcz_store.cc:276] 0709 13:03:12.630965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.631099 (+   134us) service_pool.cc:224] Handling call
0709 13:03:12.631520 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.632040 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.632084 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.632772 (+   688us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.132707 (+499935us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:13.132752 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.132816 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.895893 (+763077us) raft_consensus.cc:1685] finished
0709 13:03:13.895921 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.896067 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1609,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":499483,"threads_started":1,"wal-append.queue_time_us":797508}
I20260709 13:03:13.900911  7485 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:13.941366  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 183387us
I20260709 13:03:13.854506  7415 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.975642  7509 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.984050  7415 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.984247  7442 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:13.956765  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:13.953619  7473 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.902s	user 0.000s	sys 0.012s
I20260709 13:03:13.833160  7519 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.989153  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:13.993840  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.933657  7279 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:13.992872  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:13.976125  7126 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.847s	user 0.008s	sys 0.004s
I20260709 13:03:13.997803  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.999281  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:14.000180  7519 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.001588  7536 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:13.981683  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2192) took 1773 ms. Trace:
I20260709 13:03:13.982558  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.996198  7481 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:03:14.002324  7536 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.004096  7527 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:14.004932  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.003616  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.984344  7509 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.985864  1706 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:13.986577  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.939393  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:13.959755  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.008229  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.008351  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:13.978852  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.989874  1705 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:13.989940  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.010468  1703 raft_consensus.cc:2410] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:13.988370  1626 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:13.989154  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.887627  7431 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:13.907159  7464 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:13.974288  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70798us
I20260709 13:03:14.018800  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:13.922689  7383 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.994522  1635 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:13.995136  2296 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.975615  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.024092  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:14.024338  2262 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.025257  1554 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:13.976608  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 193790us
I20260709 13:03:14.026021  7509 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.026347  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:13.941876  7515 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.027055  7527 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:14.026904  7335 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.027505  7431 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:13.953187  7413 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:14.001729  7518 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.006196  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.983191  7147 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:13.991390  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:13.860608  7477 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.033995  7456 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.049165  7456 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:14.052177  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.015163  1709 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:14.034768  6983 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.941308  7484 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.053156  6983 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.053675  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.054334  1709 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:14.054997  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.055218  7484 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:13.975625  7378 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:14.055666  1702 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.048147  7537 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.056533  7537 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.019567  1707 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:14.057991  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.058593  1707 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:14.058727  7537 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.059134  1707 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13.
I20260709 13:03:14.048391  7431 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:14.059861  7507 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.047807  7413 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.060698  7515 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.061501  7413 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:14.062042  7415 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.029515  1560 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.006232  1701 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:14.024241  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.031123  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:14.065243  1703 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.031617  1558 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:14.065637  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:14.002295  1555 rpcz_store.cc:276] 0709 13:03:12.208048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.217603 (+  9555us) service_pool.cc:224] Handling call
0709 13:03:12.250447 (+ 32844us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:12.270068 (+ 19621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.270103 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:12.277819 (+  7716us) log.cc:844] Serialized 52 byte log entry
0709 13:03:12.539588 (+261769us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:12.539626 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.539691 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.921972 (+1382281us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:13.981417 (+ 59445us) raft_consensus.cc:1685] finished
0709 13:03:13.981437 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.981625 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32748,"raft.queue_time_us":11888,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3946,"spinlock_wait_cycles":3188608,"thread_start_us":261393,"threads_started":1,"wal-append.queue_time_us":625279}
I20260709 13:03:14.017925  7279 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.034121  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.037177  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.031782  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.055282  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.069705  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:14.072309  1626 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:14.068779  2296 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:14.060145  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:14.037437  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.074011  1627 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:14.069976  1556 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:14.077467  1709 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:14.071338  1705 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:14.080837  7540 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:14.081570  7335 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:14.081617  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44247.3us
I20260709 13:03:14.082237  7533 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.083151  1524 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.055503  6983 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.083997  7335 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.084460  1701 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:14.022912  7369 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.080143  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.089877  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.032586  7383 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:14.088745  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:14.015096  7126 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.109737  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.067602  7521 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.131052  7383 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.131690  7521 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.133392  7383 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.133905  7521 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.135318  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:14.137027  7383 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:14.138964  1524 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.870s (SENT) [suppressed 14 similar messages]
W20260709 13:03:14.139632  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.141016  1627 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:14.142939  1560 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 8.
I20260709 13:03:14.034684  7186 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:14.144167  7186 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:14.098140  1703 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:14.145222  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.145624  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.146167  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.039945  7527 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:14.146421  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:13.996351  7473 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.146981  1706 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:14.148262  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.148838  7473 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.149252  1709 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:14.036679  6934 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.028759  7485 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:14.115317  1705 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:14.120448  7533 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.146974  7527 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.151000  7527 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:14.150956  7485 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.154843  7485 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.120792  7536 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.122357  7477 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.124502  7515 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:14.127063  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.128285  7456 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.130126  7126 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.130774  2292 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:14.111080  7530 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:03:14.131208  7237 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:14.144598  7186 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.149727  6934 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.109299  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.158155  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.158416  1702 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:14.256397  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.256814  1554 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:14.235420  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.138406  7383 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:14.262555  1701 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:14.090055  7279 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.268771  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.108330  7369 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.254099  7237 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.271378  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.249290  7536 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:14.297534  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.256114  7324 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:14.260521  7515 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:14.303323  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:14.301790  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.265442  7002 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:14.311169  2290 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:14.312502  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.314005  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.314868  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.315231  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.316272  7515 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.324043  7482 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.324735  7482 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.326983  2290 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:14.327286  7482 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.332077  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.333181  2290 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:14.334231  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.335479  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.343190  1559 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:14.301071  7237 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:14.338068  7473 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:14.360575  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.363241  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.363907  1702 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.364567  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.365109  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.365694  1705 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:14.365789  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.311434  7536 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.368752  6934 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.345269  7002 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.372056  7002 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.373179  7369 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.374181  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:14.342873  7324 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.398847  7324 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.403858  1702 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:14.405373  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.407068  2290 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:14.361989  7473 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:14.388940  7545 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:03:14.373708  7527 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.423250  1705 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:14.429426  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.436733  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.438555  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.442814  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:14.420288  7473 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.417896  7324 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:14.459362  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.383661  7383 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.439073  7098 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:14.440893  7369 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:14.448805  7413 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:14.470070  7098 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.476120  7279 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.476497  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.476792  7413 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:14.477174  1555 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:14.477713  7413 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.478093  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.479388  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.481631  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1307) took 1176 ms. Trace:
I20260709 13:03:14.484391  2294 rpcz_store.cc:276] 0709 13:03:13.305307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.477788 (+172481us) service_pool.cc:224] Handling call
0709 13:03:14.481467 (+1003679us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400644bd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:14.481568 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257263872}
I20260709 13:03:14.484792  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.485584  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.485692  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1319) took 1303 ms. Trace:
I20260709 13:03:14.485836  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.486295  2294 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:14.483253  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.483469  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1318) took 1301 ms. Trace:
I20260709 13:03:14.486120  2291 rpcz_store.cc:276] 0709 13:03:13.182424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.917347 (+734923us) service_pool.cc:224] Handling call
0709 13:03:13.917682 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.932875 (+ 15193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.932915 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.932970 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:13.933009 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.485452 (+552443us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5400653e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:14.485506 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.485633 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34193152}
I20260709 13:03:14.478003  7098 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.490341  1555 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:14.492066  7488 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.592s	user 0.003s	sys 0.001s
I20260709 13:03:14.492362  2291 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:14.493772  2291 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:03:14.495116  2291 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:14.496536  2291 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:14.508212  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.509263  1559 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:14.510567  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:14.512893  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.513616  2291 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:14.513906  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:14.510058  1555 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:14.515576  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.516398  1705 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:14.516826  7369 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:14.514808  2291 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:14.517287  7369 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:14.517742  7369 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.520076  7479 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:14.520344  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.520552  7479 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:14.520948  7479 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:14.523404  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.526340  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.526621  7254 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.527894  7335 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.529181  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.529979  1556 raft_consensus.cc:2368] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:14.531175  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.532038  1556 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:14.546643  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 137108us
I20260709 13:03:14.534338  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.526979  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:14.487977  2288 rpcz_store.cc:276] 0709 13:03:13.182155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.912447 (+730292us) service_pool.cc:224] Handling call
0709 13:03:13.912758 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.102636 (+189878us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:14.304780 (+202144us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.424091 (+119311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.424165 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:14.424230 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.483221 (+ 58991us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b540027f454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:14.483274 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.483407 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":280716,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4628,"raft.queue_time_us":13481,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":13629,"spinlock_wait_cycles":435900288,"thread_start_us":274102,"threads_started":2,"wal-append.queue_time_us":150102}
I20260709 13:03:14.536893  7186 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.542284  7527 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.543375  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.488826  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.559265  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.545131  7483 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.495518  7536 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.526878  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:14.559725  2290 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:03:14.562301  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:14.567600  2263 leader_election.cc:400] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:14.568554  2263 leader_election.cc:403] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:14.565999  7186 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:14.571031  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.572683  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:14.464847  7324 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.516398  7414 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.561760  7290 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:14.590557  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.577637  7396 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:03:14.597438  7396 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:14.597905  7396 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:14.574558  7527 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:14.574764  7483 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:14.591086  7473 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:14.605958  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.602509  7479 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.614439  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.619542  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.620565  1705 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:14.576125  7366 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:14.622933  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47061.9us
I20260709 13:03:14.598987  7396 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.626792  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.631074  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2898) took 1997 ms. Trace:
I20260709 13:03:14.631529  1631 rpcz_store.cc:276] 0709 13:03:12.634174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.662452 (+ 28278us) service_pool.cc:224] Handling call
0709 13:03:12.662754 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.827643 (+1164889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.630799 (+803156us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540014f5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:14.631006 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493271424}
I20260709 13:03:14.600993  7237 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.638541  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.639415  1708 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:14.643003  2294 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:14.615082  7483 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:14.604691  7290 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:14.644876  2290 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:14.650497  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.625348  7366 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.651839  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.633509  7476 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:14.671168  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:14.671891  1559 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.673693  1556 raft_consensus.cc:2410] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 18 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:03:14.676697  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.630800  7101 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:03:14.653206  7290 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:14.677556  1708 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:14.655084  7430 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.660288  7366 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:14.672673  7396 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.691815  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.665459  7476 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.699519  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.730686  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.731900  1705 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:14.765898  1455 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:14.768348  1455 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.772357  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:14.775990  2289 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:14.776549  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:14.777335  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.778602  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:14.778486  7098 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:14.780128  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:14.729477  7476 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.783927  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 985) took 1512 ms. Trace:
I20260709 13:03:14.784319  1700 rpcz_store.cc:276] 0709 13:03:13.271415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.271589 (+   174us) service_pool.cc:224] Handling call
0709 13:03:13.272018 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.404633 (+132615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.404680 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.404744 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:13.404790 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.783688 (+1378898us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540016c954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:14.783730 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.783869 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304313088}
I20260709 13:03:14.771031  1455 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 16, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:14.808291  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.805068  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:14.822919  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.823804  1703 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:14.798151  7098 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:14.827047  1700 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:14.828941  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:14.808832  7101 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:14.840541  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2899) took 2129 ms (client timeout 1830 ms). Trace:
W20260709 13:03:14.857694  1629 rpcz_store.cc:269] 0709 13:03:12.711410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.711548 (+   138us) service_pool.cc:224] Handling call
0709 13:03:14.840301 (+2128753us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54000dded4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:14.840450 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764236032}
W20260709 13:03:14.859122  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54170.6us
I20260709 13:03:14.859411  1551 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:03:14.841756  2295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1093) took 1695 ms (client timeout 1674 ms). Trace:
I20260709 13:03:14.846498  7476 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.817566  7530 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:03:14.848467  7369 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:14.850685  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.851419  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2738) took 1113 ms. Trace:
I20260709 13:03:14.872032  1552 rpcz_store.cc:276] 0709 13:03:13.737854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.737975 (+   121us) service_pool.cc:224] Handling call
0709 13:03:14.851215 (+1113240us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54001ee1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:14.851358 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418057984}
I20260709 13:03:14.873795  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.875790  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.876536  1702 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:14.852200  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:14.852491  7557 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:03:14.852787  6934 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.861142  7101 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.860744  2295 rpcz_store.cc:269] 0709 13:03:13.146040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.180759 (+ 34719us) service_pool.cc:224] Handling call
0709 13:03:14.841548 (+1660789us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:14.841687 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425132288}
I20260709 13:03:14.871526  1559 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:14.874760  1700 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:14.773677  7483 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.865515  7316 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:14.896908  1703 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:14.858093  7479 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.895525  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.893458  1558 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:14.910915  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58815.8us
I20260709 13:03:14.860492  7473 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:14.894327  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:14.906342  1705 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:03:14.911800  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.912298  1705 pending_rounds.cc:85] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:14.913075  1705 pending_rounds.cc:107] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:03:14.913368  1705 raft_consensus.cc:2889] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:14.915362  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:14.916239  1701 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:14.911747  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:14.898554  7101 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:14.925119  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.900614  7237 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.900357  7484 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.929754  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:14.930966  1701 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:14.915486  6836 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:14.943383  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1087) took 1799 ms. Trace:
I20260709 13:03:14.892109  7098 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:03:14.939582  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:14.920933  7530 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.953615  7476 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:14.948278  2297 rpcz_store.cc:276] 0709 13:03:13.144142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.144282 (+   140us) service_pool.cc:224] Handling call
0709 13:03:13.932865 (+788583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.977115 (+ 44250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.938292 (+961177us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54007701d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:14.943291 (+  4999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729063680}
I20260709 13:03:14.890133  7564 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:03:14.903455  6934 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:14.925808  7519 raft_consensus.cc:2764] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:14.924831  7526 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:14.990638  7526 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:14.991573  7526 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:14.978057  1702 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:15.002908  1702 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:03:15.012601  7565 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:03:15.014710  7561 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:03:14.948197  6836 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:15.005463  7562 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:03:14.998202  7186 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:15.031838  1705 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:15.015103  7237 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:14.939530  7512 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:03:15.054317  7473 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:15.068802  7563 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
W20260709 13:03:15.082154  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.084311  7569 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:03:15.086203  6836 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:15.086956  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1303) took 1974 ms. Trace:
I20260709 13:03:15.087904  2293 rpcz_store.cc:276] 0709 13:03:13.112779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.113040 (+   261us) service_pool.cc:224] Handling call
0709 13:03:13.132998 (+ 19958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.146812 (+ 13814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.146843 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.175723 (+ 28880us) log.cc:844] Serialized 26 byte log entry
0709 13:03:14.775526 (+1599803us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b180021a780. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:14.896362 (+120836us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:14.896398 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.896538 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.086651 (+190113us) raft_consensus.cc:1685] finished
0709 13:03:15.086671 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.086883 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9256,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":30680,"spinlock_wait_cycles":92530944,"thread_start_us":120679,"threads_started":1,"wal-append.queue_time_us":1825402}
I20260709 13:03:15.091373  6934 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:15.081830  7509 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.118865  7369 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:15.120004  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.121774  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:15.126884  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2812) took 1360 ms. Trace:
I20260709 13:03:15.069839  7473 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.127478  1634 rpcz_store.cc:276] 0709 13:03:13.766308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.766452 (+   144us) service_pool.cc:224] Handling call
0709 13:03:13.766871 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.809034 (+ 42163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.126495 (+1317461us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b5400296dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:15.126798 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69511424}
I20260709 13:03:15.129887  7473 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.065062  7237 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.131346  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2212) took 1152 ms. Trace:
I20260709 13:03:15.131968  1560 rpcz_store.cc:276] 0709 13:03:13.978502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.991095 (+ 12593us) service_pool.cc:224] Handling call
0709 13:03:15.131048 (+1139953us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b540036bfd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:15.131259 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76172672}
I20260709 13:03:15.133581  1555 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:15.112715  6836 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:15.144577  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 154954us
I20260709 13:03:15.148592  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.091450  7290 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.150991  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.158803  7366 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:15.161284  7366 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:15.147253  7550 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.176072  7550 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:15.178498  7550 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.164278  7071 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:15.186813  7101 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.188896  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.166173  7311 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.144055  7509 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.151530  7333 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:15.168263  7290 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.171519  7182 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.209818  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:15.194650  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.192638  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:15.208313  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 268790us
I20260709 13:03:15.207661  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.212998  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:15.215751  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:15.166249  7237 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.211321  7071 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:15.215973  7369 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:15.230263  7388 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:15.244226  7388 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:15.245406  7388 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.246838  7388 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:15.253677  7101 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.256623  7509 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.260555  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2210) took 1309 ms. Trace:
I20260709 13:03:15.261045  1551 rpcz_store.cc:276] 0709 13:03:13.950803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.950961 (+   158us) service_pool.cc:224] Handling call
0709 13:03:14.855913 (+904952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.859862 (+  3949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.260327 (+400465us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54001ee1d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:15.260488 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080441728}
I20260709 13:03:15.269236  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:15.269757  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:15.270874  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:15.275156  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:15.277159  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64496.8us
I20260709 13:03:15.278221  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:15.279976  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.219501  7455 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:15.220376  7311 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:15.216019  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:15.281899  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:15.285233  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69321.7us
I20260709 13:03:15.285912  2260 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:15.286056  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:15.240643  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.234421  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.282730  1702 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:15.237934  7366 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:15.292312  1707 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:03:15.289193  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54865.8us
I20260709 13:03:15.294600  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.294929  1702 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:15.296304  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.297083  1629 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:15.297828  1626 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:15.297951  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:15.298974  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.300518  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:15.301076  1702 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.303332  1635 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:15.303252  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:15.301877  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:15.304134  1701 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:15.299404  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.309937  1706 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:15.310334  1703 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:15.308820  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:15.311321  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:15.314296  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.315157  1629 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.315907  1708 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:15.316815  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.317605  1635 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:15.317605  1702 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:15.318531  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.232935  7383 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.319687  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.313772  7126 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:15.320873  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.325209  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.325650  1631 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:15.321131  2291 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:15.321970  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.287243  7279 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.329109  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:15.321130  1627 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:15.325373  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.332125  1633 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:15.332607  1702 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:03:15.276579  7369 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.321282  2293 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:15.334501  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.334858  1634 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:15.336151  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:15.293843  6934 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:15.336145  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:15.330710  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.348527  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.373402  1635 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:15.313355  7509 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:15.303083  6836 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:15.315443  7461 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:15.371610  1626 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:15.329357  7479 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:15.391284  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.375198  2295 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:15.353407  1706 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:15.302929  7473 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:15.311995  7484 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:15.309667  7483 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:15.303732  7455 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:15.291028  7311 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.385977  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.325274  7270 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.374590  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.421121  1627 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
W20260709 13:03:15.389012  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52992.9us
I20260709 13:03:15.390125  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.395485  1629 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
W20260709 13:03:15.395990  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67523.8us
I20260709 13:03:15.388975  7461 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.399605  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.293712  7290 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:03:15.431998  7270 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:15.436195  7270 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.444211  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.399205  7479 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:15.447185  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.405619  7484 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.378022  7369 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:15.465342  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:15.425314  7311 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.400648  7473 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:15.391131  6836 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.432578  7461 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.465656  7479 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:15.469555  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 151463us
I20260709 13:03:15.401719  7483 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.468030  7484 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:15.494154  7530 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 13:03:15.467676  7126 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:15.542273  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 105735us
I20260709 13:03:15.530990  7366 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.525802  7473 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:15.542551  7568 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:15.472286  1701 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:15.562896  7585 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:15.623461  7532 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.624898  7369 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:15.557535  7455 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.421761  7237 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 13:03:15.638090  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 430484us
I20260709 13:03:15.687326  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:15.641983  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 176728us
I20260709 13:03:15.643934  1633 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:15.640733  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:15.658073  7519 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:15.645831  6934 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:15.651069  7377 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.699077  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58432.9us
I20260709 13:03:15.651196  7578 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.640329  7394 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:03:15.703032  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.662729  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:15.704778  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.704875  1632 raft_consensus.cc:2368] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:03:15.667201  2292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1314) took 2490 ms (client timeout 1870 ms). Trace:
I20260709 13:03:15.630491  7461 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.646024  7483 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.692085  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.649533  1630 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:15.664992  7530 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:15.697451  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:15.658900  7509 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:15.656771  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 374926us
I20260709 13:03:15.664116  7521 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:15.700453  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:15.661316  2288 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:15.700371  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:15.701663  7182 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:15.705766  2292 rpcz_store.cc:269] 0709 13:03:13.168908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.182213 (+ 13305us) service_pool.cc:224] Handling call
0709 13:03:15.659243 (+2477030us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:15.659390 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001570688}
I20260709 13:03:15.690295  7535 raft_consensus.cc:2764] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:15.670135  7366 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.735812  7333 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:15.739367  7333 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:15.739789  7333 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:15.740810  7366 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.743459  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2742) took 1505 ms. Trace:
I20260709 13:03:15.744071  1555 rpcz_store.cc:276] 0709 13:03:14.237639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.841435 (+603796us) service_pool.cc:224] Handling call
0709 13:03:15.133264 (+291829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.134152 (+   888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.743191 (+609039us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540036bfd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:15.743394 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670207232}
I20260709 13:03:15.744976  7509 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.729372  1523 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13.
I20260709 13:03:15.748039  2297 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:15.686162  7532 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.749646  1553 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:15.748672  7578 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:03:15.714663  7593 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:15.758579  7593 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.759028  7593 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:15.679050  7590 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 13:03:15.759989  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60246us
I20260709 13:03:15.760591  7590 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:15.760985  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.762002  7592 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:15.761799  7126 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.762395  7592 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:15.763255  7592 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.764511  7592 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.766996  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:15.767421  7237 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:15.767835  7237 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.768211  7237 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:15.768231  7527 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.770005  6983 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:15.770035  7527 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.770975  7392 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:15.726603  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.729619  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:15.704118  6934 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.706760  7377 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.735736  7335 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:03:15.738639  7521 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:15.702486  7098 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:03:15.706931  7512 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:03:15.696991  7482 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:03:15.776604  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.732328  7388 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:15.733644  7071 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.005s
W20260709 13:03:15.779347  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49801us
I20260709 13:03:15.776211  7182 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.748668  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.770421  6983 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:15.782501  7413 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:15.782433  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:15.808621  1555 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.779335  7392 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:15.797700  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.814109  1628 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:15.814293  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.736318  7461 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.795197  7071 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:15.800354  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.800285  6983 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.797389  7429 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 13:03:15.803231  7566 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.898s	user 0.003s	sys 0.007s
I20260709 13:03:15.804119  7388 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:15.805974  7377 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.807099  7530 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:15.781869  7311 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.784363  7521 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.828348  7550 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:15.828526  7527 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.782424  7519 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:03:15.813308  7182 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.793730  7335 consensus_queue.cc:1243] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:15.844106  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:15.861606  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.862622  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:15.807229  7532 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.871309  1552 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:15.897881  1555 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:15.897774  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.833267  7473 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.900059  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:15.899942  7592 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:15.949116  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51412.5us
I20260709 13:03:15.901010  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:15.902542  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:15.903824  1557 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:15.831669  6934 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.918286  1633 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:15.813154  7413 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.889424  7550 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:15.942230  7591 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.913735  7377 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:03:15.924582  7479 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.973423  7324 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:15.957213  7590 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:15.919723  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:15.880295  7461 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.915087  6983 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.832654  7484 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:15.880959  7530 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:15.880844  7366 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:15.942803  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:15.944136  1632 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:15.946244  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 185311us
I20260709 13:03:16.017419  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.900794  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:15.952018  1630 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:15.949425  1628 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
W20260709 13:03:15.951426  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 202833us
I20260709 13:03:15.911985  6836 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:15.954321  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:15.963973  7413 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:15.889102  1635 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:16.007526  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.011813  1633 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:15.965802  7550 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:16.017422  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74713.5us
W20260709 13:03:15.950613  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 142516us
I20260709 13:03:15.971626  7527 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:16.025063  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 70896us
I20260709 13:03:16.026333  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:16.027822  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:15.980573  7377 consensus_queue.cc:1243] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:16.024348  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:16.030857  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1104) took 1473 ms. Trace:
I20260709 13:03:16.033371  2294 rpcz_store.cc:276] 0709 13:03:14.557377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.558980 (+  1603us) service_pool.cc:224] Handling call
0709 13:03:16.030609 (+1471629us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b540040a954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:16.030778 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190792576}
I20260709 13:03:16.036759  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.021965  7366 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:16.037292  7598 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:03:16.037420  1704 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.955088  7388 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.039042  7366 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.040297  1709 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:16.040655  1709 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:16.041325  1709 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.041450  7590 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.042001  7589 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:16.042459  1709 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.020510  7461 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:16.044714  7586 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.625s	user 0.003s	sys 0.002s
I20260709 13:03:16.046346  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.050952  1702 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:16.035800  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.052393  1632 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:03:16.022405  7442 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:15.956799  7392 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:15.971294  7591 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.045245  7592 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:16.082769  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.035995  7126 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.076397  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.077090  7335 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:16.078648  7605 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:16.086221  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.100778  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.050971  7429 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.087586  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.088258  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.107007  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.083963  7530 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.108433  7530 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:16.109911  7377 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:16.110795  7366 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:16.111232  7366 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:16.090058  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.058292  7602 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:16.101491  1632 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:16.102056  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:16.115671  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.102409  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.119056  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.113276  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:16.114549  1555 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:16.115451  7369 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:16.121049  7369 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.121407  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.122162  1554 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:16.123261  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.124171  7366 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.124719  1551 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:16.125398  7366 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.126899  1704 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:16.127722  7592 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:16.128162  7592 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.128521  7592 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.128615  7602 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.132310  7602 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.133247  7377 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.134773  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.134977  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.137357  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.137681  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:16.137933  7532 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:16.138288  7532 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.138381  7071 raft_consensus.cc:2764] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:03:16.139140  7071 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:16.139564  7071 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.139957  7071 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.144701  7495 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.145309  7495 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:16.154667  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 128375us
I20260709 13:03:16.158769  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:16.159262  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.123502  7369 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.160597  7369 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:16.161216  7369 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.166802  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.168915  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.169023  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.170683  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:16.104683  7521 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:16.148239  7574 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:16.178634  7616 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:16.179288  7616 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:16.180181  7616 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.182312  7527 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.183081  7527 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:16.183820  2288 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:03:16.164729  7474 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:03:16.186137  7473 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:16.187074  7279 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.813s	user 0.005s	sys 0.007s
I20260709 13:03:16.187669  7279 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.188346  7279 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.188069  7483 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.188795  7279 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.190356  7483 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.190793  7182 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.192152  1700 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:16.186981  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:16.193696  2294 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:16.195425  2290 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:16.196215  2288 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:16.200047  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:16.200817  1632 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:16.201668  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.183547  7527 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:16.148640  7311 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.156064  7377 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.114847  7589 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:16.208068  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.208808  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.209508  6983 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:16.210127  7383 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:16.210181  7070 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.222963  7413 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.223483  7070 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.163316  7593 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.225559  7070 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.227208  7593 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:16.228399  7617 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 13:03:16.210588  7527 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.212638  7182 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:03:16.230239  7182 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:16.230608  7182 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.219379  7377 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:16.232403  7608 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:03:16.228410  7589 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.233639  7377 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.231453  7182 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.190837  7279 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.234567  7182 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.235579  7369 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.236272  7479 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.234172  7377 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.237805  7071 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.139209  7532 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.221048  7484 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.222138  6983 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.239569  7333 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.249212  6983 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.249780  7333 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.228571  7521 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.230669  7592 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.238706  2291 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:16.216821  1628 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:16.239980  2297 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:16.240163  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:16.240554  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:16.261315  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:16.241303  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:16.245167  2294 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:16.196218  7461 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.251868  1551 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:16.252292  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75714.2us
W20260709 13:03:16.297346  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2215) took 2181 ms (client timeout 2323 ms). Trace:
I20260709 13:03:16.298087  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:16.281924  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:16.300778  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.217299  7388 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:03:16.297942  1553 rpcz_store.cc:269] 0709 13:03:14.077276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.079844 (+  2568us) service_pool.cc:224] Handling call
0709 13:03:16.258743 (+2178899us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:16.258883 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3792085888}
I20260709 13:03:16.304145  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:16.290400  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:16.304368  1557 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:03:16.239248  7413 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.305044  2292 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:16.305974  2288 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:16.307994  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 194780us
W20260709 13:03:16.312273  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51921.9us
W20260709 13:03:16.312804  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72750.2us
I20260709 13:03:16.299320  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:16.313565  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.314100  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.315231  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.315706  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.317755  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.318917  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.319574  1630 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.311673  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:16.316927  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.324276  1632 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:16.239503  7495 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.313064  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:16.327888  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:16.328687  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:16.331041  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 171847us
I20260709 13:03:16.332053  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:16.337065  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.337743  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.236968  7377 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.262315  7509 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.304628  7532 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:16.333928  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.358701  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.353304  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.352275  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:16.349892  1630 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
W20260709 13:03:16.333451  7614 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:03:16.304206  7512 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:16.336408  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.339107  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.357962  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.366432  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.329802  7606 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:16.353073  7369 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:16.296921  7333 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:03:16.338266  7335 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:16.339804  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:16.358273  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.366456  1631 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:03:16.369891  7237 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
I20260709 13:03:16.369632  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.389788  1700 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.256173  7519 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:03:16.351995  7429 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:16.376122  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.399612  1702 raft_consensus.cc:2410] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:16.340050  7383 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:16.377281  1709 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:03:16.379019  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.334527  7521 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:03:16.380270  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.378024  7369 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.336738  7605 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:16.392971  1704 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:16.361456  7509 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.393610  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.388394  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:16.388682  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.353672  7611 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:16.401240  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.355989  7413 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:03:16.401566  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.403820  1709 pending_rounds.cc:85] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:16.387740  1634 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:03:16.405030  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65332.9us
I20260709 13:03:16.406097  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.408413  1557 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:16.484335  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.410378  1705 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.485761  1702 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:16.410833  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:16.486748  1703 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:16.413435  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:16.483745  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.487576  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:16.488999  1630 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:16.487874  1708 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:16.435616  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:16.441454  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 61308.3us
I20260709 13:03:16.388887  7335 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.494439  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:16.473810  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:16.492020  1700 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:16.488209  1709 pending_rounds.cc:107] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:03:16.505625  1709 raft_consensus.cc:2889] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:16.505795  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 95043.9us
I20260709 13:03:16.431525  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:16.477454  7615 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.299s	user 0.001s	sys 0.002s
I20260709 13:03:16.438503  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.515147  1555 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:16.402416  7383 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:16.404979  7606 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:16.510217  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.510232  1700 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:16.496326  1706 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:16.408790  7369 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.442243  6983 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:16.480144  7377 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:16.548974  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54615.5us
I20260709 13:03:16.600965  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.408125  7455 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:16.551035  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.436563  7521 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:16.560920  7588 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:16.516808  7392 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:03:16.444326  7592 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.488387  7611 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.487560  7618 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:16.411154  7605 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.411037  7591 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:16.567378  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:16.398211  7237 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.452689  7071 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:16.655962  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 266058us
I20260709 13:03:16.430759  7429 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.590972  1455 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 13, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 13 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:16.441794  7621 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.605726  7590 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.605553  7588 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:16.603932  7626 raft_consensus.cc:2764] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:03:16.597092  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 102548us
I20260709 13:03:16.514154  7418 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.508875  7335 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:16.442014  7622 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.517518  7482 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:16.562209  7526 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:16.608021  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.778510  7526 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:16.779210  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 171297us
I20260709 13:03:16.780182  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.779940  7526 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.781529  7526 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:16.782951  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.783272  7626 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:16.609472  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 277512us
I20260709 13:03:16.783958  7591 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.784263  7622 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:16.641160  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 73933.5us
I20260709 13:03:16.646960  1705 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:16.579800  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:16.786437  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 206733us
I20260709 13:03:16.786698  1704 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:03:16.787245  7622 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.574302  7383 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.622018  7611 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.604094  7521 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.793255  7631 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:16.736902  7630 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:16.794420  7521 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.795519  7630 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:16.738883  7071 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:16.740458  7429 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.795894  7521 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.796101  7071 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.797111  7071 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:16.797922  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.741659  1455 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 12 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.614785  7377 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.594478  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:16.748067  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 266269us
I20260709 13:03:16.802452  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.592267  7606 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.638602  7605 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.614827  7279 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:16.746692  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:16.808383  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61799.8us
W20260709 13:03:16.757736  7366 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
W20260709 13:03:16.743422  7530 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.002s
I20260709 13:03:16.671305  7618 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.612124  7519 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.136s
I20260709 13:03:16.766254  7588 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.750761  7621 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.673466  7455 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:16.773351  7182 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.003s
I20260709 13:03:16.774595  7369 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:16.759377  7482 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:16.784965  7642 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.788439  7500 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:16.890574  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45248.2us
I20260709 13:03:16.788861  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:16.892757  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 103961us
I20260709 13:03:16.893916  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.789274  7333 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:16.592746  2294 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:16.789335  7070 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:16.734658  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 329) took 35937 ms (client timeout 15000 ms). Trace:
I20260709 13:03:16.804888  7071 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.804582  1455 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 14, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:16.813856  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:16.912189  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 98456.6us
I20260709 13:03:16.913092  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.789510  7593 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:16.838488  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:16.605155  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54237.3us
I20260709 13:03:16.850721  7530 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.922545  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.924134  7418 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:16.795950  7630 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:16.925549  7418 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:16.926412  7630 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:16.926576  7418 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:16.902287  1674 leader_election.cc:400] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:03:16.905797  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49682us
I20260709 13:03:16.927719  1674 leader_election.cc:403] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:16.929469  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.930087  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.930656  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.927328  7418 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:16.855350  1558 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:16.890069  7588 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:16.928361  1455 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 7 to 11, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:16.817425  7639 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.890185  7366 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.906162  7626 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:16.804605  7377 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:16.905653  1457 rpcz_store.cc:269] 0709 13:02:40.797335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.804963 (+  7628us) service_pool.cc:224] Handling call
0709 13:02:41.706813 (+901850us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94e090 00007f70ce9ecf3a 00007f70d47a31cc 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc 00007f70d2d61392 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:16.734378 (+35027565us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:02.346053 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:02.346779 (+   726us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:02.346834 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:03:02.346851 (+    17us) tablet.cc:662] Decoding operations
0709 13:03:02.359405 (+ 12554us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:02.359541 (+   136us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:02.359572 (+    31us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:03:02.363142 (+  3570us) tablet.cc:701] Row locks acquired
0709 13:03:02.363214 (+    72us) write_op.cc:260] PREPARE: finished
0709 13:03:02.363448 (+   234us) write_op.cc:270] Start()
0709 13:03:02.363659 (+   211us) write_op.cc:276] Timestamp: P: 1783602182363355 usec, L: 0
0709 13:03:02.363705 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:03:02.364694 (+   989us) log.cc:844] Serialized 7860 byte log entry
0709 13:03:04.449671 (+2084977us) op_driver.cc:464] REPLICATION: finished
0709 13:03:04.550574 (+100903us) write_op.cc:301] APPLY: starting
0709 13:03:04.570401 (+ 19827us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:05.266288 (+695887us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:05.266438 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:06.014679 (+748241us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:06.052296 (+ 37617us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:03:06.052349 (+    53us) write_op.cc:312] APPLY: finished
0709 13:03:06.053106 (+   757us) log.cc:844] Serialized 180 byte log entry
0709 13:03:06.920866 (+867760us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:06.947713 (+ 26847us) write_op.cc:454] Released schema lock
0709 13:03:06.971253 (+ 23540us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:07.072840 (+101587us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:07.077210 (+  4370us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1342750,"reported_tablets":48,"spinlock_wait_cycles":698526592,"child_traces":[["op",{"apply.queue_time_us":35340,"apply.run_cpu_time_us":55096,"apply.run_wall_time_us":2494146,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87807,"num_ops":20,"prepare.queue_time_us":884893,"prepare.run_cpu_time_us":23757,"prepare.run_wall_time_us":909446,"raft.queue_time_us":439428,"raft.run_cpu_time_us":1922,"raft.run_wall_time_us":35749,"replication_time_us":2085735,"thread_start_us":2653833,"threads_started":5,"wal-append.queue_time_us":1748741,"wal-append.run_cpu_time_us":123396,"wal-append.run_wall_time_us":3474616}]]}
I20260709 13:03:16.998217  7237 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:16.954466  7603 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.813s	user 0.005s	sys 0.007s
W20260709 13:03:17.006402  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.049s
W20260709 13:03:17.006832  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.016s	sys 0.049s
I20260709 13:03:16.974088  1552 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:17.008173  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:16.978410  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:16.937532  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:17.011113  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 73699.1us
I20260709 13:03:17.011940  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:16.912498  7509 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:17.013036  6983 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:17.013718  6983 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:17.014458  6983 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:17.014828  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:16.972934  1455 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 11, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 11 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:17.015515  1631 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:17.015707  1633 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:17.016433  1455 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.017505  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:17.018085  1552 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:17.020071  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 107059us
I20260709 13:03:17.020767  1559 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:17.021132  1559 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:17.021298  7509 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:17.018283  1455 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 15, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.022001  1559 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.023270  1559 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:17.022755  1455 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 16, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 16 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:17.025141  1455 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.935982  7530 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.028023  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:17.028342  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:17.028976  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:17.029551  7530 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:16.979682  7311 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:16.988396  7369 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:16.947050  7589 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:17.030437  7311 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.030979  7589 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.031454  1630 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:17.033126  1630 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:16.990911  7564 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:17.034042  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:17.034654  7444 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:17.035228  7444 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:17.035674  7616 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.036286  7616 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.036136  7444 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.035745  1455 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.038313  1455 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 11, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:17.039105  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:17.040628  7392 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.040005  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:17.041409  7316 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:17.041822  7316 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:17.042168  7316 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:17.043314  7484 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.043852  7484 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.045176  7474 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:17.045599  7474 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:17.046800  7484 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:16.949695  6934 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:03:16.995708  7535 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:16.995883  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.025913  1631 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:17.027740  1633 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:16.991917  7643 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:17.051273  1455 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 10 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:17.040004  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:17.046464  7474 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:16.982833  1560 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:03:17.101663  1560 pending_rounds.cc:85] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 14
I20260709 13:03:16.986033  7500 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:17.049275  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:16.993793  7606 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:17.050720  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:16.948794  7455 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
W20260709 13:03:17.090787  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.058354  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:17.041126  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:17.115187  1554 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
W20260709 13:03:17.115857  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 74752.3us
I20260709 13:03:17.076363  1559 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:17.117758  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 953) took 1010 ms. Trace:
I20260709 13:03:17.097453  1552 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:16.975464  7593 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:17.102065  1560 pending_rounds.cc:107] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:03:17.119304  1560 raft_consensus.cc:2889] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:17.041127  7392 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:17.048956  7377 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:17.098692  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 77948.4us
I20260709 13:03:17.100522  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:16.929390  7642 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.168s
W20260709 13:03:17.108669  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58066.3us
I20260709 13:03:17.050825  7388 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:03:17.054672  7509 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:17.144455  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:17.118263  1559 rpcz_store.cc:276] 0709 13:03:16.107497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.135714 (+ 28217us) service_pool.cc:224] Handling call
0709 13:03:16.136052 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.020674 (+884622us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b540010c820. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:03:17.117369 (+ 96695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.117679 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76305152}
W20260709 13:03:17.097726  7550 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.065s	user 0.009s	sys 0.004s
I20260709 13:03:17.150023  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:17.056210  7639 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.097934  7535 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:17.168469  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:17.002748  7366 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.147181  7530 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:17.142834  6934 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:17.152037  7641 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.146220  7392 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:17.153733  7578 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:03:17.107089  7500 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.234103  1455 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 10, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.984301  7482 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.169651  7377 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:17.160156  7638 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:17.256713  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88387.8us
I20260709 13:03:17.226550  7606 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:17.112772  7643 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.168494  7550 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.254680  6934 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.306103  1455 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 3 to 8, VOTER 444b494bb53842b59408a7230884398a (127.0.107.193) evicted. New cstate: current_term: 10 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:17.316504  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 304631us
I20260709 13:03:17.216229  7429 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:03:17.168083  7509 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.313607  7377 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.266424  7591 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:17.269786  7512 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:17.305991  7500 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:17.316850  7606 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:17.160737  7418 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.377964  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:17.237246  7639 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:03:17.378171  1556 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:17.328372  7070 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:03:17.399528  1455 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 8 to 9, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 9 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:03:17.332327  7648 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:17.332152  6934 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:17.270143  7641 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.365602  7388 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.406493  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 238205us
I20260709 13:03:17.376067  7512 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.388525  7606 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:17.389093  7621 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:17.334256  7377 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:17.411274  7618 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:17.376696  7455 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.537343  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 159453us
I20260709 13:03:17.745533  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:17.747373  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:17.410380  1455 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:17.432261  7512 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:17.520437  7388 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:17.629320  1707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.011s	sys 0.033s
I20260709 13:03:17.314128  7593 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:17.734212  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:17.736510  7649 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:17.737026  7526 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.955s	user 0.009s	sys 0.012s
W20260709 13:03:17.737372  7605 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.930s	user 0.019s	sys 0.000s
I20260709 13:03:17.739310  7650 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:17.740911  7316 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.005s
I20260709 13:03:17.386221  6983 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:17.410760  7369 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:17.833297  7611 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.043s	user 0.013s	sys 0.003s
I20260709 13:03:17.414534  7413 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
W20260709 13:03:17.774609  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.781461  1707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.018s	sys 0.037s
I20260709 13:03:17.808326   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:17.808197  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60940.6us
W20260709 13:03:17.818269  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.960338  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:17.821606  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 405362us
I20260709 13:03:17.962090  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:17.744153  7576 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.808954  7532 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.441s	user 0.007s	sys 0.013s
I20260709 13:03:17.317239  7535 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:17.813166  7652 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.464s	user 0.005s	sys 0.007s
W20260709 13:03:17.949841  7335 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.181s	user 0.015s	sys 0.000s
I20260709 13:03:17.756206  7455 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.818879  7623 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:17.964833  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:17.808419  7662 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:17.830313  7630 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:17.824246  7182 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:17.971614  7626 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:17.970633  7642 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:17.825254  7660 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:17.828997  7388 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:17.829712  7366 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:17.761474  7290 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.202s
I20260709 13:03:17.977067  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:17.977267  1557 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:17.978336  1632 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:17.995319  7369 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:17.920375  7071 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.986s	user 0.014s	sys 0.000s
I20260709 13:03:17.985525  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.154968  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:17.957033  7663 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:18.000878  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:17.898535  7649 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.160523  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.002661  7626 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.966763  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.036s
W20260709 13:03:18.162150  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.015s	sys 0.037s
I20260709 13:03:17.930058  7605 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:17.939842  7316 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.011488  1627 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:17.984079  7532 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:17.987962  7429 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:17.992118  7535 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.028726  1557 pending_rounds.cc:85] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 9
I20260709 13:03:18.028918  1631 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:17.981118  1630 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:18.030226  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:17.933001  7650 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.171875  7532 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.174067  7532 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.174113  7650 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.177217  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2301) took 1168 ms. Trace:
I20260709 13:03:18.178846  7643 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.177762  1707 rpcz_store.cc:276] 0709 13:03:17.008676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.011849 (+  3173us) service_pool.cc:224] Handling call
0709 13:03:17.012185 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.957510 (+945325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.957540 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.958017 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.039011 (+ 80994us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:18.039070 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.039214 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.176947 (+137733us) raft_consensus.cc:1685] finished
0709 13:03:18.176975 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.177144 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1785,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":988223,"threads_started":2,"wal-append.queue_time_us":172307}
I20260709 13:03:18.180713  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1348) took 1339 ms. Trace:
I20260709 13:03:18.181645  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2304) took 1172 ms. Trace:
I20260709 13:03:18.182109  1706 rpcz_store.cc:276] 0709 13:03:17.009593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.014603 (+  5010us) service_pool.cc:224] Handling call
0709 13:03:17.014851 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.308471 (+293620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.308517 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:17.308576 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:17.308621 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.181416 (+872795us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:18.181476 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.181590 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674677504}
W20260709 13:03:18.183526  7659 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 13:03:18.185272  7588 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.208s	user 0.012s	sys 0.000s
I20260709 13:03:18.189867  7649 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:18.192183  7661 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.409s	user 0.000s	sys 0.001s
I20260709 13:03:18.193599  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2299) took 1185 ms. Trace:
I20260709 13:03:18.194074  7482 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.193986  1709 rpcz_store.cc:276] 0709 13:03:17.008012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.009743 (+  1731us) service_pool.cc:224] Handling call
0709 13:03:17.010098 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.010363 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.010388 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.010896 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.234624 (+223728us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:17.234669 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.234752 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.193380 (+958628us) raft_consensus.cc:1685] finished
0709 13:03:18.193400 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.193539 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":223267,"threads_started":1,"wal-append.queue_time_us":397395}
I20260709 13:03:18.028066  7660 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.198961  7660 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:18.210076  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.210675  2294 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
W20260709 13:03:18.162392  7383 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.372s	user 0.017s	sys 0.006s
W20260709 13:03:18.032455  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71477.5us
I20260709 13:03:18.031688  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.032912  7366 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.997076  7484 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.201372  7455 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:18.227131  7623 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:17.964743  7611 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.157202  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.028679  7279 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.011034  7662 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:17.989228  7335 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.167891  1557 pending_rounds.cc:107] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:03:17.978513  7576 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.167315  7663 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.169761  7388 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.181139  2291 rpcz_store.cc:276] 0709 13:03:16.840962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.841181 (+   219us) service_pool.cc:224] Handling call
0709 13:03:16.841652 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.842207 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:17.072368 (+230161us) log.cc:844] Serialized 14 byte log entry
0709 13:03:17.397504 (+325136us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.505319 (+107815us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.741814 (+236495us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:17.741870 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.741959 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.180490 (+438531us) raft_consensus.cc:1685] finished
0709 13:03:18.180512 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.180651 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30264,"prepare.queue_time_us":176673,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":51348,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":47582,"spinlock_wait_cycles":719104,"thread_start_us":376306,"threads_started":2,"wal-append.queue_time_us":1065665}
I20260709 13:03:17.993512  7550 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:18.204578  7521 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.408s	user 0.015s	sys 0.001s
I20260709 13:03:18.183554  7605 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:18.199680  7606 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.000s
W20260709 13:03:18.217588  7392 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.912s	user 0.013s	sys 0.000s
W20260709 13:03:18.218497  7500 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.819s	user 0.011s	sys 0.000s
I20260709 13:03:18.220369  7512 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:18.209388  6934 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.001s
I20260709 13:03:17.994999  7182 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.222752  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:18.128826  2295 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:18.232710  1628 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:18.165814  7626 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.216739  1633 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:18.227949  7509 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:18.242625  1557 raft_consensus.cc:2889] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:18.287730  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65054.4us
I20260709 13:03:18.227878  7484 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.235595  7641 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.235288  7455 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.027997  7642 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.242622  7279 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.246941  7662 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:18.271003  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.007s	sys 0.013s
W20260709 13:03:18.295799  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.486s	user 0.007s	sys 0.013s
W20260709 13:03:18.296343  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 1.48669e+06us
I20260709 13:03:18.271867  7392 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.273240  7611 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.274148  7474 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:18.264683  7606 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.278149  7335 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.287657  7182 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:18.238183  7366 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.236363  7383 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.259829  7388 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.281353  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.288851  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:18.304243  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.305413  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.357321  2297 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:03:18.306151  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.004s	sys 0.004s
I20260709 13:03:18.294242  7588 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:03:18.310421  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.311707  2295 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:18.361366  7509 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:18.359409  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70622.5us
I20260709 13:03:18.364148  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:18.365123  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.365350  7388 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.365859  7651 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.365976  7388 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.366698  7311 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:18.316269  7666 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:03:18.280089  7663 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:18.357949  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.005s	sys 0.004s
I20260709 13:03:18.287446  6934 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:18.415454  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.01859e+06us
I20260709 13:03:18.320672  7621 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.366533  7388 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.367197  7649 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.312362  2293 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:18.283171  7500 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.365350  7589 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.427467  1708 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.371536  7369 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.372071  7101 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:18.372524  7671 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:03:18.372679  7622 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.373219  7623 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.373736  6983 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.374583  7366 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.376037  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.376837  7279 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.377501  7616 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:18.302724  7455 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.378417  7655 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.378821  7484 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.378959  7639 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.378860  7675 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.458645  7675 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.459600  7500 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.448278  6983 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.425179  7621 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.462699  6983 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:18.383104  7665 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.600s	user 0.000s	sys 0.001s
I20260709 13:03:18.383988  7611 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.463054  7621 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.384164  7237 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:18.384531  7532 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.463493  7611 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.463737  7532 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.448560  7622 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.465584  7611 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.465834  7532 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.466785  7622 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.430596  7101 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:18.382323  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.468006  7101 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.384480  7429 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.469107  7429 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.469445  7101 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.385761  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.471369  7429 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.472152  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.473311  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:18.474303  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.475243  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.386684  7495 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.476167  7495 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.478642  7495 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.387732  7656 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:18.366380  7651 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.468020  1559 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:18.480093  7656 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.481364  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.481775  7651 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.482268  7656 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.388813  7677 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:18.483043  7677 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.485467  7677 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.389463  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.389703  7662 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.390762  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 967) took 1351 ms. Trace:
I20260709 13:03:18.486318  1557 rpcz_store.cc:276] 0709 13:03:17.038959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.039847 (+   888us) service_pool.cc:224] Handling call
0709 13:03:17.040213 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.287988 (+1247775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.390534 (+102546us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54002b98d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:18.390703 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148442368}
I20260709 13:03:18.487717  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.488323  1557 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 8.
I20260709 13:03:18.489154  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.391420  7521 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.489914  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.490288  7521 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.492435  7521 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.378988  7413 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.494033  7413 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.490569  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:18.496428  7413 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.392755  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:18.497359  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 104680us
I20260709 13:03:18.395924  7654 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.499750  7654 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.501997  7324 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:18.502713  7324 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.502166  7651 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:18.503237  7324 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.504415  7651 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:18.504854  7651 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:18.505714  7651 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.506909  7651 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:18.395977  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.501988  7654 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.397342  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.509030  2292 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:18.509356  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.510079  1631 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:18.510227  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.510869  2292 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:18.511116  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.398092  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:18.512087  2297 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.399055  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 965) took 1468 ms. Trace:
I20260709 13:03:18.513837  7316 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.400195  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.512857  1555 rpcz_store.cc:276] 0709 13:03:16.930311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.930713 (+   402us) service_pool.cc:224] Handling call
0709 13:03:16.931081 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.270771 (+339690us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:17.638217 (+367446us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.187232 (+549015us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b1800429700. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:18.373702 (+186470us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b1800429700. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:03:18.373967 (+   265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.374537 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.374710 (+   173us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:18.374749 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.374834 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.398840 (+ 24006us) raft_consensus.cc:1685] finished
0709 13:03:18.398861 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.398994 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78519,"prepare.queue_time_us":720511,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":11242,"raft.queue_time_us":546975,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":45682,"spinlock_wait_cycles":295453824,"thread_start_us":765073,"threads_started":3,"wal-append.queue_time_us":303355}
I20260709 13:03:18.514701  1552 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:18.515100  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.515581  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.516124  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.400116  7637 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:18.516692  1629 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.401082  7631 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:18.517604  7418 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:18.403823  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2239) took 2560 ms. Trace:
I20260709 13:03:18.519145  1556 rpcz_store.cc:276] 0709 13:03:15.843591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.844854 (+  1263us) service_pool.cc:224] Handling call
0709 13:03:15.880394 (+ 35540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.400624 (+1520230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.403516 (+1002892us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003098d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:18.403717 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3443793792}
I20260709 13:03:18.520110  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.407130  7383 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.522082  7444 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.516705  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.525251  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:18.526070  2292 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:18.525166  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:18.292270  7596 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:18.425886  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.573117  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.573798  1706 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:18.574851  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.575439  1706 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:18.416924  6836 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.430617  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.442287  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.576162  1703 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:18.443578  2295 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:18.429750  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.445596  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.578001  1700 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:18.444815  1709 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:03:18.578521  1709 pending_rounds.cc:85] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 5
I20260709 13:03:18.578841  1709 pending_rounds.cc:107] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:03:18.579134  1709 raft_consensus.cc:2889] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:18.378073  7071 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.416680  7311 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:18.451335  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 966) took 1412 ms. Trace:
I20260709 13:03:18.452020  1635 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:03:18.453866  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.447896  7623 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.454844  1701 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:18.454929  7606 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.456801  7655 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.457612  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1115) took 1129 ms. Trace:
I20260709 13:03:18.457679  7639 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.460942  7675 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:18.468494  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1349) took 1462 ms (client timeout 1761 ms). Trace:
I20260709 13:03:18.385977  7673 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.385484  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:18.387499  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.482445  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:18.618479  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 246683us
I20260709 13:03:18.621409  1559 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:18.622675  1559 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:18.527103  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.624176  1708 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:18.430963  7663 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.626907  1554 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:18.561050  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.627812  1626 raft_consensus.cc:2368] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:18.489881  1557 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:18.526731  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.565161  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:18.559459  1632 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
W20260709 13:03:18.628576  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101905us
W20260709 13:03:18.628832  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63780.6us
I20260709 13:03:18.629603  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.629832  1555 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:18.568204  7631 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:18.562116  1629 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:18.569118  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.573704  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:18.635821  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 271720us
I20260709 13:03:18.636339  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.636803  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:18.636910  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 67970.6us
I20260709 13:03:18.637792  1557 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:03:18.584110  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53818.2us
I20260709 13:03:18.585004  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.449440  1707 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:03:18.590606  1700 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:18.582849  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:18.583739  1558 rpcz_store.cc:276] 0709 13:03:17.038625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.040126 (+  1501us) service_pool.cc:224] Handling call
0709 13:03:17.040403 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.098881 (+ 58478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.098913 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.099423 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.507227 (+407804us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:17.507286 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.507392 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.451124 (+943732us) raft_consensus.cc:1685] finished
0709 13:03:18.451144 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.451275 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58101,"raft.queue_time_us":108370,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":13947,"thread_start_us":407420,"threads_started":1,"wal-append.queue_time_us":925494}
I20260709 13:03:18.592310  2292 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:18.641296  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.591614  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:18.642041  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.641991  1708 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:18.594964  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:18.642699  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.597016  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:18.604874  2290 rpcz_store.cc:276] 0709 13:03:17.328270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.365594 (+ 37324us) service_pool.cc:224] Handling call
0709 13:03:17.365997 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:17.399034 (+ 33037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.399064 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:17.405686 (+  6622us) log.cc:844] Serialized 52 byte log entry
0709 13:03:17.848566 (+442880us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:17.848634 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.848740 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.456906 (+608166us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b480024e1d8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:18.457279 (+   373us) raft_consensus.cc:1685] finished
0709 13:03:18.457300 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.457553 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32475,"raft.queue_time_us":33050,"raft.run_cpu_time_us":2890,"raft.run_wall_time_us":1055325,"spinlock_wait_cycles":7383936,"thread_start_us":112750,"threads_started":1,"wal-append.queue_time_us":940532}
I20260709 13:03:18.610144  2289 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:18.609980  1703 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:18.611212  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:18.448961  7593 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
W20260709 13:03:18.612885  2294 rpcz_store.cc:269] 0709 13:03:17.006403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.007893 (+  1490us) service_pool.cc:224] Handling call
0709 13:03:18.468313 (+1460420us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002c8154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:18.468430 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763747712}
I20260709 13:03:18.617368  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.439709  7649 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.438988  7369 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.566684  7279 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.629366  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.567219  7333 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:18.562407  7500 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.582796  7383 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.580715  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.639120  1705 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:18.585419  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 155700us
I20260709 13:03:18.640944  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.646034  2297 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:18.667217  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.668001  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:18.650871  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53944.1us
I20260709 13:03:18.651932  2293 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:18.610621  7311 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:18.679199  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.680454  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.682001  1558 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:18.687268  1705 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:18.626703  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:18.638126  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.608981  7461 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.658124  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.601724  7195 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.663021  1555 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:03:18.700428  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62376.4us
I20260709 13:03:18.626035  7687 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:18.665035  1709 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:18.668285  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.581027  7641 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:03:18.669880  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.670271  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.727316  7641 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:03:18.727501  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57698.2us
I20260709 13:03:18.727624  7290 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:18.728389  7290 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.728631  7098 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:18.728812  7290 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.671597  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.662971  7662 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.672920  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:18.690236  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63626.6us
I20260709 13:03:18.669090  7578 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.670786  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:18.695649  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.696561  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.696753  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:18.742725  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72019.8us
I20260709 13:03:18.743060  2294 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:18.697089  7482 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:18.595577  7589 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:03:18.699194  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.750214  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.657409  7619 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:03:18.750850  2289 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:18.716394  7622 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:18.662586  7070 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:03:18.663242  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.596267  7071 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.596930  6836 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.650173  7530 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:18.647725  7474 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:18.659242  7596 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:18.720300  1703 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:18.662431  7316 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:18.662703  7690 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:18.648532  7654 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:18.669318  7673 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.796072  7654 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.797088  7369 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.797700  7369 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:18.798396  7654 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.798179  7369 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.724607  1700 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:18.799698  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.800493  1700 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:18.801323  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.720124  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.801741  7455 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.687021  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:18.774137  7631 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:18.802757  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 115850us
I20260709 13:03:18.802680  7631 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.769191  7237 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:18.649413  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.805472  7237 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:18.801333  7509 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:18.805933  7237 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:18.806289  7509 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:18.807952  7637 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:18.807536  7509 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.808491  7637 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:18.808744  7564 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:18.809151  7509 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.809414  7637 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.809806  7564 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:18.727792  7641 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:18.671514  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.812439  7641 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.656119  1557 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:18.814644  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.815189  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.816843  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:18.817566  1557 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.665491  7576 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.671631  2297 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
W20260709 13:03:18.743425  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 180845us
I20260709 13:03:18.821004  7631 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.743741  1707 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:18.822197  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.739334  1708 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
W20260709 13:03:18.822685  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 114698us
I20260709 13:03:18.823405  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.824570  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.825148  1708 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:18.750905  1702 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:18.701074  7461 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:18.702754  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.699654  7500 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.703001  1635 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:03:18.752846  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.668900  7484 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:03:18.768147  7530 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.719630  7413 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:18.651684  7663 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.765982  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.679725  7383 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.738376  7687 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:18.698774  7333 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.751614  7366 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:18.799589  7673 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.713855  7195 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.699002  7279 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.781847  1560 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:18.890728  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.754168  7622 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.786470  7605 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:18.786926  7630 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.789s	user 0.007s	sys 0.003s
I20260709 13:03:18.787750  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.788746  7642 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.789167  6934 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:18.885977  7606 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:18.803580  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:18.728768  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.813869  7641 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:18.826893  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.822753  1707 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:18.827862  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.930176  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.817587  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:18.828562  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.932950  1703 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.828955  1525 leader_election.cc:400] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
W20260709 13:03:18.731702  7535 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:03:18.835403  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:18.835721  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:18.840651  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.941963  1551 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:18.840857  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:18.841987  1635 pending_rounds.cc:85] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 9
I20260709 13:03:18.842247  1555 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:18.745872  7662 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:18.715446  7512 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:18.848491  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:18.946696  1702 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:18.865284  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.866137  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.873425  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.947918  1634 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:18.802495  7392 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.784061  7316 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.882051  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:18.887056  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.841369  7461 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.840464  7694 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:03:18.784265  7690 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:18.891533  1629 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:18.843031  7455 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.895756  1709 raft_consensus.cc:2410] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:18.979411  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:18.896210  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 191773us
I20260709 13:03:18.882934  7530 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:18.980665  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:18.791620  7324 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.921653  1552 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:18.933808  1525 leader_election.cc:403] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:18.949842  7623 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.943442  1635 pending_rounds.cc:107] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:03:18.996510  1635 raft_consensus.cc:2889] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:18.945560  2290 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:18.842206  7618 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
W20260709 13:03:18.945907  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.874595  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.890672  7366 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.948879  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:18.950887  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 222178us
I20260709 13:03:19.016108  1628 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:19.017670  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:19.019112  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:18.968634  7070 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:18.952728  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 111942us
I20260709 13:03:18.896060  7622 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:18.891638  7413 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:18.751493  7578 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.896330  7388 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:18.920603  7621 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.894750  7605 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:18.966867  1555 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:18.970798  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 167296us
W20260709 13:03:18.970907  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 88927.2us
I20260709 13:03:18.976536  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:19.049830  1626 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.833110  7311 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:18.888486  7521 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:03:18.790069  7688 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:03:18.950187  6934 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.945760  7098 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.991194  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.085204  1709 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:19.086536  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.087278  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:18.931663  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:18.994261  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 165527us
I20260709 13:03:19.088774  1705 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:18.994421  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.090036  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:18.998981  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.093304  1702 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:18.964103  7316 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.001221  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:18.957746  7535 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:18.985220  7377 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.356s	user 0.012s	sys 0.001s
I20260709 13:03:19.096882  7535 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.097041  7377 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.097621  7535 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:19.099562  7377 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:18.951725  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:18.979741  7690 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:19.100483  1708 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:19.025135  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 325464us
W20260709 13:03:19.061462  7696 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:03:18.982308  7611 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:18.994957  7596 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:19.052552  7474 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:03:19.052824  7444 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 13:03:19.054322  7484 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:19.055199  7333 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:19.056807  7651 consensus_meta.cc:287] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.003s
I20260709 13:03:19.063225  7195 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.041399  7655 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.065457  7605 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.062085  1557 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:19.069674  1703 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:19.179342  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.021771  7366 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:19.020457  7369 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.074193  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.185124  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.185901  1552 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:19.186271  1703 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:19.188153  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:19.188900  1554 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.075510  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.077486  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:19.080017  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.189909  1555 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:19.026342  7392 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:18.990425  7455 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:19.103116  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108807us
W20260709 13:03:19.122123  7101 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.001s
I20260709 13:03:19.027132  7639 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.061326  7578 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.201576  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:19.202241  1553 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:19.119810  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 100792us
I20260709 13:03:19.124780  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.034224  7687 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:19.068357  7512 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:03:19.110538  7396 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:19.180687  1557 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:19.069535  7621 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.181859  1633 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:18.984800  7630 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:19.109475  7705 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:19.195359  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.198781  7605 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:19.197147  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.224350  2294 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:19.121613  7654 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:19.182428  7673 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:03:19.196139  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:19.150103  1707 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:19.203578  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:19.110409  6934 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.236928  1707 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:19.238562  1626 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:19.206557  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:19.109997  7098 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.180088  7631 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
W20260709 13:03:19.214126  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 89452.9us
I20260709 13:03:19.177855  7071 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.182884  7484 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:19.183466  7482 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:19.276309  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.277148  2293 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:19.216786  7700 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:03:19.230815  7654 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:19.183748  7333 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:03:19.204547  7392 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:19.243454  1553 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:19.249405  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.168095  2288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:03:19.252638  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:19.217034  7369 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.177052  7413 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.212088  7383 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:19.258371  1709 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:19.219264  7396 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:19.265563  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.266688  1703 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:19.267571  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.270625  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.221572  7630 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:19.273003  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:19.183131  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 323509us
W20260709 13:03:19.304543  2260 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.821s (SENT) [suppressed 11 similar messages]
I20260709 13:03:19.232507  7279 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.236984  7591 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:19.236581  7444 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:19.300949  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 211003us
I20260709 13:03:19.298107  2288 raft_consensus.cc:606] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:19.301254  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.310948  2288 raft_consensus.cc:614] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:19.290709  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 242153us
W20260709 13:03:19.307204  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.307644  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.312376  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:19.312660  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.312345  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.313987  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:19.299007  7290 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.001s
I20260709 13:03:19.309180  7630 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:19.290222  7482 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:19.321050  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.339052  7482 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.321568  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.321950  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:19.342026  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.300354  7500 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.298012  7654 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:19.336663  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.303676  7324 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.322644  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.314410  7591 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.308038  6983 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:19.320361  7383 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.320041  7279 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:19.345670  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.254400  7596 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.314589  7444 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:19.342774  7642 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:19.367048  7704 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:03:19.375661  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.357568  7706 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:19.323387  7716 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.335557  7713 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:03:19.359376  7619 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.357780  7623 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.377408  1703 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:19.372720  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:19.355854  7377 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:19.365446  7591 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.388306  1628 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:19.397043  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.375654  7642 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.423640  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.425540  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:19.427121  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:19.379626  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.393116  1708 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
W20260709 13:03:19.379750  7701 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:03:19.436062  7098 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.340943  7290 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:19.437904  7237 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.004s
I20260709 13:03:19.438364  7237 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.439466  7693 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:19.440711  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:19.422396  7716 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:19.377384  7509 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:19.399828  7714 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:03:19.443984  7642 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:19.379132  7651 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.447772  7651 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.439375  7290 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:19.456853  7710 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
W20260709 13:03:19.458801  7182 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.042s	user 0.004s	sys 0.008s
I20260709 13:03:19.451330  7651 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:19.400471  7709 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:03:19.403385  7626 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.110s	user 0.010s	sys 0.000s
I20260709 13:03:19.407323  7530 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.408460  7070 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:19.466806  7070 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:19.468087  7070 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.468804  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2872) took 1021 ms. Trace:
I20260709 13:03:19.469794  7070 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:19.471087  7702 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
I20260709 13:03:19.469386  1709 rpcz_store.cc:276] 0709 13:03:18.447536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.152732 (+705196us) service_pool.cc:224] Handling call
0709 13:03:19.153133 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.298510 (+145377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.468466 (+169956us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400166a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:19.468718 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240876928}
I20260709 13:03:19.472833  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.372254  7484 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.473579  7626 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.411684  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:19.476256  1703 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:19.476744  7606 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:19.372983  7444 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.477213  7606 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.394507  7377 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:19.388451  7392 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.478348  7377 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:19.412254  7418 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.894s	user 0.007s	sys 0.007s
I20260709 13:03:19.375984  7383 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.432912  7455 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.485901  2291 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:19.486320  7663 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.487241  7705 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:19.487694  7705 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:19.487932  2291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1116) took 1207 ms (client timeout 1530 ms). Trace:
I20260709 13:03:19.488070  7705 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:19.488343  2291 rpcz_store.cc:269] 0709 13:03:18.280898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.281069 (+   171us) service_pool.cc:224] Handling call
0709 13:03:19.487716 (+1206647us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005186d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:19.487869 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596177280}
I20260709 13:03:19.398654  7619 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.368053  7396 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:19.472770  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:19.529865  7706 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.490913  1701 raft_consensus.cc:2368] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:19.536278  7509 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:19.544286  7626 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:19.546015  7461 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.002s
I20260709 13:03:19.525617  7619 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:19.526777  7383 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.493656  7694 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.546195  7530 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.546810  7182 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.416867  7366 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.629004  7693 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:19.550145  7535 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.551504  6836 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.552207  7444 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.556051  7621 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:19.565008  7589 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.578351  7703 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.524s	user 0.002s	sys 0.000s
I20260709 13:03:19.589672  7626 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:19.594357  7662 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:19.605607  7482 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:19.611238  7396 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.535983  7316 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.631083  6934 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:19.751906  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.773728  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 458213us
W20260709 13:03:19.776310  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 465098us
I20260709 13:03:19.779081  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 972) took 1280 ms. Trace:
I20260709 13:03:19.782436  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2769) took 1402 ms. Trace:
I20260709 13:03:19.785552  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2247) took 1404 ms. Trace:
W20260709 13:03:19.775380  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 522647us
I20260709 13:03:19.400077  1634 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
W20260709 13:03:19.793414  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.795120  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 989) took 1201 ms. Trace:
I20260709 13:03:19.805540  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2860) took 1238 ms. Trace:
W20260709 13:03:19.810992  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.795284  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2850) took 1285 ms. Trace:
W20260709 13:03:19.816537  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 462283us
I20260709 13:03:19.594836  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:19.829574  1700 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:19.581439  1701 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:19.582582  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2309) took 1141 ms. Trace:
W20260709 13:03:19.625041  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 197996us
W20260709 13:03:19.963387  7596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.020s
W20260709 13:03:19.964063  7596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.001s	sys 0.020s
I20260709 13:03:19.854758  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.965306  2290 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:19.856519  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.858110  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:19.859846  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.968485  2291 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:19.861106  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:19.859846  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.861763  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:19.971540  6983 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:19.971863  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 376939us
I20260709 13:03:19.862963  2294 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:03:19.972250  6983 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:19.972860  2294 raft_consensus.cc:606] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:19.866729  7182 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.868266  7237 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:19.973395  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:19.974496  7237 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.975278  7237 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.662297  7316 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.657397  7366 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.658897  7444 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.872447  7071 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:03:19.983747  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2940) took 1118 ms. Trace:
I20260709 13:03:19.630327  7461 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.984510  1634 rpcz_store.cc:276] 0709 13:03:18.865447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.948613 (+ 83166us) service_pool.cc:224] Handling call
0709 13:03:19.983236 (+1034623us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400343fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:19.983498 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882368768}
I20260709 13:03:19.693188  7535 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:19.988987  2297 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:19.653179  7589 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:19.665519  7622 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.003s
I20260709 13:03:19.996567  7622 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.655695  7626 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.690240  7630 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:19.697216  7637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.009s	sys 0.013s
I20260709 13:03:20.090899  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:20.091575  7637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.009s	sys 0.014s
I20260709 13:03:19.681414  7642 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.650683  7649 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:19.652926  7667 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:20.093461  7667 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:20.094858  7667 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:19.555519  7690 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:19.650079  7693 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:19.631059  7694 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.095700  7642 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.096513  1702 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:19.661012  7705 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:19.930301  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.867468  6836 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.876431  7333 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:19.920719  1556 rpcz_store.cc:276] 0709 13:03:18.380794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.520168 (+139374us) service_pool.cc:224] Handling call
0709 13:03:19.783011 (+1262843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.783543 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.783573 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:19.783645 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:19.783681 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.783792 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.785078 (+  1286us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540036e7d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:19.785281 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2903278848}
I20260709 13:03:19.936781  1635 rpcz_store.cc:276] 0709 13:03:18.593221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.693744 (+100523us) service_pool.cc:224] Handling call
0709 13:03:18.694178 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.997406 (+303228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.793390 (+795984us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b54001e8054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:19.794792 (+  1402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19406720}
I20260709 13:03:19.919891  1558 rpcz_store.cc:276] 0709 13:03:18.498067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.699052 (+200985us) service_pool.cc:224] Handling call
0709 13:03:18.734057 (+ 35005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.923641 (+189584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.923692 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.923737 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:18.923797 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.778463 (+854666us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54003fb954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:19.778538 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.778836 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435210496}
I20260709 13:03:19.937642  1633 rpcz_store.cc:276] 0709 13:03:18.509247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.510860 (+  1613us) service_pool.cc:224] Handling call
0709 13:03:19.794643 (+1283783us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54001405d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:19.794958 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541098880}
I20260709 13:03:19.920356  1560 rpcz_store.cc:276] 0709 13:03:18.380199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.389216 (+  9017us) service_pool.cc:224] Handling call
0709 13:03:19.781916 (+1392700us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540036e7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:19.782186 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679717632}
I20260709 13:03:19.992614  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1026) took 1256 ms. Trace:
I20260709 13:03:19.973922  7388 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.973397  2294 raft_consensus.cc:614] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:20.036019  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.039958  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48176.7us
W20260709 13:03:20.040189  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75222.1us
I20260709 13:03:20.041316  7589 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.015455  7237 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:20.047966  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74820us
I20260709 13:03:20.047525  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1027) took 1306 ms. Trace:
I20260709 13:03:19.972879  6983 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.955799  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.055639  1707 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:20.061456  7622 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.061784  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1034) took 1212 ms. Trace:
I20260709 13:03:20.063947  7461 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.065438  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 975) took 1566 ms. Trace:
I20260709 13:03:20.065207  7675 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 13:03:20.066087  1704 rpcz_store.cc:276] 0709 13:03:18.441405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.441513 (+   108us) service_pool.cc:224] Handling call
0709 13:03:18.444067 (+  2554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.525327 (+ 81260us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:18.597127 (+ 71800us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.698092 (+100965us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b18004c0600. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:18.893901 (+195809us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.893951 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:18.894002 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.582279 (+688277us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:19.582334 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.582516 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4411,"prepare.queue_time_us":69595,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":44,"raft.queue_time_us":116,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":8515,"spinlock_wait_cycles":199443456,"thread_start_us":195569,"threads_started":1,"wal-append.queue_time_us":366061}
W20260709 13:03:20.063515  7654 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.699s	user 0.014s	sys 0.007s
I20260709 13:03:20.068395  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.042142  7716 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:03:20.071590  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:20.112536  1700 rpcz_store.cc:276] 0709 13:03:18.739861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.801059 (+ 61198us) service_pool.cc:224] Handling call
0709 13:03:20.000954 (+1199895us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:20.045950 (+ 44996us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360639232}
I20260709 13:03:19.936873  1627 rpcz_store.cc:276] 0709 13:03:18.566504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.697885 (+131381us) service_pool.cc:224] Handling call
0709 13:03:18.698234 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.951057 (+252823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.951104 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.951739 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.068232 (+116493us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:19.068278 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.804623 (+736345us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001e8a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:19.804711 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.804804 (+    93us) raft_consensus.cc:1685] finished
0709 13:03:19.804828 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.805298 (+   470us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10170,"raft.queue_time_us":12649,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"spinlock_wait_cycles":553484160,"thread_start_us":116102,"threads_started":1,"wal-append.queue_time_us":307884}
I20260709 13:03:20.090134  7366 raft_consensus.cc:2764] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:03:20.090665  7732 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 13:03:20.146449  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 103950us
I20260709 13:03:20.146710  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.147131  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.092309  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.148245  1629 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:20.149469  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.150688  1631 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:20.150949  7719 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.151057  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:20.152197  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.152908  7637 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.153445  7637 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.153591  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.153911  7637 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.974514  7098 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:03:20.151700  7719 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:20.156736  7388 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.158186  7673 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:20.158597  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 205523us
I20260709 13:03:20.107180  7333 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:20.127533  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:20.099848  1554 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.904s	user 0.004s	sys 0.025s
I20260709 13:03:20.108124  1701 rpcz_store.cc:276] 0709 13:03:18.739516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.580507 (+840991us) service_pool.cc:224] Handling call
0709 13:03:19.580972 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.991957 (+410985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.992350 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.151655  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.098114  7690 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.099073  7630 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.127317  6836 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.128181  1709 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:20.128530  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 141137us
I20260709 13:03:20.129210  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.129841  7290 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.131047  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.132355  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.136535  1704 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:20.116879  1551 rpcz_store.cc:276] 0709 13:03:18.498805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.942649 (+443844us) service_pool.cc:224] Handling call
0709 13:03:18.942950 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.137541 (+194591us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:19.183876 (+ 46335us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.754413 (+570537us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b18003b0260. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:20.052473 (+298060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.059693 (+  7220us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:20.059749 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.064955 (+  5206us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540036ecd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:20.065014 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.065221 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119884,"prepare.queue_time_us":54757,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":37,"raft.queue_time_us":163282,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":42581,"spinlock_wait_cycles":282106496,"thread_start_us":240824,"threads_started":2,"wal-append.queue_time_us":771909}
I20260709 13:03:20.138371  7654 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.143998  2290 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:20.144606  7716 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:20.159740  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.161391  1634 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
W20260709 13:03:20.160918  7743 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:20.160018  7333 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.114758  1703 rpcz_store.cc:276] 0709 13:03:18.849709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.602240 (+752531us) service_pool.cc:224] Handling call
0709 13:03:19.602725 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.607499 (+  4774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.607532 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.608045 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.993700 (+385655us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:19.993818 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.993980 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.059894 (+ 65914us) raft_consensus.cc:1685] finished
0709 13:03:20.059914 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.061567 (+  1653us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4145,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":321979,"threads_started":1,"wal-append.queue_time_us":447757}
I20260709 13:03:20.169322  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:20.203215  1554 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:20.203070  7290 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.204113  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 2780) took 1074 ms. Trace:
I20260709 13:03:20.204527  1554 rpcz_store.cc:276] 0709 13:03:19.129596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.129873 (+   277us) service_pool.cc:224] Handling call
0709 13:03:20.204019 (+1074146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.205590  2297 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:20.207096  7290 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.138453  7578 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:20.203147  1628 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:20.208526  1631 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:20.209192  1630 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:20.213363  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:20.207708  7578 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:20.214915  7237 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:20.215834  7578 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.216931  7316 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:20.217394  7694 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.217537  7316 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.217962  7578 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:20.218880  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 129277us
W20260709 13:03:20.226115  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66614.7us
I20260709 13:03:20.226678  6836 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:20.227871  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 77005us
I20260709 13:03:20.220223  7616 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:03:20.228636  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:20.219151  7366 raft_consensus.cc:2764] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:03:20.227530  7722 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:20.218160  7316 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.222399  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1024) took 1487 ms. Trace:
I20260709 13:03:20.223466  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:20.165292  7719 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.247962  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.182935  7716 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:20.239497  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:20.259152  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:20.247241  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.240335  1709 rpcz_store.cc:276] 0709 13:03:18.735305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.529869 (+794564us) service_pool.cc:224] Handling call
0709 13:03:20.127742 (+597873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.202571 (+ 74829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.222148 (+ 19577us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400042750. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:20.222291 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321515904}
I20260709 13:03:20.264823  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:20.269367  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46042us
I20260709 13:03:20.239614  7642 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:20.099534  7693 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.235598  7388 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.280827  2294 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:20.247520  7461 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:20.291946  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 146463us
W20260709 13:03:20.293790  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 94469.2us
I20260709 13:03:20.272835  7675 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.227644  7622 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 13:03:20.301581  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.276821  7642 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.311928  2295 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:20.312261  7606 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.005s
I20260709 13:03:20.312805  7606 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.314944  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.315824  7642 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.319558  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.319687  7719 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.320516  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:20.320830  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.322045  7461 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.322615  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.322718  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.323658  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:20.325361  7461 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.325716  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:20.326942  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49423.7us
I20260709 13:03:20.327469  7290 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:20.328035  7290 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:20.328579  7290 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:20.328920  7741 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.173s	user 0.003s	sys 0.000s
I20260709 13:03:20.330510  6934 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.330340  7649 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:20.332129  1629 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:20.333070  7649 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:20.333549  7649 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:20.334470  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43772.3us
I20260709 13:03:20.334583  7649 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.336321  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:20.336642  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1354) took 1939 ms. Trace:
I20260709 13:03:20.337066  7722 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.337741  7722 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:20.338132  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:20.338240  7722 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.337097  2295 rpcz_store.cc:276] 0709 13:03:18.397400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.577756 (+180356us) service_pool.cc:224] Handling call
0709 13:03:18.578104 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.312610 (+1734506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.336309 (+ 23699us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400771ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:20.336559 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3640596736}
I20260709 13:03:20.339151  1702 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:20.330508  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 977) took 1119 ms. Trace:
I20260709 13:03:20.341429  1555 rpcz_store.cc:276] 0709 13:03:19.211356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.241398 (+ 30042us) service_pool.cc:224] Handling call
0709 13:03:19.241844 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.242387 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.242426 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.243039 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.769168 (+526129us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18003f5860. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:19.954618 (+185450us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:19.961485 (+  6867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.049155 (+ 87670us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.327927 (+278772us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:20.330106 (+  2179us) raft_consensus.cc:1685] finished
0709 13:03:20.330129 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.330309 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":684166,"raft.queue_time_us":173,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12898,"spinlock_wait_cycles":59932288,"thread_start_us":164247,"threads_started":1,"wal-append.queue_time_us":871970}
I20260709 13:03:20.346493  7649 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:20.352085  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.352751  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:20.352116  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:20.353941  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:20.350442  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:20.354598  1709 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:20.302343  7675 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:20.336916  7588 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:03:20.355681  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:20.365952  7626 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.366786  7626 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.355979  7366 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:20.347623  7622 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:20.359964  7687 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:20.320611  7719 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:20.369174  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 137250us
I20260709 13:03:20.369897  6983 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.370904  7719 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.290006  7705 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.372619  7667 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.351258  1704 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:20.376533  7369 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.071s	user 0.010s	sys 0.017s
I20260709 13:03:20.377054  7369 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:20.377977  1708 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:20.378469  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 100937us
I20260709 13:03:20.378894  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:20.360867  7675 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.365646  2295 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:20.361305  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:20.367187  7626 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.364105  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.365082  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.351962  7690 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:20.380515  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.386857  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:20.378949  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:20.326649  7591 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.879s	user 0.008s	sys 0.011s
I20260709 13:03:20.372639  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.403801  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:20.404187  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48719.4us
I20260709 13:03:20.396836  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2327) took 1125 ms. Trace:
I20260709 13:03:20.331256  6934 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.408938  1709 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:20.405902  1708 rpcz_store.cc:276] 0709 13:03:19.271828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.836417 (+564589us) service_pool.cc:224] Handling call
0709 13:03:19.837189 (+   772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.378393 (+541204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.385667 (+  7274us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54001b7954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:20.396780 (+ 11113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1242713344}
I20260709 13:03:20.404781  2292 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:20.362215  7654 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.394019  7690 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.369571  7642 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.367892  7622 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:20.361901  7098 raft_consensus.cc:2764] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:20.388077  7626 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.420403  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.422751  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.433943  1629 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:20.419288  1704 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.427896  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:20.437943  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:20.442284  7630 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:20.442767  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.395591  7606 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:20.446991  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.448040  1635 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:20.437815  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:20.428356  7690 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.443306  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.445991  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:20.462239  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90579.8us
I20260709 13:03:20.456943  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.463686  2290 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:20.464967  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:20.444201  7388 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:20.468362  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.442842  7630 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.468888  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.468209  1560 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:20.476167  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.476951  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.483320  1626 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
W20260709 13:03:20.483965  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122850us
I20260709 13:03:20.409901  7591 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.490765  1702 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:20.469405  6983 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.498040  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.445556  7637 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.496382  1630 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:20.479579  7687 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:20.498766  1631 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:20.500886  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:20.472607  7630 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:20.499866  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.478168  7369 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:20.506225  7290 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:20.526592  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.530153  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.506829  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:20.508168  7377 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.011s
I20260709 13:03:20.508553  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.508718  7591 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.477180  7333 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.509706  6934 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.510216  1706 raft_consensus.cc:2368] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
I20260709 13:03:20.539412  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2323) took 1269 ms. Trace:
I20260709 13:03:20.539799  1706 rpcz_store.cc:276] 0709 13:03:19.270169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.440379 (+170210us) service_pool.cc:224] Handling call
0709 13:03:20.539168 (+1098789us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400132ed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:20.539343 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456871168}
I20260709 13:03:20.510135  7622 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.541391  7622 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.542094  7333 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.542834  7687 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:20.543995  7687 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.546419  7290 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.547366  7377 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.512075  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.514616  7637 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:20.515751  7396 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.005s
I20260709 13:03:20.551936  1629 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:20.479621  7642 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.522859  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.547904  1555 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:20.512460  2291 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:20.549233  1556 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:20.551920  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.521410  2292 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.531145  7369 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.534878  7654 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:20.564106  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:20.558960  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.574245  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.549024  7388 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:20.577576  2297 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:20.578050  1709 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:20.580874  2290 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:20.581856  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.583473  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.535300  7649 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:20.590243  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:20.590602  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:20.585930  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:20.569128  7369 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.568789  7654 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.600922  2294 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.579591  7333 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:20.604427  1629 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:20.492565  7716 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.617386  1559 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:20.600922  7654 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:20.590255  7388 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:20.643580  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 150746us
I20260709 13:03:20.608433  7333 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:20.648483  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.650087  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.660250  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.665943  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.648938  7333 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:20.673357  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:20.666675  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.645180  7642 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:20.667829  7649 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:20.669607  7591 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:20.669974  7756 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:20.660084  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.678262  7649 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:20.647274  7316 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.725445  7316 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.732370  7377 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.734001  7722 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.684546  7642 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:20.739977  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.740660  7642 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.741110  1555 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:20.742193  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.742982  2297 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:20.743093  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.744067  1555 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
W20260709 13:03:20.748365  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.693092  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.693449  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.688298  7756 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.693424  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:20.698160  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 130442us
I20260709 13:03:20.689930  7591 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.691388  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.704582  7388 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.692198  6983 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:20.763252  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.720706  7649 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.754024  1630 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:20.763947  1560 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:20.757844  2294 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:20.759143  1702 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:20.760803  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.761478  7756 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.762250  7716 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.711511  7366 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.763198  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.763221  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:20.769645  1556 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:20.754817  1704 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:20.777604  1552 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:20.780057  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.781025  1553 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:20.781302  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.781980  2297 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.782001  1555 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:20.785645  1557 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:20.788226  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:20.789108  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.792517  1556 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:20.794530  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.787358  2292 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:20.798267  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.806042  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.806751  1556 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:20.808066  1556 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:20.810359  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:20.748031  7675 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.812047  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.812721  1553 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.812968  7675 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.796103  6983 raft_consensus.cc:2764] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:03:20.816458  6983 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:20.816962  6983 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:20.817870  6983 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.818987  7333 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.826828  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.828239  1560 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 9.
I20260709 13:03:20.803046  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.829021  1556 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:20.831315  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:20.832767  1551 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.838387  1557 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:20.841020  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.825241  7722 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.845230  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:20.845325  7690 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.806955  7591 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:20.845160  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:20.845212  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:20.827904  7649 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:20.857452  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 178782us
I20260709 13:03:20.841796  1555 raft_consensus.cc:2376] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17. Re-sending same reply.
I20260709 13:03:20.804045  7756 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:03:20.870311  1552 pending_rounds.cc:85] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 13
I20260709 13:03:20.870716  1552 pending_rounds.cc:107] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:03:20.871031  1552 raft_consensus.cc:2889] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:20.871747  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.872079  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:20.853477  6934 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:20.876152  7722 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:20.876626  7722 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.877032  7722 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:20.877161  7637 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.877816  7637 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:20.878249  7637 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.878451  1525 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.879354  7719 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:20.848428  7098 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:20.880713  7606 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:20.881137  7098 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.848685  7630 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.882323  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:20.888031  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:20.895731  2292 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.886031  1558 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:20.897632  1551 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
W20260709 13:03:20.902361  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 139254us
I20260709 13:03:20.905750  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:20.908381  2293 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:20.825647  7654 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.867062  7591 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:20.856343  7377 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:20.921756  1553 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:20.887156  7630 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.886930  7690 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.890250  7333 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.883729  7098 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.856168  7366 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.923329  6934 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:20.892691  7637 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.948124  2297 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:20.945473  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.955368  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:20.961793  7626 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:03:20.962333  7626 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.944654  7316 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.964365  7316 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:20.966483  7316 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:20.971733  7626 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.980109  7626 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:20.927924  7377 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:20.990300  7377 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.992614  7637 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:20.952947  7071 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:20.993695  7071 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:20.994728  7071 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:20.993700  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.999876  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:21.002912  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.004385  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:21.017930  7654 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.023084  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:21.023903  2297 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:21.027757  7366 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:21.029264  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2324) took 1758 ms. Trace:
I20260709 13:03:21.029660  1705 rpcz_store.cc:276] 0709 13:03:19.270457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.616062 (+345605us) service_pool.cc:224] Handling call
0709 13:03:19.616382 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.313256 (+696874us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:20.589487 (+276231us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.899552 (+310065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.899612 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:20.899663 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.028816 (+129153us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:21.028866 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.029063 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372184,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":958,"raft.queue_time_us":48502,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":1551164672,"thread_start_us":539454,"threads_started":2,"wal-append.queue_time_us":430577}
I20260709 13:03:21.031230  2297 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:20.953498  7630 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:21.034469  7630 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:21.034973  7630 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:21.040763  7759 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:03:21.041432  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.043201  6934 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.045933  7578 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.050021  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:21.048673  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:20.945688  7098 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.055920  2292 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:21.046996  7690 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:21.058552  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:21.071779  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.079370  1557 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:21.080251  1556 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:21.062429  7098 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:21.083292  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.057265  7654 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:21.068013  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 454646us
W20260709 13:03:21.088855  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108158us
I20260709 13:03:21.090148  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:21.072326  7366 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:21.098937  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 253861us
I20260709 13:03:21.082754  7649 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:21.083175  7690 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:21.054047  7578 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } attrs { replace: true } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.103968  6836 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.106456  7535 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:03:21.117178  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.118669  7388 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:21.120334  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.121165  7388 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:21.121268  7578 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.122699  7722 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.125424  7098 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:21.126487  7098 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:21.127140  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.128795  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.097179  7377 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:21.130934  7535 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:21.098205  7654 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:21.134002  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43984.6us
I20260709 13:03:21.135563  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:21.131491  7290 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:21.113637  7649 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:21.132393  7369 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.001s
I20260709 13:03:21.132791  7535 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:21.146482  7290 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:21.156781  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 274497us
I20260709 13:03:21.133010  6836 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.153091  7642 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:21.168906  7722 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:21.163836  7535 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.155488  7626 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.156008  7290 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:21.177977  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 285363us
W20260709 13:03:21.203348  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 67881us
W20260709 13:03:21.206734  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 301065us
I20260709 13:03:21.150811  7369 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.221220  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:21.234236  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.244115  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:21.262183  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 133483us
I20260709 13:03:21.166725  7654 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.166669  7690 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.263231  7535 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.280011  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:21.280342  1557 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:21.266651  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.268563  7675 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:21.256577  7773 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:21.235409  7722 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:03:21.274637  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.286945  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:21.272271  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:21.278486  7767 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:21.278807  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.291195  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:21.291489  1629 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:21.261219  7756 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:21.284054  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:21.273540  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:21.296041  1628 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:21.298019  2297 raft_consensus.cc:2368] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:21.285667  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:21.276090  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:21.288765  1556 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:21.290032  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55877.2us
I20260709 13:03:21.301118  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:21.284972  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:21.279111  1634 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:21.291811  1554 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:21.301950  2295 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:21.235348  7626 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.269403  7098 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.291270  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:21.314750  1552 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:21.292179  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:21.296736  7761 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:03:21.302762  2295 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:21.282464  7675 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:21.372226  7675 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.374171  7675 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:21.237753  7649 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.295192  7767 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.379242  7767 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:21.319592  7772 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:21.295635  7782 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:03:21.321103  7770 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:21.325680  7369 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:21.327188  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47262.1us
I20260709 13:03:21.327838  2293 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:21.328663  7716 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:21.333003  7768 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:03:21.386085  7716 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.385882  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:21.386613  7716 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:21.319290  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 87595.1us
I20260709 13:03:21.317819  7773 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:21.303810  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:21.302814  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.296711  1557 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:21.316130  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:21.922456  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 615952us
I20260709 13:03:21.324512  1628 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:21.927008  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.928020  7690 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:21.928494  7690 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.928845  7690 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:21.328742  1634 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:21.933674  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.315461  1631 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:21.681675  1629 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:21.415825  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:21.606552  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:21.615384  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:21.613024  1703 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:21.605618  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:22.050613  2260 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.821s (SENT)
I20260709 13:03:21.606456  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.051908  1704 raft_consensus.cc:2376] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19. Re-sending same reply.
I20260709 13:03:21.605224  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.053239  1705 raft_consensus.cc:2376] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15. Re-sending same reply.
I20260709 13:03:22.054527  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:21.730919  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.056245  1557 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:21.731308  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 340172us
I20260709 13:03:22.057605  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:22.059059  7782 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:22.059477  7782 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:21.737644  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:22.059892  7782 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.061851  7333 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:21.614413  1707 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:22.065048  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.065686  1707 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:21.740703  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 426876us
I20260709 13:03:21.607481  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.068528  7782 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.069516  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.069801  1598 leader_election.cc:400] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
I20260709 13:03:21.625284  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.070895  7782 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.742375  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.071705  1555 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:21.749533  1524 leader_election.cc:400] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:22.072578  1524 leader_election.cc:403] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:03:22.073125  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 773757us
W20260709 13:03:21.657855  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 70953.9us
I20260709 13:03:21.316269  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:21.789110  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:21.789496  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.088240  1560 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:21.368781  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:22.090233  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:21.611364  6934 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.092856  6934 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.353821  6836 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:21.812572  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.354524  7098 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.096840  7719 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.098208  7098 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.480294  7290 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.101449  7290 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.357586  7388 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:21.462607  7591 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:22.105340  7591 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:22.106046  1524 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:22.106441  1524 leader_election.cc:403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:21.350652  7630 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.422420  7716 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.126066  7716 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.126565  7630 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:22.126848  7716 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.127501  7716 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.129923  7716 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:21.445811  7694 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:03:21.397094  7369 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:21.366690  7626 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.317656  7637 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.141753  7637 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.142285  7637 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.730983  7722 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.297525  7756 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:21.609874  7767 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:21.408411  7773 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.150110  7767 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:21.296576  7777 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.151513  7777 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.151930  7777 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.712487  7606 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:21.650913  7772 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:22.157801  7772 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.866518  6983 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:21.899025  1627 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:21.900484  7687 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.160060  6983 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.160430  7772 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.160770  7687 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.162144  6983 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.162809  7687 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:21.901577  1632 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:21.902132  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.902796  7675 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:03:21.903858  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.903592  7667 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:21.912417  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:21.924074  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:21.938781  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:21.951920  7622 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.939980  7690 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:21.962822  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:21.944924  1628 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:21.983305  1634 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:21.983669  7797 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:22.170255  1631 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:22.005963  1629 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:22.014787  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1007) took 1451 ms. Trace:
W20260709 13:03:22.018607  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.039995  7798 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:22.040724  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 638536us
I20260709 13:03:22.048175  1700 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:22.051169  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.052198  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.070164  1598 leader_election.cc:403] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:03:22.093986  1523 leader_election.cc:400] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:22.079552  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.094101  1708 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:22.096127  7071 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:22.097396  7719 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.098989  6934 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:22.111413  1554 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:22.116113  1707 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:22.181602  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:22.117048  7578 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.106240  7591 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.131140  7716 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:22.133252  7694 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:22.134482  1702 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:22.144716  7722 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.128685  1452 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 15, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:22.157002  7606 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:22.147529  7756 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.208576  7716 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.225608  7716 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:22.228051  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.228839  1633 raft_consensus.cc:2368] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
W20260709 13:03:22.165208  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 848635us
I20260709 13:03:22.169937  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.238966  1634 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:21.993302  7366 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.183507  1631 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 13: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:22.186762  1628 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:22.188172  7802 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:22.192199  2296 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:22.241709  7802 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:22.242215  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:22.242866  2296 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:22.242746  6983 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:22.193518  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:22.194525  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:22.242266  7802 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.203568  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:22.168620  7675 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:22.246547  7675 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:22.247674  7802 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.247406  7675 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.248713  7675 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.202248  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:22.207744  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.170923  1635 rpcz_store.cc:276] 0709 13:03:20.563310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.574295 (+ 10985us) service_pool.cc:224] Handling call
0709 13:03:20.585762 (+ 11467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:20.668244 (+ 82482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.668286 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:20.669130 (+   844us) log.cc:844] Serialized 52 byte log entry
0709 13:03:21.059847 (+390717us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:21.072719 (+ 12872us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.072987 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.073059 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.206558 (+133499us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b180017e9c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:22.014419 (+807861us) raft_consensus.cc:1685] finished
0709 13:03:22.014530 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.014721 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57315,"prepare.queue_time_us":11503,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":59821,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":15447,"spinlock_wait_cycles":12212608,"thread_start_us":390298,"threads_started":1,"wal-append.queue_time_us":506220}
I20260709 13:03:22.211470  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:22.212354  1708 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:22.253991  6983 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:22.213680  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:22.254532  6983 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:22.214358  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 290556us
I20260709 13:03:22.254899  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.255523  6983 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.256135  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.256174  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.257423  7369 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:22.257826  7772 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:22.257947  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.176183  7388 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.258661  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.258847  7388 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.259415  7772 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:22.259851  7772 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.260174  7772 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:22.261037  7388 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:22.221158  7606 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.222168  7098 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:22.222980  7654 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.261863  7606 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.222185  6836 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.264253  6836 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.266396  1553 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:22.266675  6836 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.196074  2288 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:22.267014  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:22.267669  1554 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.270913  7772 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.272475  1560 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:22.274080  1555 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:22.274905  7654 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
W20260709 13:03:22.275058  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.275267  7369 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.275866  1557 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:03:22.277632  1556 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:22.279021  7772 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.280440  7377 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:22.285440  1630 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:22.254855  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:22.275928  7369 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.288403  7377 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.289234  7377 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.291018  7369 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.292742  7377 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:22.297891  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.299494  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:22.301590  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.304684  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.306208  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.307080  1556 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:22.308121  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:22.308763  1555 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:22.316309  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.317283  1557 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:22.318343  7316 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:22.320495  7316 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:22.323236  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65356.9us
I20260709 13:03:22.324208  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.326876  7388 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:22.327462  7388 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:22.330022  7290 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:22.330535  7290 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.330907  7290 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:22.338701  7377 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:22.339727  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.340632  1557 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:22.342109  7377 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:22.343058  7377 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:22.343439  7377 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:22.345204  7377 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.338588  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.368798  1556 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:22.368481  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.173393  7622 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.207172  7694 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:22.219689  6934 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:22.393087  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.393272  1555 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:22.176537  7797 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.201973  7687 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.167s
I20260709 13:03:22.381763  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.368573  7591 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:22.423105  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 353421us
I20260709 13:03:22.400520  1633 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:22.426436  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 366674us
W20260709 13:03:22.427400  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55019.1us
I20260709 13:03:22.399354  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.414757  1554 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:22.431044  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:22.431990  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 382063us
W20260709 13:03:22.433555  2260 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.918s (SENT)
I20260709 13:03:22.433962  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:22.445508  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.455003  1629 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:22.462121  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.462878  1703 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:22.431183  7377 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:22.471791  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.479395  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84475.8us
I20260709 13:03:22.480497  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:22.395752  7798 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:03:22.395596  7637 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.446663  7772 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:22.496455  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:22.499286  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50895.3us
I20260709 13:03:22.453933  7369 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:22.504479  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:22.505270  6836 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:22.505533  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:22.505833  6836 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:22.506309  7098 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:22.506845  7098 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:22.506779  6836 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.394811  7366 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.526991  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.462669  7802 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:22.528663  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48273.3us
I20260709 13:03:22.392812  7626 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.103s
I20260709 13:03:22.376433  7388 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.532087  1705 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:22.534698  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76241.7us
I20260709 13:03:22.475919  7801 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:22.481920  7675 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.537062  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 103523us
I20260709 13:03:22.541301  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.547652  1707 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:22.509601  7772 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.497630  7654 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:22.553220  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.545030  6983 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.566854  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.542496  7801 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.535557  7690 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:22.560705  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 98563.1us
W20260709 13:03:22.584833  7803 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:03:22.585041  7801 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.584918  1633 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
W20260709 13:03:22.592549  7807 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:22.593411  7316 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:03:22.595148  7637 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.597771  7767 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:22.598246  7767 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:22.599124  7767 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.579785  6983 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.605584  1706 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:22.618072  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:22.593819  7808 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.115s	user 0.003s	sys 0.000s
I20260709 13:03:22.558259  7772 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:22.638563  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1040) took 1032 ms. Trace:
W20260709 13:03:22.664394  7804 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:03:22.665190  7797 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.665685  7814 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:22.666795  7578 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 14.
I20260709 13:03:22.664027  1706 rpcz_store.cc:276] 0709 13:03:21.606172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.606282 (+   110us) service_pool.cc:224] Handling call
0709 13:03:22.638365 (+1032083us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540016bf54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:22.638496 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874039168}
I20260709 13:03:22.666455  1452 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 15, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:22.670619  1703 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:22.581233  7687 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:22.546761  7675 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.613217  7369 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:22.548218  7777 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.691404  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:22.679616  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 136227us
W20260709 13:03:22.704833  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.738389  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.740304  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.757865  1700 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.542135  7694 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.546366  7098 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.759793  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.771162  1708 raft_consensus.cc:2368] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:22.772387  1633 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:03:22.761533  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 232427us
I20260709 13:03:22.760727  1703 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:22.781030  1701 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:22.782312  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 220931us
I20260709 13:03:22.784137  1629 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:22.785494  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:22.755730  7716 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.005s
I20260709 13:03:22.787564  1630 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:22.790267  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.790915  1704 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:22.783128  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.643579  7290 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.794605  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.755921  7719 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.799479  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
W20260709 13:03:22.803977  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.810765  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.776803  7687 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:22.813370  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.814908  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.691638  7802 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:03:22.811685  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.815743  1552 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:22.822638  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1050) took 1181 ms. Trace:
I20260709 13:03:22.824915  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.826332  1633 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:22.828862  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.804487  7626 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.837248  1704 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:22.838107  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.837536  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.802963  7622 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:22.845021  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.848671  7813 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:22.849414  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.850414  1633 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:22.857049  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:22.861248  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106070us
I20260709 13:03:22.825138  7396 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:22.792520  7716 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.865085  6836 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:22.842550  7767 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:22.828138  7802 consensus_queue.cc:1243] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:03:22.868597  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 264496us
I20260709 13:03:22.868724  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.872484  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.852332  7690 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:22.848721  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08003166e0 after lost signal to thread 431
I20260709 13:03:22.849473  7813 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.876825  1551 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:22.881844  1560 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:22.883688  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.884586  1552 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:22.878957  1553 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:22.879457  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:22.876554  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08003221e0 after lost signal to thread 434
W20260709 13:03:22.897338  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08002d5f20 after lost signal to thread 1492
W20260709 13:03:22.899179  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800330e60 after lost signal to thread 1584
I20260709 13:03:22.901026  1703 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:22.883104  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.907217  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.908162  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:22.851379  7626 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.876344  7690 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.910605  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:22.916664  1559 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.910802  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:22.917508  1551 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.915273  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.875398  7818 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:22.872097  7675 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:22.886955  7813 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.900381  7622 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.877676  6934 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:22.931360  1674 leader_election.cc:400] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:22.943033  1674 leader_election.cc:403] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:22.944428  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:22.931960  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.946632  1703 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:22.929603  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.936770  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.948210  1628 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.938462  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.863255  7369 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:22.943065  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.920676  7690 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:22.960086  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.955178  7396 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:22.959750  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:22.964768  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:22.949254  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 136117us
W20260709 13:03:22.941605  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 211) took 28060 ms (client timeout 15000 ms). Trace:
I20260709 13:03:22.965348  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:22.977938  1632 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:22.969151  1626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:22.927981  7818 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.971297  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:22.975466  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:22.998176  1634 raft_consensus.cc:2410] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:22.975914  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:22.975245  7578 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:22.992326  7290 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:22.980041  1626 raft_consensus.cc:606] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:23.010948  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:23.011324  1551 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:22.930408  7801 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:22.972445  7798 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.938738  7637 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:22.970834  7675 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:23.022522  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.027387  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.014786  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:23.028810  1559 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
W20260709 13:03:23.029294  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 138228us
I20260709 13:03:22.994287  7649 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:22.964514  7606 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:22.977015  7773 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:22.951506  7622 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.002229  7818 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:23.038559  1632 raft_consensus.cc:2376] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17. Re-sending same reply.
I20260709 13:03:23.053330  2260 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:22.862901  1452 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 18, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:23.006718  7777 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.003144  7823 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:23.010387  7667 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:23.057128  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.111829  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:23.059406  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:23.118395  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:23.119889  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:22.942337  7687 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.100731  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:23.049254  7756 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.825s	user 0.011s	sys 0.000s
I20260709 13:03:23.122434  7756 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.125342  7756 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.126214  7756 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:23.097234  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1045) took 1458 ms. Trace:
I20260709 13:03:23.127213  1701 rpcz_store.cc:276] 0709 13:03:21.639292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.642376 (+  3084us) service_pool.cc:224] Handling call
0709 13:03:21.643638 (+  1262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.782789 (+1139151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.060727 (+277938us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001b8854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:23.097163 (+ 36436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2615229824}
W20260709 13:03:23.128895  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75617.6us
I20260709 13:03:23.129112  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:23.129974  1701 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:23.132035  1709 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:23.104926  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.132989  1629 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:23.097096  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:23.142102  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.142882  1630 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:23.044643  7675 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:23.143558  1555 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:23.033651  7333 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:03:23.150409  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:23.021075  7578 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:23.116123  7810 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 13:03:23.158818  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 282587us
I20260709 13:03:23.055213  7815 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:03:23.117348  7823 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:23.054973  7649 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.145908  7675 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:23.144143  7071 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:03:23.183537  7845 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:23.181638  7716 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.163568  7396 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.205078  1627 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
W20260709 13:03:23.184442  7591 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.756s	user 0.013s	sys 0.000s
I20260709 13:03:23.195313  7316 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:23.187165  7823 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:23.202708  7813 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.141760  7773 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:23.208042  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72335.2us
W20260709 13:03:23.213081  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 112469us
I20260709 13:03:23.208511  1635 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:23.217285  1628 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:23.184010  7819 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:03:23.219485  7833 raft_consensus.cc:2764] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:03:23.223165  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9120 after lost signal to thread 1659
W20260709 13:03:23.305645  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08000e1380 after lost signal to thread 1660
W20260709 13:03:23.306840  1656 debug-util.cc:398] Leaking SignalData structure 0x7b080040c820 after lost signal to thread 1733
W20260709 13:03:23.307152  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800347560 after lost signal to thread 1734
I20260709 13:03:23.146216  6934 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:23.311184  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08002328e0 after lost signal to thread 2321
W20260709 13:03:23.314030  1656 debug-util.cc:398] Leaking SignalData structure 0x7b080035fb60 after lost signal to thread 2322
I20260709 13:03:23.210697  7845 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:23.226907  7626 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.328346  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1150) took 1045 ms. Trace:
I20260709 13:03:23.327173  7606 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:23.336256  7606 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:23.337292  7606 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.338639  7773 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.339295  7773 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.223701  1700 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:03:23.301081  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.977378  1455 rpcz_store.cc:269] 0709 13:02:54.860865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.861063 (+   198us) service_pool.cc:224] Handling call
0709 13:03:22.911527 (+28050464us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:19.212532 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:19.389917 (+177385us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:19.398544 (+  8627us) write_op.cc:435] Acquired schema lock
0709 13:03:19.419827 (+ 21283us) tablet.cc:662] Decoding operations
0709 13:03:19.421501 (+  1674us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:19.421599 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:19.421627 (+    28us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:03:19.433632 (+ 12005us) tablet.cc:701] Row locks acquired
0709 13:03:19.433682 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:03:19.433944 (+   262us) write_op.cc:270] Start()
0709 13:03:19.434629 (+   685us) write_op.cc:276] Timestamp: P: 1783602199433840 usec, L: 0
0709 13:03:19.434830 (+   201us) op_driver.cc:348] REPLICATION: starting
0709 13:03:19.440717 (+  5887us) log.cc:844] Serialized 8193 byte log entry
0709 13:03:20.358239 (+917522us) op_driver.cc:464] REPLICATION: finished
0709 13:03:20.845541 (+487302us) write_op.cc:301] APPLY: starting
0709 13:03:20.845665 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:21.053667 (+208002us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:21.075841 (+ 22174us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:21.342569 (+266728us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:21.343218 (+   649us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:03:21.343306 (+    88us) write_op.cc:312] APPLY: finished
0709 13:03:21.350379 (+  7073us) log.cc:844] Serialized 188 byte log entry
0709 13:03:21.495054 (+144675us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:21.650879 (+155825us) write_op.cc:454] Released schema lock
0709 13:03:21.650961 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:21.651528 (+   567us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:21.651560 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7204939,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":445449,"apply.run_cpu_time_us":69069,"apply.run_wall_time_us":805959,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91991,"num_ops":21,"prepare.queue_time_us":344604,"prepare.run_cpu_time_us":28436,"prepare.run_wall_time_us":536820,"raft.queue_time_us":366440,"raft.run_cpu_time_us":79229,"raft.run_wall_time_us":246182,"replication_time_us":923157,"thread_start_us":1012655,"threads_started":5,"wal-append.queue_time_us":685291,"wal-append.run_cpu_time_us":69946,"wal-append.run_wall_time_us":2258250}]]}
I20260709 13:03:23.196362  7841 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:23.224784  1701 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:23.320142  7722 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:23.383379  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:23.342898  7821 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 13:03:23.349555  7817 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:03:23.308192  7316 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.331317  2288 rpcz_store.cc:276] 0709 13:03:22.283134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.283320 (+   186us) service_pool.cc:224] Handling call
0709 13:03:22.283682 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:22.284203 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.284236 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:22.284894 (+   658us) log.cc:844] Serialized 52 byte log entry
0709 13:03:22.559123 (+274229us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:23.023617 (+464494us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.116163 (+ 92546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.116253 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.328003 (+211750us) raft_consensus.cc:1685] finished
0709 13:03:23.328037 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.328251 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":624389,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":169246,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":18612,"spinlock_wait_cycles":16077568,"thread_start_us":747438,"threads_started":3,"wal-append.queue_time_us":411688}
I20260709 13:03:23.391891  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:23.338608  7606 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:23.386323  7377 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.905s	user 0.011s	sys 0.000s
I20260709 13:03:23.345005  7854 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:23.349674  7787 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.342182  6983 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.407853  7814 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:03:23.339939  1452 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 14, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:23.345880  7622 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:23.414801  7822 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
I20260709 13:03:23.349318  7366 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.339742  7773 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:23.473554  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 471438us
W20260709 13:03:23.381091  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 421797us
W20260709 13:03:23.441021  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 264977us
W20260709 13:03:23.474947  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.412531  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 353148us
I20260709 13:03:23.390112  7578 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:23.443454  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134900us
I20260709 13:03:23.478857  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:23.445917  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:23.482776  2288 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:23.465921  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 155585us
W20260709 13:03:23.460852  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 416360us
I20260709 13:03:23.391402  7801 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.471155  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:23.342751  7802 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:03:23.482018  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:23.416709  7844 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.458606  6934 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.411176  7813 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.513984  2294 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:23.476649  7845 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.476984  1452 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 15, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.484572  1701 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.468377  1705 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:23.486440  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:23.488200  7853 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.488535  7814 consensus_queue.cc:1243] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:23.489095  7578 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:23.488703  7396 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:23.491425  7833 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:23.471387  7810 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:03:23.420267  7854 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:23.385308  7832 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 13:03:23.412793  7841 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:23.487824  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:23.495865  1700 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:03:23.475533  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.501969  7773 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.502099  7802 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:03:23.502839  1445 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.009s	sys 0.005s
W20260709 13:03:23.504824  7772 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.000s
I20260709 13:03:23.397964  7722 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:23.509392  7856 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.599182  7856 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.509433  7366 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.601696  7856 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.602632  7797 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:23.603144  7797 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:23.599028  7675 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:23.606321  7675 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.565181  7847 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:23.513528  7801 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.609951  7801 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.610666  7366 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.612326  7801 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:23.527278  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:23.613464  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86469.8us
W20260709 13:03:23.613605  7863 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:23.613865  7716 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:23.614441  7716 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:23.528432  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:23.616586  7847 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:23.617785  7847 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.536428  2295 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:23.621183  7854 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.622566  7854 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.506630  6836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.007s	sys 0.011s
W20260709 13:03:23.538376  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52013.2us
W20260709 13:03:23.623275  6836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.007s	sys 0.012s
I20260709 13:03:23.468796  7787 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:23.582012  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106587us
I20260709 13:03:23.625312  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.626147  6836 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:23.537779  7396 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.630292  7396 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:23.589068  1445 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.010s	sys 0.005s
I20260709 13:03:23.510448  7825 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:03:23.636629  7825 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:23.637014  7825 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:23.637917  7825 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.639200  7825 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.640077  7316 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.593888  7842 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:23.583559  7649 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.588179  7802 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:22.823873  1702 rpcz_store.cc:276] 0709 13:03:21.640992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.369350 (+728358us) service_pool.cc:224] Handling call
0709 13:03:22.377275 (+  7925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.377535 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:22.538243 (+160708us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.822279 (+284036us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.822334 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:22.822381 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.822447 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.822569 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":176180,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":1649,"thread_start_us":393928,"threads_started":2}
I20260709 13:03:23.641937  7871 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:23.641902  7802 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.642581  7871 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:23.643162  7802 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:23.643527  7871 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.644857  7871 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.645613  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.638842  7690 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.644s	user 0.004s	sys 0.004s
I20260709 13:03:23.608264  7841 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.562810  7814 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.569007  7767 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.001s
I20260709 13:03:23.612742  7366 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:23.494975  7844 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:23.530773  7667 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:23.623832  7722 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:23.526101  7853 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:23.627862  6983 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.628494  7772 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:23.629544  7824 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.487s	user 0.003s	sys 0.000s
I20260709 13:03:23.623610  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:23.597561  7848 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:23.647629  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:23.690680  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67166.4us
I20260709 13:03:23.690819  2292 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:23.691650  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:23.691862  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:23.692469  2292 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:23.693095  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:23.521096  6934 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.562538  7578 consensus_queue.cc:1243] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:03:23.659126  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:23.672466  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 184737us
I20260709 13:03:23.676122  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:23.680392  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:23.677000  1633 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:23.681885  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.684844  7823 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.695144  1701 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:23.665043  7777 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:23.689101  6836 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.694635  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:23.615411  7716 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:23.718273  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 201583us
I20260709 13:03:23.687619  7853 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.700525  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1039) took 2094 ms (client timeout 1727 ms). Trace:
I20260709 13:03:23.564172  7855 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.153s
I20260709 13:03:23.699127  7767 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:23.671967  7844 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.676245  7366 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:03:23.728538  2293 raft_consensus.cc:2368] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:23.728848  2290 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:23.729898  1708 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:23.724148  1706 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:23.700857  1704 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:23.731194  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:23.793524  1709 rpcz_store.cc:269] 0709 13:03:21.605812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.624840 (+ 19028us) service_pool.cc:224] Handling call
0709 13:03:23.700306 (+2075466us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400042c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:23.700454 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440095616}
I20260709 13:03:23.794534  1707 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:23.794715  1523 leader_election.cc:400] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:23.794787  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:23.541635  7833 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.811743  6934 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:23.812932  1701 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:23.680122  7637 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:23.677438  7814 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.686791  7071 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:23.818058  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 123493us
I20260709 13:03:23.722847  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.694908  7848 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:23.838411  1523 leader_election.cc:403] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.894260  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:23.894610  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 163511us
I20260709 13:03:23.840669  1705 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:23.699381  7649 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:23.882397  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.908629  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.857834  2296 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:23.670321  7690 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:23.858335  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:23.822676  7798 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.780s	user 0.011s	sys 0.000s
I20260709 13:03:23.735023  7860 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.872032  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.730926  7813 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.895516  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:23.892293  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:23.907534  1700 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:03:24.004537  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 90709.6us
I20260709 13:03:24.008263  2294 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:23.736358  7777 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:23.899906  7831 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.048705  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.815001  7756 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
W20260709 13:03:24.072995  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 201075us
W20260709 13:03:24.040804  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 318035us
I20260709 13:03:24.042569  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 2971) took 1179 ms. Trace:
I20260709 13:03:24.168452  1633 rpcz_store.cc:276] 0709 13:03:22.863518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.864314 (+   796us) service_pool.cc:224] Handling call
0709 13:03:24.042326 (+1178012us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b540019cb54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.042484 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812675328}
W20260709 13:03:24.169327  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.045692  7819 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:23.820785  7369 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.132s
W20260709 13:03:24.055740  2288 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.002s
I20260709 13:03:24.170687  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.170974  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.175241  2288 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:24.178422  2293 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:24.060286  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.565246  1452 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 18, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.178505  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:24.220732  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 492746us
I20260709 13:03:24.251327  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.260044  2288 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:24.022450  7798 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:24.007494  7622 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.049413  7858 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:23.954972  7637 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:24.049052  7834 raft_consensus.cc:2764] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:24.046080  7840 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:24.060557  7851 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:24.262632  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.264972  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86528.7us
I20260709 13:03:24.058486  7290 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.266204  7366 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.261263  2262 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:24.070972  7333 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.056373  7820 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:24.269312  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 536483us
I20260709 13:03:24.269737  2293 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:24.024618  6934 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:24.072570  7852 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.854919  7722 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.070812  7578 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:24.288524  7773 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:24.320086  7773 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:24.321337  7773 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:24.322242  7818 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.214s	user 0.005s	sys 0.010s
I20260709 13:03:24.322705  7818 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:24.305866  7871 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
I20260709 13:03:24.315884  7851 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.268810  7626 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:24.326165  7626 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.326506  7316 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.326723  7626 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.327081  7316 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.328320  7787 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.328876  7787 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.329171  7316 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.329319  7787 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.331430  7787 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.331887  7890 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.332500  1629 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:24.294140  1627 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:03:24.048228  7850 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.571s
I20260709 13:03:24.059945  7801 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:23.921378  7667 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.305049  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:24.267684  7815 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:03:24.387069  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:24.046289  7829 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.559s
I20260709 13:03:24.250342  7819 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.293129  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.389789  1626 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:24.316291  1633 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:24.073947  7098 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.221s
W20260709 13:03:24.319193  6983 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.003s
I20260709 13:03:24.320140  7722 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:24.048897  7843 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:24.076560  7767 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:24.294323  7694 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:24.303200  7396 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.003s
I20260709 13:03:24.333993  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.360116  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:24.347738  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.326107  7851 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.166179  7860 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.177583  7831 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.295789  7591 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.344408  7868 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:24.301384  7622 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.332592  7890 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.387718  1627 raft_consensus.cc:606] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
W20260709 13:03:24.389717  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84741.1us
I20260709 13:03:24.426816  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:24.305832  7290 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.314761  7333 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.381522  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:03:24.372370  7847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.006s	sys 0.043s
I20260709 13:03:24.370010  7820 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.296579  7858 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.428977  7687 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.441430  7820 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.441804  7687 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.382403  7772 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.399569  1632 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.399585  7369 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:24.444725  7369 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:24.446652  7825 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.000s
I20260709 13:03:24.446738  7369 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:24.406035  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46046.2us
W20260709 13:03:24.407053  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 145840us
I20260709 13:03:24.407712  7316 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:24.270349  7848 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:24.413614  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 518167us
I20260709 13:03:24.455705  1674 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:24.414114  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 521898us
I20260709 13:03:24.405997  7871 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:24.398756  6983 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:24.399003  7843 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.400825  7823 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.398809  7649 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:24.408327  7862 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:24.450116  1632 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:24.387291  7852 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:24.319690  1452 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:24.389609  7801 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.463506  1559 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:24.504541  1559 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:24.462226  7071 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.448175  7847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.007s	sys 0.043s
I20260709 13:03:24.473598  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:24.460884  7880 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.514s	user 0.000s	sys 0.001s
I20260709 13:03:24.478302  6836 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.480120  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:24.451660  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.484194  1673 leader_election.cc:400] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:24.490747  1553 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:24.490954  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.492141  1554 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:24.492027  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.525278  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.473485  7316 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:24.526180  1553 raft_consensus.cc:2410] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:03:24.472906  7772 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:24.526489  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.524935  1673 leader_election.cc:403] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:24.507054  1556 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:24.444445  7890 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.462512  7871 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.461874  7814 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.421427  7591 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.545130  7819 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.546788  7591 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.505632  1559 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.521800  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.534279  7818 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.548105  7716 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:24.397706  7637 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.525990  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.551397  7820 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.475553  7843 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:24.470950  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44251.3us
W20260709 13:03:24.554945  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.408128  7396 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.556981  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:24.557981  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1013) took 1606 ms (client timeout 1819 ms). Trace:
I20260709 13:03:24.524921  1552 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 9.
I20260709 13:03:24.524277  2293 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:24.558432  1634 rpcz_store.cc:269] 0709 13:03:22.951754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.023177 (+ 71423us) service_pool.cc:224] Handling call
0709 13:03:24.557732 (+1534555us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001e8554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.557889 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3070770944}
I20260709 13:03:24.548264  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:03:24.529819  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49807.1us
I20260709 13:03:24.476159  7833 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.483711  7865 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.562920  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.560714  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:24.533389  7898 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:24.533700  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 60170.5us
I20260709 13:03:24.487994  7687 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.525594  6836 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.486610  7719 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.501070  7690 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.503997  7777 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.547644  1559 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:24.549090  7823 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
W20260709 13:03:24.549665  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67911us
I20260709 13:03:24.577824  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:24.552505  7716 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.561528  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:24.538465  6983 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.522076  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.488858  7787 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:03:24.567315  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:24.596767  7716 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.484020  7098 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:03:24.528007  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:24.581293  7891 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:03:24.612444  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.612978  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.613699  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:24.614053  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2355) took 1803 ms. Trace:
I20260709 13:03:24.614405  1708 rpcz_store.cc:276] 0709 13:03:22.810207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.810476 (+   269us) service_pool.cc:224] Handling call
0709 13:03:22.838875 (+ 28399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.851956 (+1013081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.304259 (+452303us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b5400272020. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e7a02 00007f70ce9e3dd6 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:24.613810 (+309551us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54003e2454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:24.613989 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059459584}
I20260709 13:03:24.615134  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2349) took 1939 ms. Trace:
I20260709 13:03:24.615499  1706 rpcz_store.cc:276] 0709 13:03:22.675425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.679045 (+  3620us) service_pool.cc:224] Handling call
0709 13:03:22.679373 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.838140 (+1158767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.614925 (+776785us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540016ba54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:24.615081 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2402306944}
I20260709 13:03:24.616401  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1383) took 1076 ms. Trace:
I20260709 13:03:24.617189  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2359) took 1679 ms. Trace:
I20260709 13:03:24.507047  7649 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.559159  7847 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:24.562352  7848 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:24.648315  7847 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:24.649276  7847 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.649843  7801 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.650512  7847 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.650574  7316 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.652064  7871 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.653357  7871 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:24.654666  7879 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.385s	user 0.005s	sys 0.000s
I20260709 13:03:24.656141  7848 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:24.656352  7333 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.656693  7848 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.657037  7626 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.599882  6934 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:24.658098  6934 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:24.658485  6934 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:24.658792  7848 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.659070  7626 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.569864  7865 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.661702  7865 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.661949  6836 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.587857  7396 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.526423  7767 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:24.612618  7887 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:03:24.574587  7855 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:03:24.542078  7772 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.565287  7852 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.600912  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.603868  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.617548  1704 rpcz_store.cc:276] 0709 13:03:22.938109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.525418 (+587309us) service_pool.cc:224] Handling call
0709 13:03:23.700715 (+175297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.101174 (+400459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.101223 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.101291 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:24.101339 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.101437 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.617035 (+515598us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400042c50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.617136 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64227,"raft.queue_time_us":140122,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7449,"spinlock_wait_cycles":369930624}
I20260709 13:03:24.627475  1559 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:03:24.676891  1559 pending_rounds.cc:85] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 15
I20260709 13:03:24.677182  1559 pending_rounds.cc:107] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:03:24.677457  1559 raft_consensus.cc:2889] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.677223  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:24.627804  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2892) took 1790 ms (client timeout 1872 ms). Trace:
I20260709 13:03:24.628484  1702 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:24.629118  2295 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:24.533039  7071 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.630191  1634 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:24.630765  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.616750  2294 rpcz_store.cc:276] 0709 13:03:23.539818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.540609 (+   791us) service_pool.cc:224] Handling call
0709 13:03:23.583259 (+ 42650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.075094 (+491835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.616171 (+541077us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54002c8154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:24.616345 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976404352}
I20260709 13:03:24.634493  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.634243  1452 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 13, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 13 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.635942  1705 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:03:24.640152  7900 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:03:24.684536  2293 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:24.686888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.630127  7777 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:24.641224  7622 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.689098  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:24.591712  7578 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:24.642555  7860 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.577986  7814 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 13:03:24.643160  7873 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.548s	user 0.001s	sys 0.004s
I20260709 13:03:24.643445  7893 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.643776  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.667874  1626 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:24.591589  1632 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:24.697865  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.698717  1706 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.666697  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.699751  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:24.682634  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.682904  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:24.678689  1707 rpcz_store.cc:269] 0709 13:03:22.836989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.837182 (+   193us) service_pool.cc:224] Handling call
0709 13:03:24.613325 (+1776143us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.627699 (+ 14374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055845632}
I20260709 13:03:24.688220  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:24.667187  7802 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.000s
I20260709 13:03:24.665846  7855 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:24.694357  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:24.727355  2293 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:24.630903  7842 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:24.700232  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.704756  7071 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.704800  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2888) took 1768 ms. Trace:
I20260709 13:03:24.706960  7772 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.708012  7903 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:24.708673  7290 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.708952  7833 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:24.708599  1452 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 17, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:24.695953  7856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.004s	sys 0.001s
I20260709 13:03:24.710304  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.710765  7865 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:24.726716  7855 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.714066  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.714994  7801 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:24.715238  7853 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.000s
I20260709 13:03:24.717664  1708 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:24.718106  2297 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.746729  7578 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:24.686410  7852 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.746726  7903 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:24.757848  7856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.005s	sys 0.001s
I20260709 13:03:24.722540  7893 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:24.721999  7854 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.099s	user 0.013s	sys 0.000s
I20260709 13:03:24.723549  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.728489  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:24.725752  1598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.730s (SENT) [suppressed 18 similar messages]
I20260709 13:03:24.737648  2288 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:24.697191  7823 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:24.747758  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.700682  7366 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:24.738919  1631 rpcz_store.cc:276] 0709 13:03:22.936680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.937618 (+   938us) service_pool.cc:224] Handling call
0709 13:03:22.968569 (+ 30951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.004292 (+ 35723us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:23.020969 (+ 16677us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.505114 (+484145us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b1800082540. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:23.623556 (+118442us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.624108 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.624253 (+   145us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:23.624283 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.624357 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.732806 (+108449us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b1800082540. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:24.704586 (+971780us) raft_consensus.cc:1685] finished
0709 13:03:24.704609 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.704720 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":927,"prepare.queue_time_us":16560,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":57,"raft.queue_time_us":64149,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":27122,"spinlock_wait_cycles":24879232,"thread_start_us":87924,"threads_started":1,"wal-append.queue_time_us":632622}
W20260709 13:03:24.740960  7844 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.861s	user 0.010s	sys 0.000s
I20260709 13:03:24.726933  7860 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.712040  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.743395  7622 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.760988  7865 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.743651  7626 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.746408  7802 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:03:24.771265  1630 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:24.855669  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.857494  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.858512  1630 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:24.772828  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:24.859988  7369 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.860706  7369 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.861428  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.863016  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:24.773264  7772 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.864851  7772 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:24.859539  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:24.773645  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.867962  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.785531  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.788138  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:24.790113  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2364) took 1798 ms (client timeout 1730 ms). Trace:
I20260709 13:03:24.797678  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:24.870432  1703 rpcz_store.cc:269] 0709 13:03:22.991392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.119533 (+128141us) service_pool.cc:224] Handling call
0709 13:03:24.789955 (+1670422us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003c1754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.790050 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2715393280}
I20260709 13:03:24.794991  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.871428  1634 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:24.788823  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.798386  2296 raft_consensus.cc:2410] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:24.826875  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:24.770287  1706 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:24.874387  1554 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:24.831034  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:24.844944  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.831339  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.876209  1556 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:24.868877  1630 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:03:24.830591  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.830430  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2369) took 1109 ms. Trace:
I20260709 13:03:24.877686  1552 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:24.803736  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:24.809219  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.784976  2297 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:24.817358  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.821719  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.819612  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:24.880374  1629 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
I20260709 13:03:24.835645  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.788522  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.877557  1702 rpcz_store.cc:276] 0709 13:03:23.721489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.721655 (+   166us) service_pool.cc:224] Handling call
0709 13:03:24.628253 (+906598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.678993 (+ 50740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.830240 (+151247us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54001cbbd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:24.830375 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084236672}
I20260709 13:03:24.804157  1555 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:24.812459  2294 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:24.823658  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.825668  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:24.882925  1627 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:24.834893  1551 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
W20260709 13:03:24.844779  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.883505  1551 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 10
I20260709 13:03:24.884001  1551 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:03:24.884280  1551 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.881951  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.775681  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:24.893932  1708 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:24.842715  2293 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:24.891731  1553 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:24.902102  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.903740  1559 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:24.938184  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:24.939260  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:24.811688  7888 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.522s	user 0.004s	sys 0.000s
W20260709 13:03:24.824594  7872 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.553s	user 0.003s	sys 0.000s
I20260709 13:03:24.938246  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.917042  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:24.829324  7861 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 13:03:24.778002  7606 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.357s	user 0.012s	sys 0.001s
W20260709 13:03:24.820470  7906 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:03:24.905015  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:24.791070  7907 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:03:24.978492  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.788800  7903 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:24.985245  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:24.986111  1634 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:24.961505  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.808987  7823 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.962600  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:24.916355  7719 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:24.960223  1552 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:25.002430  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:24.957763  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:24.786618  7852 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.867559  7772 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:24.799695  7893 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.773546  7801 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.822214  7649 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.770581  7842 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:24.800735  7913 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:24.969220  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.001856  7823 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.962076  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.043282  1556 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:24.824548  7802 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:24.783360  7831 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.790827  6983 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:24.939541  2288 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:24.892416  7369 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:24.979421  1559 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:24.904141  7819 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.948168  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.804250  7858 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.956212  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:24.961505  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:24.795972  7690 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:25.060899  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:25.061244  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 335489us
I20260709 13:03:24.805302  7898 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:24.991219  1557 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:24.993479  1702 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:24.994608  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.004s	sys 0.017s
I20260709 13:03:25.092525  1559 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:25.092972  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.004s	sys 0.017s
W20260709 13:03:25.093327  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 652567us
I20260709 13:03:25.094013  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:24.955809  7854 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:24.850037  6934 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.095075  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:24.776218  7578 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.006062  1554 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:25.104777  1554 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:25.104923  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.007238  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:25.007913  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50207.2us
I20260709 13:03:25.105844  1559 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:24.968891  7637 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.026288  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:25.107689  1635 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:24.820873  7626 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.110179  7895 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.208s
I20260709 13:03:25.110347  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:25.111187  1703 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:24.777518  7855 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.112090  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:25.112282  7626 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.112746  1703 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:25.108134  7854 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:25.113884  7854 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.114176  7801 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.026546  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:25.060674  7690 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:25.116818  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:25.116607  7690 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.117805  7690 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:25.118033  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:25.118592  6934 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:25.119383  7649 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:25.119822  7893 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:25.120214  7649 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:25.120697  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.027166  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:25.110370  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:25.121944  1635 raft_consensus.cc:606] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:24.977856  7861 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.987190  7845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.015s	sys 0.000s
I20260709 13:03:25.028960  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.129190  1626 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:25.031380  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.035292  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.036419  1560 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 13.
I20260709 13:03:24.798543  7813 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:24.962837  7862 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 13:03:25.039572  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 288493us
I20260709 13:03:25.040028  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.178893  1556 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:25.045444  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.047101  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.047711  1558 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:24.774405  7853 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.818534  7844 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.806340  7797 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.029558  7772 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:25.132951  7845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.015s	sys 0.001s
I20260709 13:03:25.091358  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.062013  1627 raft_consensus.cc:2410] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:25.004330  1552 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:24.820639  7865 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:24.961572  7843 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.198559  7843 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.106566  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:25.177943  7654 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:03:25.199965  7865 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:25.200119  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 93630.4us
W20260709 13:03:25.202731  7914 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:03:25.202888  1545 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 with delete_type TABLET_DATA_TOMBSTONED (TS 444b494bb53842b59408a7230884398a not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:51896
I20260709 13:03:25.204104  1709 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
W20260709 13:03:25.204504  7915 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:03:25.204299  7722 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:25.204150  7719 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:25.205587  7719 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.205989  7719 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.206804  1634 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:03:25.108037  1553 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:25.166632  7777 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:25.208562  1632 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:03:25.210038  7797 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.094759  7819 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.214100  7578 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:24.818913  7860 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.057199  7802 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:25.216506  7802 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.217403  7860 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.217726  7802 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.217953  7842 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.122488  1635 raft_consensus.cc:614] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:25.126819  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100365us
I20260709 13:03:24.819921  7622 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.224217  7622 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:25.133113  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:25.152012  1703 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:25.224992  7853 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.225417  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.225488  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.226270  1706 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:25.226495  1703 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:25.227083  2292 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:25.227176  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.227360  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.128185  7895 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:03:25.180737  1554 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:25.179320  1629 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:25.230305  1626 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:03:24.776826  1452 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 14, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 14 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:25.028713  7897 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.237679  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.239034  1703 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:25.240727  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.120734  7649 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:25.239233  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55125.3us
I20260709 13:03:25.187448  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.253677  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:25.254371  1555 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:25.184562  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.133180  7861 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.133689  7626 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:25.178184  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.053126  7913 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.228005  1628 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:03:25.231482  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:25.236626  1553 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:25.238080  2291 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:03:25.277889  2291 pending_rounds.cc:85] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 13
I20260709 13:03:25.278234  2291 pending_rounds.cc:107] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:03:25.278525  2291 raft_consensus.cc:2889] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.248493  1704 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:25.256461  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:25.185190  7884 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.269932  1702 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:25.272840  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.273931  1556 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:25.187865  7858 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.126505  7637 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.291431  1629 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:03:25.311980  1629 pending_rounds.cc:85] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 5
I20260709 13:03:25.312392  1629 pending_rounds.cc:107] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:03:25.312695  1629 raft_consensus.cc:2889] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.315706  1554 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:03:25.321719  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48950.7us
I20260709 13:03:25.323069  1674 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
I20260709 13:03:25.323593  1674 leader_election.cc:403] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:03:25.320183  1705 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:25.269701  7626 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:25.284849  7910 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:03:25.359025  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.359712  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:25.360262  1702 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:25.270426  7772 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:25.327731  1523 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.872s (SENT)
I20260709 13:03:25.365538  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.368229  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45376.1us
I20260709 13:03:25.369131  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1023) took 1075 ms. Trace:
W20260709 13:03:25.372121  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 118494us
I20260709 13:03:25.373067  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:25.374569  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:25.376159  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.375399  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 119926us
I20260709 13:03:25.379057  1557 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:25.369644  1628 rpcz_store.cc:276] 0709 13:03:24.293217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.293324 (+   107us) service_pool.cc:224] Handling call
0709 13:03:24.293898 (+   574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.276358 (+982460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.368744 (+ 92386us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b5400280854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:25.369051 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147696384}
I20260709 13:03:25.193837  7813 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.387424  7578 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:25.392674  7649 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.393167  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.393996  1702 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:25.394330  7843 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.394853  7843 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.395085  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.387602  7845 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
W20260709 13:03:25.398074  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 80410.6us
I20260709 13:03:25.398588  1632 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:25.399119  1632 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:25.400559  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2374) took 1669 ms (client timeout 1620 ms). Trace:
I20260709 13:03:25.401546  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.402523  7767 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:25.402700  7917 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.367s	user 0.000s	sys 0.002s
I20260709 13:03:25.387494  7290 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:25.403492  7290 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.403873  7290 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:25.409380  7919 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:03:25.401157  1709 rpcz_store.cc:269] 0709 13:03:23.731232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.858056 (+126824us) service_pool.cc:224] Handling call
0709 13:03:25.400293 (+1542237us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540016bf54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:25.400461 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2881324928}
I20260709 13:03:25.424795  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.426296  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.426990  1709 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:25.428656  1452 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 15, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 15 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:25.430294  7578 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:25.438282  7855 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:25.259284  7892 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:25.358934  7369 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:25.365746  7626 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:25.258936  7897 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.396503  7845 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:25.450390  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.419420  1702 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:25.397156  7843 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.421098  1632 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:25.466531  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.469349  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.390232  7862 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:25.482002  1709 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:25.484266  1557 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:25.315568  7858 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.483834  7862 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:25.310806  7884 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.501866  1703 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:25.494684  7654 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:03:25.499353  7862 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:25.483386  7787 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 13:03:25.506788  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.508291  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.598387  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:25.602380  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.602864  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.514389  7819 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.517128  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.517346  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:25.515035  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.605041  1704 raft_consensus.cc:1813] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Handling vote request from an unknown peer 444b494bb53842b59408a7230884398a
I20260709 13:03:25.605247  1705 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:25.605540  1704 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 15.
I20260709 13:03:25.520973  7844 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.523132  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1386) took 1263 ms. Trace:
I20260709 13:03:25.526172  7833 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:25.526610  1702 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:25.531075  7832 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:25.533102  7861 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.465687  7847 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:03:25.534266  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:25.645653  2291 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.534909  7913 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.536152  7772 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.460122  7860 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:25.459311  7870 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
W20260709 13:03:25.537686  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.665973  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:25.538606  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.459903  7831 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:03:25.677258  1705 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:25.543782  7637 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.549620  7858 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.678203  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2913) took 1023 ms. Trace:
I20260709 13:03:25.554963  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.678727  1705 rpcz_store.cc:276] 0709 13:03:24.654800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.632030 (+977230us) service_pool.cc:224] Handling call
0709 13:03:25.676958 (+ 44928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.677879 (+   921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.678127 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.418967  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.628365  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111113us
I20260709 13:03:25.633563  1704 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:25.635623  1703 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:25.624889  2292 rpcz_store.cc:276] 0709 13:03:24.259801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.260011 (+   210us) service_pool.cc:224] Handling call
0709 13:03:24.260279 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.227590 (+967311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.522834 (+295244us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:25.523057 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223008128}
I20260709 13:03:25.626653  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:25.637763  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.639801  1707 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.683943  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2912) took 1029 ms. Trace:
I20260709 13:03:25.700479  1703 rpcz_store.cc:276] 0709 13:03:24.654549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.635024 (+980475us) service_pool.cc:224] Handling call
0709 13:03:25.635398 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.683624 (+ 48226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.683873 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:25.705042  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 286429us
I20260709 13:03:25.706394  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.686555  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.687290  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.709373  2292 raft_consensus.cc:2410] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:03:25.693485  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55845.4us
I20260709 13:03:25.714183  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.642338  1627 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:03:25.642851  2289 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:25.487072  7845 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:25.644824  1675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.006s	sys 0.040s
W20260709 13:03:25.753082  1675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.006s	sys 0.040s
W20260709 13:03:25.753427  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 661888us
W20260709 13:03:25.644731  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44658.1us
I20260709 13:03:25.431206  1452 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:03:25.654170  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.658880  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.659372  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:25.766252  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.658254  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:25.766897  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.767032  1634 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:25.633014  7578 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.768155  7578 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.770362  7578 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.666597  1628 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.666730  1702 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: [13.13-15.14]
I20260709 13:03:25.770920  1452 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:25.772256  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1060) took 1033 ms. Trace:
I20260709 13:03:25.772415  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.772632  1702 rpcz_store.cc:276] 0709 13:03:24.739233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.634901 (+895668us) service_pool.cc:224] Handling call
0709 13:03:25.666372 (+ 31471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:25.771991 (+105619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.772202 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.773710  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:25.774268  1702 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.773098  1452 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 13, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 13 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:25.775337  1558 raft_consensus.cc:2410] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:25.490964  7889 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.155s
I20260709 13:03:25.640664  7654 consensus_queue.cc:1243] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:25.644543  7819 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:25.784085  7819 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:25.784379  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.784605  7819 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:25.784629  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1154) took 2118 ms. Trace:
I20260709 13:03:25.785154  2289 rpcz_store.cc:276] 0709 13:03:23.666187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.666299 (+   112us) service_pool.cc:224] Handling call
0709 13:03:23.687387 (+ 21088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.716301 (+2028914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.784300 (+ 67999us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b5400329e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:25.784532 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4418293120}
I20260709 13:03:25.785871  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.786342  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.787117  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.676517  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.686445  7860 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:25.719350  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:25.626178  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.790280  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164170us
I20260709 13:03:25.789728  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:25.791034  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.792310  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.717695  7861 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:25.794421  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.729274  7871 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:25.729849  7637 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:25.688138  7884 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:25.795348  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76104.2us
I20260709 13:03:25.716497  7814 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.660145  7832 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.736449  1707 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:25.737577  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.738945  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.679345  7813 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:25.740391  7903 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.732s	user 0.001s	sys 0.009s
I20260709 13:03:25.741600  7290 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:25.744112  7865 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.003s
I20260709 13:03:25.634191  7833 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:25.745195  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.749296  7897 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.760040  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.820422  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 60448us
I20260709 13:03:25.821313  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:25.773710  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.822139  2292 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:25.823383  7626 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.720911  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.791149  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.699319  7843 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:25.796747  1632 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:25.824738  1706 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:25.824959  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 104162us
I20260709 13:03:25.825897  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.797473  2291 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:03:25.828599  1551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.011s	sys 0.107s
W20260709 13:03:25.831985  1551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.011s	sys 0.107s
W20260709 13:03:25.799887  1545 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.025s	sys 0.027s
I20260709 13:03:25.787424  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:25.804050  1628 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:25.641901  7622 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:03:25.633886  7719 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:25.809262  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:25.754540  7845 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:25.809435  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.815907  1559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.008s	sys 0.051s
I20260709 13:03:25.816468  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:25.827292  2295 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:25.804834  7637 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.800040  7860 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:25.872061  1629 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:03:25.873003  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 63660.2us
W20260709 13:03:25.811717  7842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.030s	sys 0.039s
W20260709 13:03:25.835282  1545 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.026s	sys 0.027s
W20260709 13:03:25.844247  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.851075  1632 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:03:25.810660  7578 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:25.755116  7913 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:25.805085  7722 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:25.900795  7860 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:25.868975  1559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.008s	sys 0.051s
I20260709 13:03:25.902592  7860 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:25.805243  7871 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:25.904230  7871 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:25.811141  7802 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:25.907691  7802 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.908217  7802 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:25.872320  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 51089.6us
I20260709 13:03:25.810484  7773 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:25.875334  2291 pending_rounds.cc:85] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 10
I20260709 13:03:25.827098  7813 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:25.889207  7842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.031s	sys 0.040s
W20260709 13:03:25.870329  7854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.014s	sys 0.048s
I20260709 13:03:25.870874  7637 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:25.819154  7861 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.141s
I20260709 13:03:25.888897  7897 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:25.872269  7777 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.817663  7903 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.811975  7814 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:25.911734  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:25.988102  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76497.9us
I20260709 13:03:25.989812  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:25.912662  2291 pending_rounds.cc:107] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:25.991349  2291 raft_consensus.cc:2889] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.894006  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1065) took 1147 ms. Trace:
I20260709 13:03:25.897431  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.901614  7722 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:26.000290  1704 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:03:26.002624  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 105287us
W20260709 13:03:25.999449  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2900) took 1413 ms (client timeout 1843 ms). Trace:
W20260709 13:03:26.004383  1707 rpcz_store.cc:269] 0709 13:03:24.585534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.728227 (+142693us) service_pool.cc:224] Handling call
0709 13:03:25.999165 (+1270938us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54001333d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:25.999355 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920772992}
I20260709 13:03:25.967334  1633 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21.
I20260709 13:03:25.891996  1632 pending_rounds.cc:85] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 9
I20260709 13:03:26.003350  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:26.015961  1632 pending_rounds.cc:107] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:26.016386  1632 raft_consensus.cc:2889] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:26.016763  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.974515  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1166) took 1255 ms. Trace:
I20260709 13:03:26.018147  2295 rpcz_store.cc:276] 0709 13:03:24.718972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.785186 (+ 66214us) service_pool.cc:224] Handling call
0709 13:03:25.974224 (+1189038us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54005e06d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:25.974386 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201856896}
W20260709 13:03:26.017302  2288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.032s	sys 0.040s
W20260709 13:03:26.019527  2288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.032s	sys 0.040s
I20260709 13:03:26.019834  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1061) took 1280 ms. Trace:
I20260709 13:03:26.020493  1707 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:26.020344  1704 rpcz_store.cc:276] 0709 13:03:24.739477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.682785 (+943308us) service_pool.cc:224] Handling call
0709 13:03:25.999938 (+317153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.000907 (+   969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.019511 (+ 18604us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54001333d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:26.019705 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728379648}
I20260709 13:03:25.994455  1702 rpcz_store.cc:276] 0709 13:03:24.746392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.888147 (+1141755us) service_pool.cc:224] Handling call
0709 13:03:25.893266 (+  5119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.893552 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.893591 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.893650 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:25.893706 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.893777 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.893928 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.022626  1704 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:03:26.013000  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:26.023428  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2376) took 1225 ms. Trace:
I20260709 13:03:26.023676  1630 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.024032  1704 rpcz_store.cc:276] 0709 13:03:24.797635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.021815 (+1224180us) service_pool.cc:224] Handling call
0709 13:03:26.022201 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.023108 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.023357 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.014947  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:25.972699  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2388) took 1155 ms. Trace:
I20260709 13:03:26.025321  1706 rpcz_store.cc:276] 0709 13:03:24.816793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.226939 (+410146us) service_pool.cc:224] Handling call
0709 13:03:25.972412 (+745473us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b5400096d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:25.972613 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372083072}
I20260709 13:03:26.027612  1704 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:26.034897  2263 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.014s	sys 0.056s
W20260709 13:03:26.035286  2263 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.014s	sys 0.056s
W20260709 13:03:26.035531  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 852742us
I20260709 13:03:26.035617  1702 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:26.040201  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2984) took 1306 ms. Trace:
W20260709 13:03:25.948232  7854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.014s	sys 0.048s
I20260709 13:03:26.050935  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2379) took 1238 ms. Trace:
I20260709 13:03:26.042970  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2375) took 1224 ms. Trace:
W20260709 13:03:26.043259  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1028) took 1441 ms (client timeout 1551 ms). Trace:
W20260709 13:03:25.965989  7855 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.003s
I20260709 13:03:25.932155  6983 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:26.047166  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.021s	sys 0.049s
W20260709 13:03:26.060058  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.021s	sys 0.049s
I20260709 13:03:26.047588  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2380) took 1229 ms. Trace:
I20260709 13:03:25.912438  7773 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:26.060462  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 650950us
I20260709 13:03:26.051450  1702 rpcz_store.cc:276] 0709 13:03:24.798349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.021992 (+1223643us) service_pool.cc:224] Handling call
0709 13:03:26.035433 (+ 13441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.036622 (+  1189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.036815 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:26.054117  1709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.032s
W20260709 13:03:26.053018  1633 rpcz_store.cc:269] 0709 13:03:24.602277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.614508 (+ 12231us) service_pool.cc:224] Handling call
0709 13:03:26.043066 (+1428558us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540024fc54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:26.043200 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2920032000}
I20260709 13:03:26.054872  1634 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:25.931635  7843 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:26.040879  1632 rpcz_store.cc:276] 0709 13:03:24.733678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.209335 (+475657us) service_pool.cc:224] Handling call
0709 13:03:25.209620 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.398456 (+188836us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54002cd920. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:03:25.796308 (+397852us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54002cd920. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce959a1d 00007f70ce9401a4 00007f70ce9c2447 00007f70ce9dafe7 00007f70ce9dc7de 00007f70ce9e6b49 00007f70ce9e7416 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419
0709 13:03:26.039860 (+243552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.040120 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890914304}
W20260709 13:03:26.057508  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.020s	sys 0.037s
I20260709 13:03:25.997819  7722 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.059114  2292 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:03:26.074141  1709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.012s	sys 0.032s
I20260709 13:03:25.950354  7832 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:25.962172  7290 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:26.060998  1704 rpcz_store.cc:276] 0709 13:03:24.798636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.027097 (+1228461us) service_pool.cc:224] Handling call
0709 13:03:26.027395 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.028129 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.028340 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.034632  7913 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:25.938176  7719 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:26.073741  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.020s	sys 0.037s
I20260709 13:03:26.017622  7813 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:26.017263  7845 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:26.086849  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.931972  7622 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.128s
I20260709 13:03:25.963430  7777 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:26.082247  7823 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.015s	user 0.011s	sys 0.001s
I20260709 13:03:26.080112  1702 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:26.109489  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2386) took 1293 ms. Trace:
I20260709 13:03:26.110029  1702 rpcz_store.cc:276] 0709 13:03:24.816298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.075461 (+1259163us) service_pool.cc:224] Handling call
0709 13:03:26.079675 (+  4214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.109050 (+ 29375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.109393 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.086445  7813 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.053159  1707 rpcz_store.cc:276] 0709 13:03:24.797350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.019664 (+1222314us) service_pool.cc:224] Handling call
0709 13:03:26.020074 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.021190 (+  1116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.021450 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:26.056082  7893 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:03:26.044968  7369 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:26.085281  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 703668us
W20260709 13:03:26.070470  7316 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.017s	sys 0.038s
I20260709 13:03:26.083099  7855 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:26.105875  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.059914  6983 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:26.076053  7858 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.077147  7722 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:26.112944  1707 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:25.968403  7833 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.117343  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:26.117833  2295 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:03:26.079919  1552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.040s
W20260709 13:03:26.129081  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.007s	sys 0.015s
I20260709 13:03:26.142098  1707 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:26.105806  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.115839  1702 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:26.227284  1557 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:26.226982  6983 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:26.228806  6983 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:26.229224  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.230062  1557 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:26.231678  1555 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:26.215234  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:26.235561  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.011s	sys 0.017s
W20260709 13:03:26.129307  1560 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.015s	sys 0.015s
W20260709 13:03:26.237592  1560 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.019s	sys 0.018s
W20260709 13:03:26.235262  1552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.010s	sys 0.040s
I20260709 13:03:26.116604  7813 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:26.095177  7832 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.237069  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:26.107048  7290 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.095649  7913 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:26.123093  7316 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.018s	sys 0.038s
I20260709 13:03:26.260424  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:26.259654  2293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.009s	sys 0.028s
W20260709 13:03:26.280548  2293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.010s	sys 0.028s
W20260709 13:03:26.268541  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.091089  7654 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
W20260709 13:03:26.143095  7801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.026s	user 0.016s	sys 0.028s
W20260709 13:03:26.145153  7649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.004s	sys 0.019s
I20260709 13:03:26.094763  7843 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.179s
W20260709 13:03:26.271314  1631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.017s	sys 0.012s
I20260709 13:03:26.275022  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:26.307327  1554 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:26.308563  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.309396  1554 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:26.311321  1554 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:26.091657  7862 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.092450  7819 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.282197  1558 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:26.282923  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.154533  7814 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:26.275490  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:26.288653  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.234117  7895 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.236634  7845 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:26.058491  1630 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:26.279220  7833 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:26.300244  7929 tablet_replica.cc:333] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:03:26.121508  7823 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.205071  7903 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:26.208998  7855 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.205994  7870 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.155s
W20260709 13:03:26.293349  7801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.017s	sys 0.030s
I20260709 13:03:26.287389  7290 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.301761  7626 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:26.287675  7832 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:26.296439  7649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.005s	sys 0.020s
W20260709 13:03:26.315594  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.017s	sys 0.012s
I20260709 13:03:26.314810  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:03:26.324412  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49015.5us
I20260709 13:03:26.350449  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:26.325289  2290 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:26.351800  1701 raft_consensus.cc:2368] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:26.352010  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.144459  7831 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
W20260709 13:03:26.329406  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.026s	sys 0.021s
I20260709 13:03:26.289800  7865 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:25.947880  1452 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 14, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.235351  7858 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.342759  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:26.367681  1554 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:26.346822  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.017s	sys 0.025s
W20260709 13:03:26.371863  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.018s	sys 0.025s
W20260709 13:03:26.372289  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 964949us
I20260709 13:03:26.332652  7814 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:26.357645  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.361852  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:26.363139  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.026s	sys 0.022s
I20260709 13:03:26.338845  7690 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:26.360041  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:26.336429  7845 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:26.371201  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:26.389381  1704 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:26.390201  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1400) took 1268 ms. Trace:
I20260709 13:03:26.392783  7290 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:26.395686  7862 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:26.415426  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:26.328871  7895 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:26.461632  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46320.5us
W20260709 13:03:26.417114  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 44046.3us
I20260709 13:03:26.464184  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:26.423372  1556 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:26.424007  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1070) took 1589 ms. Trace:
I20260709 13:03:26.465713  1704 rpcz_store.cc:276] 0709 13:03:24.834929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.080967 (+1246038us) service_pool.cc:224] Handling call
0709 13:03:26.081376 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.389105 (+307729us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b5400272f20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:03:26.423594 (+ 34489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.423918 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60419840}
I20260709 13:03:26.423602  2288 rpcz_store.cc:276] 0709 13:03:25.121754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.122155 (+   401us) service_pool.cc:224] Handling call
0709 13:03:25.122514 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.122864 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.122892 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.123307 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.028079 (+904772us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:26.028140 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.028234 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.389939 (+361705us) raft_consensus.cc:1685] finished
0709 13:03:26.389963 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.390133 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1651,"raft.run_cpu_time_us":4041,"raft.run_wall_time_us":54046,"thread_start_us":896341,"threads_started":1,"wal-append.queue_time_us":1264332}
I20260709 13:03:26.368976  7801 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:26.366536  7865 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:26.482184  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:26.491936  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 131978us
I20260709 13:03:26.416544  7690 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:26.511546  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.512243  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.467828  7854 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:26.518227  1703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.013s	sys 0.023s
I20260709 13:03:26.520969  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:26.538861  1703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.013s	sys 0.025s
I20260709 13:03:26.482372  7865 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:26.528378  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.550665  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:26.530299  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2901) took 1944 ms (client timeout 1606 ms). Trace:
I20260709 13:03:26.481292  7290 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.563592  7819 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:26.563323  7290 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.563902  7654 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:26.566126  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 58611.4us
I20260709 13:03:26.566396  7290 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:26.567293  7290 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:26.545238  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63188.8us
I20260709 13:03:26.448925  7903 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:03:26.517880  7862 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:26.455984  7889 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:03:26.611268  7654 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:26.611889  7654 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:26.366443  7913 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.500687  7893 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:26.591826  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:26.591127  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.551621  1701 rpcz_store.cc:269] 0709 13:03:24.585759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.666746 (+ 80987us) service_pool.cc:224] Handling call
0709 13:03:26.530062 (+1863316us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:26.530220 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3794407168}
I20260709 13:03:26.469107  7942 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 13:03:26.611359  7819 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.548180  7871 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.006s
I20260709 13:03:26.526824  7801 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:26.618342  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.010s	sys 0.035s
I20260709 13:03:26.621337  1554 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 10.
W20260709 13:03:26.642884  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.010s	sys 0.036s
I20260709 13:03:26.644570  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:26.568841  7649 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:26.646489  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.018s	sys 0.027s
W20260709 13:03:26.647069  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.019s	sys 0.028s
I20260709 13:03:26.612195  7889 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:26.643735  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:26.569319  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:26.656994  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 831089us
I20260709 13:03:26.711246  1523 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 13. Current term is 15.
W20260709 13:03:26.631524  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.666684  1629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.014s	sys 0.024s
W20260709 13:03:26.713848  1629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.017s	sys 0.025s
W20260709 13:03:26.687642  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.019s	sys 0.030s
I20260709 13:03:26.607494  7854 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:26.715636  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.019s	sys 0.031s
W20260709 13:03:26.716197  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 922095us
I20260709 13:03:26.717792  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:26.699815  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:26.569893  7777 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:26.618881  7772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.011s	sys 0.032s
I20260709 13:03:26.525718  7922 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:03:26.692452  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:26.640743  7853 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.007s	sys 0.040s
I20260709 13:03:26.608497  7862 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:26.698540  7928 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.415s	user 0.000s	sys 0.001s
W20260709 13:03:26.736315  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 167073us
I20260709 13:03:26.763906  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:26.747243  7290 raft_consensus.cc:2764] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:03:26.741555  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 97876.9us
I20260709 13:03:26.743343  7654 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:26.712744  7578 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.000s
I20260709 13:03:26.666950  7889 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:26.699158  7893 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:26.741747  7772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.015s	sys 0.032s
W20260709 13:03:26.778568  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.021s	sys 0.021s
W20260709 13:03:26.779034  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.021s	sys 0.021s
W20260709 13:03:26.779450  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 732546us
I20260709 13:03:26.780702  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:26.743948  7929 raft_consensus.cc:2243] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:26.752384  7903 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:26.756807  2291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.012s	sys 0.028s
I20260709 13:03:26.753413  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:26.752128  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.037s	sys 0.044s
I20260709 13:03:26.766656  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:26.772375  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:26.747400  7823 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:26.752107  7853 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.007s	sys 0.040s
I20260709 13:03:26.701622  7819 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.642607  7845 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.765174  7832 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:26.797538  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.037s	sys 0.044s
I20260709 13:03:26.760581  7862 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:26.797930  2291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.013s	sys 0.028s
I20260709 13:03:26.672123  7871 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.740801  7719 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:26.802932  1598 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
I20260709 13:03:26.822135  1598 leader_election.cc:403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:26.801551  2294 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:03:26.753342  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.012s	sys 0.044s
W20260709 13:03:26.829114  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.012s	sys 0.044s
W20260709 13:03:26.829444  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 810629us
I20260709 13:03:26.829668  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:26.832324  1706 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.024s	sys 0.042s
I20260709 13:03:26.776994  7870 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:26.646739  1452 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:26.820989  7802 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.004s
I20260709 13:03:26.689445  7690 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:26.807415  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:26.887542  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:26.776608  7578 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:26.830933  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:26.836020  1706 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.025s	sys 0.043s
I20260709 13:03:26.839638  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:26.840662  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:26.841022  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:26.893853  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:26.799331  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:26.895722  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 96466.2us
I20260709 13:03:26.848416  7772 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:26.783149  7930 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.436s	user 0.000s	sys 0.001s
I20260709 13:03:26.806905  7847 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:03:26.813119  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.816594  7654 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.903023  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2384) took 2103 ms. Trace:
I20260709 13:03:26.903569  1706 rpcz_store.cc:276] 0709 13:03:24.799589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.026403 (+1226814us) service_pool.cc:224] Handling call
0709 13:03:26.026782 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.902519 (+875737us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:26.902568 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.902632 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:26.902689 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.902784 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.902951 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7778,"raft.queue_time_us":80254,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11812,"thread_start_us":857553,"threads_started":1}
W20260709 13:03:26.905826  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.905731  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:26.906651  1706 raft_consensus.cc:2376] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 17. Re-sending same reply.
I20260709 13:03:26.810422  7944 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:26.874274  2289 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:26.814884  7903 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:26.878428  7819 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:26.879441  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1064) took 2139 ms (client timeout 1941 ms). Trace:
I20260709 13:03:26.848981  7622 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:03:26.885842  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1396) took 2111 ms. Trace:
W20260709 13:03:26.889752  6983 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.002s
W20260709 13:03:26.897512  1627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.007s	sys 0.016s
W20260709 13:03:27.154433  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.008s	sys 0.018s
W20260709 13:03:27.208559  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.008s	sys 0.018s
W20260709 13:03:27.205684  1627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.008s	sys 0.016s
W20260709 13:03:27.206823  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.007s	sys 0.018s
W20260709 13:03:27.210793  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.154529  1551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.287s	user 0.005s	sys 0.026s
W20260709 13:03:27.213150  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 495444us
I20260709 13:03:27.214000  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:27.215463  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.002s	sys 0.025s
W20260709 13:03:27.213382  1551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.380s	user 0.005s	sys 0.026s
I20260709 13:03:27.217161  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2836) took 2183 ms. Trace:
I20260709 13:03:26.913532  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:27.220594  7578 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:27.227218  2290 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:03:26.880313  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:26.915187  1708 rpcz_store.cc:269] 0709 13:03:24.740134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.961258 (+221124us) service_pool.cc:224] Handling call
0709 13:03:26.879220 (+1917962us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54001351d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:26.879367 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4007111552}
I20260709 13:03:26.852409  7895 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:26.876423  7870 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:26.915720  1558 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:03:27.123920  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.020s	sys 0.022s
W20260709 13:03:27.013722  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.012s	sys 0.020s
W20260709 13:03:27.230436  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.023s	sys 0.022s
W20260709 13:03:27.230515  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.020s	sys 0.030s
W20260709 13:03:27.230810  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.231181  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.21624e+06us
W20260709 13:03:27.014319  1709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.011s	sys 0.039s
W20260709 13:03:27.232769  1709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.011s	sys 0.057s
W20260709 13:03:27.072479  2296 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.001s	sys 0.039s
W20260709 13:03:26.999645  1626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.016s	sys 0.024s
I20260709 13:03:26.986919  7858 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:27.233162  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:27.233212  2296 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.002s	sys 0.048s
I20260709 13:03:27.233965  7870 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:26.933100  7945 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
W20260709 13:03:27.105569  7935 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:03:26.958824  6983 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.244711  6983 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.798693  7854 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.956667  7929 raft_consensus.cc:2272] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:26.858665  7290 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15.
I20260709 13:03:26.902558  7316 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:26.986225  7938 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:03:26.969252  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:26.926471  7944 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:27.060585  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.944051  7903 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:26.985880  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:27.322012  1634 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:26.933683  2297 rpcz_store.cc:276] 0709 13:03:24.774750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.960403 (+185653us) service_pool.cc:224] Handling call
0709 13:03:24.960747 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.970054 (+  9307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.970088 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.098029 (+127941us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b18002c5de0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a
0709 13:03:25.125326 (+ 27297us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b2c00047568. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:25.181384 (+ 56058us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.641213 (+459829us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:25.641255 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.641430 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.885538 (+1244108us) raft_consensus.cc:1685] finished
0709 13:03:26.885571 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.885753 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4084,"raft.queue_time_us":8916,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":15051,"spinlock_wait_cycles":18925568,"thread_start_us":446273,"threads_started":1,"wal-append.queue_time_us":924631}
I20260709 13:03:26.911151  7889 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.053646  7777 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.313052  7772 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:03:27.329754  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.014s
I20260709 13:03:27.145396  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.146461  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:27.332072  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.005s	sys 0.015s
W20260709 13:03:27.299296  7970 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:27.161401  7952 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.038956  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:27.066319  7637 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.110s	user 0.013s	sys 0.001s
I20260709 13:03:26.999168  7823 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.336126  1556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.008s	sys 0.008s
I20260709 13:03:27.337646  7823 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:27.338356  1556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.008s	sys 0.009s
I20260709 13:03:26.895117  7802 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.341687  7802 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:27.097504  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.342216  1635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.011s	sys 0.021s
I20260709 13:03:26.910722  7893 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.052402  7719 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:27.343477  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.011s	sys 0.021s
W20260709 13:03:27.344903  7865 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
I20260709 13:03:27.337257  7637 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.174467  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1063) took 2434 ms. Trace:
I20260709 13:03:27.177304  7690 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.345988  1705 rpcz_store.cc:276] 0709 13:03:24.739977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.702556 (+962579us) service_pool.cc:224] Handling call
0709 13:03:25.766360 (+ 63804us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:25.766747 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.766776 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:25.792469 (+ 25693us) log.cc:844] Serialized 52 byte log entry
0709 13:03:26.647624 (+855155us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:26.647689 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.647826 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.174171 (+526345us) raft_consensus.cc:1685] finished
0709 13:03:27.174199 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.174388 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":810247,"threads_started":1,"wal-append.queue_time_us":1231424}
I20260709 13:03:27.347349  7637 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.348026  1555 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:03:27.342958  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.350100  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2840) took 2165 ms. Trace:
I20260709 13:03:27.350873  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2338) took 1025 ms. Trace:
I20260709 13:03:27.246850  7369 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
W20260709 13:03:27.183219  7722 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.922s	user 0.012s	sys 0.000s
I20260709 13:03:27.351245  1555 rpcz_store.cc:276] 0709 13:03:26.325094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.326247 (+  1153us) service_pool.cc:224] Handling call
0709 13:03:26.345395 (+ 19148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.348454 (+1003059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.350628 (+  2174us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540036c9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:27.350815 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305189120}
I20260709 13:03:27.352146  7722 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.289492  7951 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:27.354359  7722 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.344120  7719 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.184553  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1179) took 1377 ms. Trace:
W20260709 13:03:27.355020  7801 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.000s
I20260709 13:03:27.202157  7862 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.355116  2291 rpcz_store.cc:276] 0709 13:03:25.806982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.868917 (+ 61935us) service_pool.cc:224] Handling call
0709 13:03:25.875086 (+  6169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.992083 (+116997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.992131 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.992710 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.814872 (+822162us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:26.814924 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.815007 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.184271 (+369264us) raft_consensus.cc:1685] finished
0709 13:03:27.184295 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.184486 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":821776,"threads_started":1,"wal-append.queue_time_us":1080033}
I20260709 13:03:27.357568  7862 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.358017  1557 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:03:27.184303  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.188549  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1408) took 1096 ms. Trace:
I20260709 13:03:27.358978  1706 raft_consensus.cc:2368] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
W20260709 13:03:27.192250  7937 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:03:27.193513  7855 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:27.138068  7940 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
W20260709 13:03:27.196439  7833 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.864s	user 0.012s	sys 0.000s
W20260709 13:03:27.202148  7813 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.907s	user 0.006s	sys 0.005s
W20260709 13:03:27.204165  7773 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.124s	user 0.008s	sys 0.005s
I20260709 13:03:27.080444  1452 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 16, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:27.204756  7860 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.302s	user 0.006s	sys 0.006s
W20260709 13:03:27.154022  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.022s	sys 0.001s
W20260709 13:03:26.901729  1700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.157s	user 0.010s	sys 0.026s
W20260709 13:03:27.395090  1700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.650s	user 0.011s	sys 0.026s
W20260709 13:03:27.395090  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.022s	sys 0.002s
I20260709 13:03:27.396739  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2829) took 2459 ms. Trace:
I20260709 13:03:27.397354  1560 rpcz_store.cc:276] 0709 13:03:24.937346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.176751 (+239405us) service_pool.cc:224] Handling call
0709 13:03:25.195014 (+ 18263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.195219 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:26.237975 (+1042756us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.396298 (+1158323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.396372 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:27.396446 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.396536 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.396673 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42606,"prepare.queue_time_us":1095483,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"thread_start_us":2106449,"threads_started":2}
W20260709 13:03:27.218088  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.002s	sys 0.025s
I20260709 13:03:27.174551  7884 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.139344  7939 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:27.153926  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.220856  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:27.405668  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.210798  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.018s
W20260709 13:03:27.406188  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 185426us
W20260709 13:03:27.406158  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.233342  1626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.025s	sys 0.033s
W20260709 13:03:27.233387  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.106549  7936 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:03:27.239008  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2909) took 2585 ms. Trace:
I20260709 13:03:27.246945  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.250048  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.415138  1701 raft_consensus.cc:2376] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 17. Re-sending same reply.
I20260709 13:03:27.256466  7942 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.257443  7654 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.250061  7858 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.258457  1551 rpcz_store.cc:276] 0709 13:03:25.034265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.185771 (+151506us) service_pool.cc:224] Handling call
0709 13:03:25.215107 (+ 29336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.215327 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:25.832668 (+617341us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.216715 (+1384047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.216788 (+    73us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:27.216854 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.216941 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.217090 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74088,"prepare.queue_time_us":1096716,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":2108,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":8901,"thread_start_us":1896532,"threads_started":2,"wal-append.queue_time_us":1413890}
I20260709 13:03:27.304363  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.307498  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:27.320358  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1403) took 2064 ms. Trace:
W20260709 13:03:27.314527  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.300654  7814 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.881s	user 0.013s	sys 0.000s
I20260709 13:03:27.333515  1707 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:27.373045  7831 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:27.438000  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 130607us
I20260709 13:03:27.163221  7871 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:27.172192  7941 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.505s	user 0.006s	sys 0.000s
W20260709 13:03:27.312433  7965 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:27.310433  7957 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:27.333101  1702 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:27.465039  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.358862  2289 rpcz_store.cc:276] 0709 13:03:26.092172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.237158 (+144986us) service_pool.cc:224] Handling call
0709 13:03:26.285153 (+ 47995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.917140 (+631987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.188270 (+271130us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:27.188475 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354120064}
W20260709 13:03:27.362648  1631 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.016s	sys 0.015s
W20260709 13:03:27.465972  1631 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.016s	sys 0.016s
W20260709 13:03:27.363193  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 524223us
I20260709 13:03:27.374622  2297 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:27.379526  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2333) took 1165 ms. Trace:
I20260709 13:03:27.381807  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2907) took 1779 ms. Trace:
W20260709 13:03:27.401757  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.03988e+06us
I20260709 13:03:27.350505  1552 rpcz_store.cc:276] 0709 13:03:25.184978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.193128 (+  8150us) service_pool.cc:224] Handling call
0709 13:03:25.231220 (+ 38092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.231560 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.231586 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.247223 (+ 15637us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.540041 (+292818us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b180047e300. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:26.240998 (+700957us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:26.241065 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.241168 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.347825 (+1106657us) raft_consensus.cc:1685] finished
0709 13:03:27.347844 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.349894 (+  2050us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54000e9f50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:27.350007 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":21549,"spinlock_wait_cycles":96412800,"thread_start_us":700707,"threads_started":1,"wal-append.queue_time_us":1271762}
I20260709 13:03:27.356994  7719 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:27.403888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.345352  7865 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.429597  7649 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.230242  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.409096  6983 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:27.409848  1559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.007s	sys 0.025s
W20260709 13:03:27.480803  1559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.007s	sys 0.026s
I20260709 13:03:27.435675  7722 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:27.418967  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 172095us
I20260709 13:03:27.355608  7801 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:27.423069  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.487632  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.487828  1704 raft_consensus.cc:2376] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 18. Re-sending same reply.
I20260709 13:03:27.424408  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.414295  1709 rpcz_store.cc:276] 0709 13:03:24.653751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.543465 (+889714us) service_pool.cc:224] Handling call
0709 13:03:25.543856 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.544131 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:26.074657 (+530526us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.238379 (+1163722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.238450 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:27.238503 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.238587 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.238909 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5154,"prepare.queue_time_us":1371741,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":66847,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":120,"spinlock_wait_cycles":18242816,"thread_start_us":1647317,"threads_started":2,"wal-append.queue_time_us":1142159}
W20260709 13:03:27.481771  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74056.1us
I20260709 13:03:27.359917  7316 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:27.490330  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.491164  1709 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.492352  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:27.436189  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:27.452536  7814 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.446573  7622 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:27.403213  7813 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.465407  7897 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.472043  1702 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:27.288866  7959 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.200s
I20260709 13:03:27.416961  7773 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:27.415459  7939 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:27.330479  7777 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.472594  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.461187  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2910) took 2807 ms. Trace:
I20260709 13:03:27.474422  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:27.414578  7922 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:27.422093  7951 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.469635  1557 rpcz_store.cc:276] 0709 13:03:26.214435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.230744 (+ 16309us) service_pool.cc:224] Handling call
0709 13:03:26.231095 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.358447 (+1127352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.379241 (+ 20794us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540030a2d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:27.379468 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2591108480}
I20260709 13:03:27.477553  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.469789  1629 rpcz_store.cc:276] 0709 13:03:25.602770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.873008 (+270238us) service_pool.cc:224] Handling call
0709 13:03:25.873393 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.873646 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:26.714610 (+840964us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.381406 (+666796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.381495 (+    89us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:27.381558 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.381625 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.381750 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45677,"prepare.queue_time_us":678810,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":5063,"raft.queue_time_us":46512,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10052,"spinlock_wait_cycles":152815104,"thread_start_us":1459889,"threads_started":2,"wal-append.queue_time_us":614442}
I20260709 13:03:27.321442  7944 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:27.528053  2294 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.524801  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:27.529102  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.483492  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.531850  2297 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:27.429481  7290 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:27.484884  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:27.532706  1700 rpcz_store.cc:276] 0709 13:03:24.654020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.632176 (+978156us) service_pool.cc:224] Handling call
0709 13:03:25.632625 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.663557 (+ 30932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.663608 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.701952 (+ 38344us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.395582 (+1693630us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:27.430743 (+ 35161us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.460753 (+ 30010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.460880 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.460939 (+    59us) raft_consensus.cc:1685] finished
0709 13:03:27.460958 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.461115 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37853,"prepare.queue_time_us":35246,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":40,"raft.queue_time_us":42133,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":37735,"spinlock_wait_cycles":23890816,"thread_start_us":1650585,"threads_started":1,"wal-append.queue_time_us":1395262}
I20260709 13:03:27.534832  1552 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:27.486112  1708 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:27.485441  1706 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:27.532397  1558 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:27.490685  6983 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.536037  6983 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.494725  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.536379  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:27.496130  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71862.8us
I20260709 13:03:27.494799  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:27.531843  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:27.532271  1702 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:27.473162  7626 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:27.540987  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:27.541685  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47037.1us
W20260709 13:03:27.541949  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69435.8us
I20260709 13:03:27.542503  1523 leader_election.cc:400] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:27.542793  1523 leader_election.cc:403] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:27.463321  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.510651  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.511397  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2987) took 2765 ms. Trace:
I20260709 13:03:27.471647  7957 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:27.513842  2292 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:27.506004  7719 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:27.451392  7832 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:03:27.438402  7942 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.519400  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2988) took 2784 ms. Trace:
I20260709 13:03:27.504509  7984 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:27.561324  1631 rpcz_store.cc:276] 0709 13:03:24.734682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.253667 (+518985us) service_pool.cc:224] Handling call
0709 13:03:25.283933 (+ 30266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.380576 (+ 96643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.380624 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.396022 (+ 15398us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.352929 (+956907us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:27.190481 (+837552us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:27.466378 (+275897us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.491652 (+ 25274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.491753 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.519107 (+ 27354us) raft_consensus.cc:1685] finished
0709 13:03:27.519190 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.519337 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161277,"prepare.queue_time_us":388283,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":127415,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":22785,"spinlock_wait_cycles":9672576,"thread_start_us":2009149,"threads_started":2,"wal-append.queue_time_us":1687892}
I20260709 13:03:27.437736  7842 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:03:27.446416  7952 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:03:27.448001  7654 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.531587  1709 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:27.532264  1551 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.532936  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:27.568852  1557 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.569068  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.570776  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.573745  1709 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:27.437930  7819 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:03:27.430037  7884 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.451671  7860 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:03:27.579469  6983 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:27.535316  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.492635  7767 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:03:27.537823  1630 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:27.459335  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1062) took 2719 ms. Trace:
I20260709 13:03:27.480983  1634 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:27.538923  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.432353  2293 rpcz_store.cc:276] 0709 13:03:25.256358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.257072 (+   714us) service_pool.cc:224] Handling call
0709 13:03:25.257380 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.257737 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.257777 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.258224 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.280951 (+1022727us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:26.280994 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.281065 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.320096 (+1039031us) raft_consensus.cc:1685] finished
0709 13:03:27.320117 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.320295 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3166,"raft.queue_time_us":793,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":16306,"thread_start_us":960684,"threads_started":1,"wal-append.queue_time_us":1469706}
I20260709 13:03:27.452833  7858 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.540412  1558 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:27.492010  7847 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:27.534422  7853 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:27.504065  7772 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.544211  1626 rpcz_store.cc:276] 0709 13:03:24.734451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.252618 (+518167us) service_pool.cc:224] Handling call
0709 13:03:25.252987 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.269080 (+ 16093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.269124 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.269706 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.236027 (+966321us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:27.407593 (+1171566us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.407862 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.407965 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.499838 (+ 91873us) raft_consensus.cc:1685] finished
0709 13:03:27.499862 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.500019 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84899,"prepare.queue_time_us":409277,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":5061,"raft.queue_time_us":51794,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":34658,"spinlock_wait_cycles":2816,"thread_start_us":1977444,"threads_started":2,"wal-append.queue_time_us":1420831}
I20260709 13:03:27.551851  1553 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:27.504864  7871 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.612347  7979 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:03:27.534451  7833 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:27.612490  7871 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.613318  7884 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:27.613633  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2345) took 1043 ms. Trace:
I20260709 13:03:27.482801  7843 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:03:27.614028  1554 rpcz_store.cc:276] 0709 13:03:26.570561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.744493 (+173932us) service_pool.cc:224] Handling call
0709 13:03:26.744835 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.745128 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.745162 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.745628 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.209374 (+463746us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:27.209453 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.209604 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.613419 (+403815us) raft_consensus.cc:1685] finished
0709 13:03:27.613441 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.613572 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":455328,"threads_started":1,"wal-append.queue_time_us":708021}
I20260709 13:03:27.511234  7722 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:27.615166  7777 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:27.564136  7814 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:27.616117  7814 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:27.616469  7985 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:27.616600  7814 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:27.616915  1629 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
W20260709 13:03:27.617720  7974 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:03:27.530812  1701 raft_consensus.cc:2376] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 15. Re-sending same reply.
I20260709 13:03:27.533197  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.619498  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.619956  7813 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:27.577885  2294 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:27.620668  7813 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:27.587787  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:27.590651  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:27.620230  1701 raft_consensus.cc:2376] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17. Re-sending same reply.
I20260709 13:03:27.622551  7972 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:27.622869  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.623539  1701 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:27.446717  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:27.492118  7649 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.548884  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:27.537019  7939 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:27.603693  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.585881  1702 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:27.631956  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.633179  1702 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:27.623155  7972 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.609016  7654 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.609594   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:27.565423  7984 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.615922  7722 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:27.620048  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 86913.9us
W20260709 13:03:27.635619  7937 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:27.625511  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.626564  1700 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:27.583932  1703 rpcz_store.cc:276] 0709 13:03:24.739741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.701645 (+961904us) service_pool.cc:224] Handling call
0709 13:03:25.702088 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.702343 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:26.539714 (+837371us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.458925 (+919211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.458988 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:27.459045 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.459124 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.459272 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52138,"prepare.queue_time_us":1253510,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":53169,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":26037,"thread_start_us":1657285,"threads_started":2}
I20260709 13:03:27.680562  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.680902  1630 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:27.681414  1703 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:03:27.619529  7942 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.629384  1552 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:27.554864  7290 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.514439  7622 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.637434  1557 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:27.594805  1709 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:27.580749  7968 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:27.638132  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 147846us
I20260709 13:03:27.693166  1551 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:27.660825  2292 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:27.633548  7982 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:03:27.665876  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 117083us
W20260709 13:03:27.591056  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48679.5us
I20260709 13:03:27.615581  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.721377  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:27.630336  7719 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:27.686908  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.723320  1709 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:27.723908  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.724455  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:27.724642  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:27.725936  1709 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:27.635234  7951 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:27.742940  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:27.743813  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.743750  2294 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:27.682812  1558 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:27.687408  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.682327  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.745435  1703 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:27.688134  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.707058  7870 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.746270  1557 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:27.745849  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:27.747316  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.666988  7984 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:27.635105  7690 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:27.651196  1701 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
W20260709 13:03:27.696854  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 131053us
I20260709 13:03:27.764257  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.764777  1701 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:27.630942  7847 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.698007  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:27.699539  7290 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:27.774623  1701 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:27.700779  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:27.704409  7996 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:03:27.716480  7968 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.780546  1632 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:27.709234  7865 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:27.557916  7922 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:27.790778  7990 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:27.652092  7853 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:27.713384  7961 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.401s	user 0.002s	sys 0.000s
I20260709 13:03:27.640113  7884 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 13:03:27.714783  7960 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.416s	user 0.000s	sys 0.001s
I20260709 13:03:27.624264  7316 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.662896  7957 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.721835  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:27.740926  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 294295us
I20260709 13:03:27.748112  1557 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:27.688657  7578 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16.
I20260709 13:03:27.748972  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:27.745960  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:27.760824  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.769114  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:27.769665  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:27.815198  2291 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:27.815402  1552 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:27.816627  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.777638  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:27.704933  7988 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:03:27.706430  7929 ts_tablet_manager.cc:1916] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:27.793744  7958 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:27.780995  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2346) took 1210 ms. Trace:
I20260709 13:03:27.781253  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2343) took 1211 ms. Trace:
I20260709 13:03:27.784030  1702 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:27.789274   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:03:27.789893  7992 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:03:27.789764  7975 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.790503  8006 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:27.790972  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:27.693497  7832 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:03:27.805670  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.817458  1552 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:27.747504  7719 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:27.819523  1554 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:27.780241  8001 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.793521  7802 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:27.787006  7968 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:27.821871  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75994.5us
I20260709 13:03:27.847525  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:27.837565  7991 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:03:27.849126  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.837443  7975 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:27.850564  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1413) took 1177 ms. Trace:
I20260709 13:03:27.845167  8006 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:27.851145  2288 rpcz_store.cc:276] 0709 13:03:26.672971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.702807 (+ 29836us) service_pool.cc:224] Handling call
0709 13:03:26.754117 (+ 51310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.758739 (+  4622us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:27.207209 (+448470us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.285065 (+ 77856us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.285726 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.285876 (+   150us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:27.285912 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.285992 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.850259 (+564267us) raft_consensus.cc:1685] finished
0709 13:03:27.850291 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.850483 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":468474,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":179185,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":19235,"thread_start_us":505976,"threads_started":2,"wal-append.queue_time_us":384893}
I20260709 13:03:27.853240  8006 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.856866  7913 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:27.825326  7854 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:27.822086  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 73196us
I20260709 13:03:27.818342  1558 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:27.822435  1703 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:27.826968  7958 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.824774  1556 rpcz_store.cc:276] 0709 13:03:26.569936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.743733 (+173797us) service_pool.cc:224] Handling call
0709 13:03:26.744138 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.340407 (+596269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.340562 (+   155us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.364077 (+ 23515us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.473445 (+109368us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:27.473511 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.473637 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.781044 (+307407us) raft_consensus.cc:1685] finished
0709 13:03:27.781069 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.781202 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":436166,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":704264,"threads_started":2,"wal-append.queue_time_us":315065}
I20260709 13:03:27.823499  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.802958  7952 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:27.830732  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:27.835101  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:27.865573  1703 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:03:27.824654  1555 rpcz_store.cc:276] 0709 13:03:26.570827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.352466 (+781639us) service_pool.cc:224] Handling call
0709 13:03:27.352835 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.360907 (+  8072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.360941 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.361396 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.521099 (+159703us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:27.521183 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.521371 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.780767 (+259396us) raft_consensus.cc:1685] finished
0709 13:03:27.780789 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.780927 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4384,"raft.queue_time_us":83,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":22039,"thread_start_us":102990,"threads_started":1,"wal-append.queue_time_us":339236}
I20260709 13:03:27.882655  1557 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:27.837214  7987 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:27.890856  1558 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:27.842798  7967 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:27.904675  7967 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:27.905644  7967 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.875463  7952 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.907562  7952 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.910651  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 2985) took 3176 ms. Trace:
I20260709 13:03:27.843854  7767 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:03:27.888612  1557 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:27.911224  1635 rpcz_store.cc:276] 0709 13:03:24.733970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.220392 (+486422us) service_pool.cc:224] Handling call
0709 13:03:25.220698 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.326468 (+105770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.326513 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.327064 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.414640 (+1087576us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:27.360491 (+945851us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.362213 (+  1722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.362505 (+   292us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.622841 (+260336us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b48001bc258. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:27.909781 (+286940us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b2400289b98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:03:27.910351 (+   570us) raft_consensus.cc:1685] finished
0709 13:03:27.910383 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.910573 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196334,"prepare.queue_time_us":368271,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3543,"raft.queue_time_us":139335,"raft.run_cpu_time_us":2875,"raft.run_wall_time_us":261616,"spinlock_wait_cycles":65386240,"thread_start_us":1818755,"threads_started":2,"wal-append.queue_time_us":1446609}
W20260709 13:03:27.888976  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 167273us
I20260709 13:03:27.889422  1551 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:27.890616  7983 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:27.895995  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2347) took 1324 ms. Trace:
I20260709 13:03:27.897006  7870 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.898082  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2908) took 2295 ms. Trace:
I20260709 13:03:27.923877  1560 rpcz_store.cc:276] 0709 13:03:26.571102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.415280 (+844178us) service_pool.cc:224] Handling call
0709 13:03:27.455247 (+ 39967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.456752 (+  1505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.456809 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.457422 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.626056 (+168634us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.626213 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.626357 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.895782 (+269425us) raft_consensus.cc:1685] finished
0709 13:03:27.895804 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.895934 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4347,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":13540,"thread_start_us":168100,"threads_started":1,"wal-append.queue_time_us":417315}
I20260709 13:03:27.924158  1627 rpcz_store.cc:276] 0709 13:03:25.603040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.970684 (+367644us) service_pool.cc:224] Handling call
0709 13:03:25.971126 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.971618 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:25.971659 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.004570 (+ 32911us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.210169 (+1205599us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:27.210226 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.210315 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.714134 (+503819us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:27.897725 (+183591us) raft_consensus.cc:1685] finished
0709 13:03:27.897755 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.898009 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8391,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":50206464,"thread_start_us":1138144,"threads_started":1,"wal-append.queue_time_us":1242708}
I20260709 13:03:27.842190  7802 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.900626  1555 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:27.903447  7972 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:27.903988  7976 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.806166  8002 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 13:03:27.921066  7975 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:27.927388  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.866308  7854 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:27.875208  7958 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:27.886976  7832 raft_consensus.cc:2764] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:03:27.936713  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 193007us
W20260709 13:03:27.939419  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 169816us
I20260709 13:03:27.918977  7865 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:27.952250  7865 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.952744  7865 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:27.953104  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.953871  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:27.959728  7843 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.960398  7843 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:27.961268  1551 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:27.962714  7843 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:27.963927  7865 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.967221  7865 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:27.967424  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 136651us
I20260709 13:03:27.927793  7983 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:27.970819  2263 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:27.922503  7987 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:27.986744  7987 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:27.975970  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:27.936347  7802 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:27.981640  1555 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:27.941565  7622 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:27.997740  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1416) took 1081 ms. Trace:
I20260709 13:03:27.999778  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:27.944350  7854 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:28.002734  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:28.001727  2290 rpcz_store.cc:276] 0709 13:03:26.916748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.249169 (+332421us) service_pool.cc:224] Handling call
0709 13:03:27.304715 (+ 55546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.305082 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.305115 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.305645 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.985419 (+679774us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b180034ef40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1
0709 13:03:27.985534 (+   115us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:27.985557 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.985605 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.985635 (+    30us) raft_consensus.cc:1685] finished
0709 13:03:27.985641 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.997667 (+ 12026us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15823,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":22598,"spinlock_wait_cycles":30812928,"thread_start_us":87424,"threads_started":1,"wal-append.queue_time_us":212661}
I20260709 13:03:27.984257  7983 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.940546  8003 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:03:28.026024  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50143.5us
W20260709 13:03:28.036870  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 83077us
I20260709 13:03:28.004276  7832 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:28.008539  7719 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.021055  7801 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:28.046447  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.020167  7865 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:28.049593  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2330) took 1836 ms (client timeout 1994 ms). Trace:
I20260709 13:03:28.048406  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2825) took 3217 ms. Trace:
I20260709 13:03:28.060073  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.027676  8005 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:28.060359  1553 rpcz_store.cc:269] 0709 13:03:26.213601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.214969 (+  1368us) service_pool.cc:224] Handling call
0709 13:03:28.049429 (+1834460us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003098d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:28.049527 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3006249856}
I20260709 13:03:28.065696  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:27.945297  7976 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.066787  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.059216  1559 rpcz_store.cc:276] 0709 13:03:24.831328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.107893 (+276565us) service_pool.cc:224] Handling call
0709 13:03:25.108160 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.178209 (+ 70049us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:25.902434 (+724225us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.481198 (+1578764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.481271 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:27.481317 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.048142 (+566825us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:28.048208 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.048348 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":965460,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":17817,"raft.queue_time_us":367870,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":64006912,"thread_start_us":2242646,"threads_started":3,"wal-append.queue_time_us":1178239}
I20260709 13:03:28.069157  1553 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:28.069588  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.070036  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.071000  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.072593  1555 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.074910  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.075090  1559 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:28.075503  1559 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:28.076452  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.077138  1706 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.077247  1559 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.081880  8012 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:28.082124  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.082406  8012 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.082791  8012 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.090942  7922 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.079479  8015 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.092124  8015 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.094437  8015 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.063014  7832 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:28.063629  7801 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:28.083451  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.096923  7832 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.104069  7984 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.070600  7719 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.111151  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.113903  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.115518  1703 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:28.116200  1560 raft_consensus.cc:2368] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21.
I20260709 13:03:28.078490  8003 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:28.120051  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2349) took 1354 ms. Trace:
I20260709 13:03:28.123275  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1409) took 1609 ms. Trace:
I20260709 13:03:28.131978  2292 rpcz_store.cc:276] 0709 13:03:26.513500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.513719 (+   219us) service_pool.cc:224] Handling call
0709 13:03:28.123109 (+1609390us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b54006c02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:28.123215 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213085056}
I20260709 13:03:28.109967  7801 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.130884  1552 rpcz_store.cc:276] 0709 13:03:26.765991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.847390 (+1081399us) service_pool.cc:224] Handling call
0709 13:03:27.873904 (+ 26514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.020110 (+146206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.020140 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.020615 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.062376 (+ 41761us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:28.062432 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.062532 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.119823 (+ 57291us) raft_consensus.cc:1685] finished
0709 13:03:28.119846 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.119992 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39056,"raft.queue_time_us":52259,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8657,"thread_start_us":172418,"threads_started":2,"wal-append.queue_time_us":97509}
W20260709 13:03:28.146456  8018 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:03:28.121248  7814 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:03:28.139098  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:28.123845  7944 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.567s	user 0.004s	sys 0.007s
I20260709 13:03:28.154160  1706 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:28.165177  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.166204  1703 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:28.130816  8003 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:28.138934  8002 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.169916  1706 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:28.121057  7719 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:28.175292  8011 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:28.168879  1555 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
W20260709 13:03:28.237937  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 117383us
I20260709 13:03:28.153093  8012 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.160655  7814 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.161327  7944 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.260442  8012 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.266021  7980 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:28.266721  7980 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:28.270946  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1405) took 2185 ms. Trace:
I20260709 13:03:28.271545  2296 rpcz_store.cc:276] 0709 13:03:26.085655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.085790 (+   135us) service_pool.cc:224] Handling call
0709 13:03:26.086108 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.363087 (+276979us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:27.234000 (+870913us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.420503 (+186503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.420555 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:27.420597 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.102936 (+682339us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b180025ed00. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419
0709 13:03:28.270603 (+167667us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:28.270676 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.270867 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4018,"prepare.queue_time_us":863889,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":4391,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":16987,"spinlock_wait_cycles":655560192,"thread_start_us":1028828,"threads_started":2,"wal-append.queue_time_us":399807}
W20260709 13:03:28.276536  8025 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:03:28.276612  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.277554  1552 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:28.287943  8004 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.288575  8004 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.289866  7858 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:28.290737  8004 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.290760  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.306277  1632 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:28.308986  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.310019  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:28.310022  7963 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.959s	user 0.004s	sys 0.000s
I20260709 13:03:28.312743  1632 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:28.314615  2262 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.551s (SENT)
I20260709 13:03:28.306423  7984 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.317725  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.318099  1635 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:28.317937  1627 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:28.252682  8015 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.248374  7853 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.320807  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.322090  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.322345  8015 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.323745  1632 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:28.325343  8015 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.329137  7719 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.331135  1555 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:28.331801  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1082) took 1094 ms. Trace:
I20260709 13:03:28.332062  1703 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:28.332267  7814 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.333525  7976 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.332171  1700 rpcz_store.cc:276] 0709 13:03:27.236948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.535065 (+298117us) service_pool.cc:224] Handling call
0709 13:03:28.331618 (+796553us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b540010d6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:28.331729 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106254592}
I20260709 13:03:28.335527  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.188465  8002 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.337466  1633 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:28.339064  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.339828  1635 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:28.340777  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.342298  7944 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.349398  8002 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.352022  7855 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:28.352560  7855 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:28.353567  7855 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:28.363937  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.366691  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.372217  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:28.372905  2290 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
W20260709 13:03:28.249220  7935 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.079s	user 0.001s	sys 0.008s
W20260709 13:03:28.378175  7989 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.049s	user 0.002s	sys 0.001s
I20260709 13:03:28.378881  2289 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.249768  7952 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.378931  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.404450  1701 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:28.379402  7865 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:28.379734  8005 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:28.381263  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:28.439101  2288 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:28.389694  7889 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.390825  7626 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:28.391499  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:28.392084  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:28.448237  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 56224.2us
I20260709 13:03:28.448582  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.449662  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.374406  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:28.392709  7939 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.002s
I20260709 13:03:28.452171  7939 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.452872  7858 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.453447  7858 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.453925  7858 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.454460  7939 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.299542  7801 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.395174  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.451710  2263 leader_election.cc:400] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21.
I20260709 13:03:28.457144  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.457479  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.457755  2263 leader_election.cc:403] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.376029  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:28.415885  1523 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.641s (SENT)
I20260709 13:03:28.421548  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:28.444857  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.376425  7855 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.362952  7802 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:28.459492  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43673.6us
I20260709 13:03:28.456339  1635 raft_consensus.cc:2376] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15. Re-sending same reply.
W20260709 13:03:28.467487  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 91527.5us
I20260709 13:03:28.446930  8005 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:28.479290  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 203236us
I20260709 13:03:28.479189  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.480234  1708 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:28.480741  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.481194  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:28.481820  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.448691  7626 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.395687  7854 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.462369  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.403658  7814 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:03:28.493821  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.492910  1632 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
W20260709 13:03:28.504271  1523 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.787s (SENT)
I20260709 13:03:28.504784  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:28.459446  1443 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68157.5us
I20260709 13:03:28.367915  8003 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.478176  7976 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.508378  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:28.493096  7893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.018s	sys 0.032s
I20260709 13:03:28.492173  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:28.447168  7983 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.313494  7980 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.514921  1708 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.505561  7987 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.497735  7845 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.510183  8006 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.516167  7832 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:28.519709  6983 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:28.542920  7845 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.566095  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:28.566805  8031 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:28.566887  7316 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.567528  8031 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.567970  8031 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.568210  7845 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.568337  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:28.568923  8013 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.569697  7316 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:28.569609  8013 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.570196  7316 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.570704  7316 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:28.528198  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.536513  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:28.579756  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53754us
W20260709 13:03:28.562845  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 88235.2us
W20260709 13:03:28.600675  1523 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.806s (SENT)
I20260709 13:03:28.601511  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.581668  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:28.604206  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:28.618672  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 167107us
I20260709 13:03:28.619623  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:28.620446  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:28.620903  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:28.543416  7958 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.009s
I20260709 13:03:28.631688  6983 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:28.646806  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:28.551538  8003 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:28.563604  7865 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21.
W20260709 13:03:28.516822  7893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.018s	sys 0.032s
I20260709 13:03:28.543924  7801 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:28.651314  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50691.1us
I20260709 13:03:28.618052  7959 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:28.603057  7845 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.645782  6983 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:28.659955  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:28.668889  8043 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:28.682607  7983 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.683691  8043 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:28.688145  7865 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:28.688787  7865 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:28.689662  7865 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.633198  7773 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.631186  7843 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:28.561128  7987 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.709544  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3008) took 1291 ms. Trace:
I20260709 13:03:28.644737  7777 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:28.517095  7976 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.645264  7813 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:28.693611  1708 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:28.672504  7959 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.717255  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.718485  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.656932  7690 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:28.670881  1523 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.843s (SENT)
I20260709 13:03:28.657253  7958 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.702369  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2859) took 1122 ms. Trace:
I20260709 13:03:28.666947  7944 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
W20260709 13:03:28.704152  8041 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:03:28.704015  7994 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:28.705156  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:28.705727  8002 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:28.707307  7893 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:28.709992  1630 rpcz_store.cc:276] 0709 13:03:27.418268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.587509 (+169241us) service_pool.cc:224] Handling call
0709 13:03:28.709356 (+1121847us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b54002751d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:28.709479 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135650176}
I20260709 13:03:28.719569  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.724144  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:28.742195  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71362.8us
I20260709 13:03:28.742537  1708 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:28.703137  8034 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:03:28.743656  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.744297  7773 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.744802  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.660934  7814 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 13:03:28.745867  7968 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.863s	user 0.008s	sys 0.003s
I20260709 13:03:28.746327  7968 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.747099  7944 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:28.747505  7944 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.747874  7944 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.727699  7952 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.750002  7654 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:28.750460  7654 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:28.751070  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1086) took 1027 ms. Trace:
I20260709 13:03:28.664268  7832 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 13:03:28.726995  1556 rpcz_store.cc:276] 0709 13:03:27.580197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.888296 (+308099us) service_pool.cc:224] Handling call
0709 13:03:27.888641 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.893661 (+  5020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.893707 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.923965 (+ 30258us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.281102 (+357137us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b18002ad000. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:28.478211 (+197109us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.478272 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.478372 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.691934 (+213562us) raft_consensus.cc:1685] finished
0709 13:03:28.691958 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.702266 (+ 10308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55840,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10884,"spinlock_wait_cycles":15927040,"thread_start_us":196868,"threads_started":1,"wal-append.queue_time_us":721405}
I20260709 13:03:28.690662  8013 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.719376  7987 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:28.719169  7959 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:28.737907  1598 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:28.751475  1706 rpcz_store.cc:276] 0709 13:03:27.723729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.179889 (+456160us) service_pool.cc:224] Handling call
0709 13:03:28.180346 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.189146 (+  8800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.189197 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.189929 (+   732us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.514241 (+324312us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:28.514282 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.514349 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.750810 (+236461us) raft_consensus.cc:1685] finished
0709 13:03:28.750833 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.751000 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8220,"raft.queue_time_us":16084,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3697,"thread_start_us":257760,"threads_started":1,"wal-append.queue_time_us":500542}
I20260709 13:03:28.765116  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.765924  1706 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.746347  7814 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:28.778841  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:28.738603  7854 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.738329  7802 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:28.739266  7994 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:28.782778  1706 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:28.783016  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.817826  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.819312  1629 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:28.755743  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 171764us
I20260709 13:03:28.786290  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:28.787019  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.820227  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.820641  1706 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [14.7-15.8]
I20260709 13:03:28.822803  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.789494  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.823951  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.824326  7637 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:28.825083  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.794569  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.784722  1635 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:28.825913  1554 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:28.790839  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.776249  8002 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:28.800014  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.776067  7814 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:28.826450  1706 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:28.827247  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.827991  8002 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:28.827612  7814 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.751272  7654 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.828789  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.828946  8002 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.776141  7893 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.828989  7814 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.828079  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.830096  7893 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:28.830096  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.831147  1632 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:28.829238  1706 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:28.832340  7893 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.832885  7893 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:28.833801  7893 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:28.797284  1627 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.834168  7893 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:28.824872  7637 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.829372  7654 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.836624  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.826651  7832 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.836767  7637 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:28.826542  1631 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:28.839102   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:28.839927  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.840505  1560 raft_consensus.cc:2376] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19. Re-sending same reply.
I20260709 13:03:28.840479  7893 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.842149  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.842695  1555 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.826752  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:28.843616  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:28.845144  7801 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:28.845547  7801 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.845927  7801 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.793027  8013 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:28.796629  7853 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:28.847275  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.819111  7958 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:28.814944  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:28.815827  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63201.8us
W20260709 13:03:28.848079  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 225334us
I20260709 13:03:28.852141  1556 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:28.847438  1706 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:28.853729  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:28.847892  7893 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.868789  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:28.876755  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.878052  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.889112  1555 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:28.889742  1635 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:28.878424  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.892974  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.894898  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.896184  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:28.896932  1554 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:03:28.867223  7819 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.202s	user 0.009s	sys 0.009s
W20260709 13:03:28.905118  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 284778us
I20260709 13:03:28.906322  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:28.906680  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:28.909583  1634 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:28.912070  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.907617  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:28.904031  7819 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.010s	sys 0.009s
I20260709 13:03:28.913717  7578 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:03:28.917404  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.925921  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:28.926751  1700 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:28.933833  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.905305  7968 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:28.865171  7853 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.931182  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:28.886696  7987 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:28.944713  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:28.920670  7813 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:28.860584  8013 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.940532  8006 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:28.924604  7976 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:28.932076  7316 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:28.931754  7819 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:28.904723  7858 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:28.934252  7967 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:28.966894  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:28.967749  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.968472  7719 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:28.969245  7861 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:03:28.972318  7854 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:28.977653  8023 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.734s	user 0.003s	sys 0.000s
W20260709 13:03:29.011009  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.013137  7929 ts_tablet_manager.cc:1929] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.7
I20260709 13:03:28.946959  7845 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:03:29.017488  7823 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:29.019745  8040 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:03:28.921077  7860 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:03:29.031308  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.958736  7813 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.959039  8049 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:03:29.032191  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 101076us
I20260709 13:03:28.964350  7626 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.034802  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2405) took 1459 ms. Trace:
I20260709 13:03:29.061683  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.061480  1703 rpcz_store.cc:276] 0709 13:03:27.575419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.167151 (+591732us) service_pool.cc:224] Handling call
0709 13:03:28.331732 (+164581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.332485 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.034522 (+702037us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540010d6d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:29.034721 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373279616}
W20260709 13:03:29.044262  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77476us
I20260709 13:03:28.940904  7773 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:03:29.049348  1630 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:29.043708  7719 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:29.032827  7976 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:28.940920  7994 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.063350  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:29.068826  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.048539  8020 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:29.065589  1635 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:29.063467  7861 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:29.066937  1703 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:29.059402  7813 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.089686  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.101532  7719 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.102974  7719 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.064129  7787 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.105301  7787 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.109165  7944 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.110874  1631 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:29.067456  7983 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:29.112574  7983 consensus_queue.cc:1243] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:29.113291  1559 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.115530  7626 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.061481  7929 log.cc:1199] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 444b494bb53842b59408a7230884398a: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-0-root/wals/0ab7f997f0f54ab6bfebcbbd01c136d6
W20260709 13:03:29.121073  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.121425  7929 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51896 (request call id 0) took 3920 ms. Trace:
I20260709 13:03:29.121865  7929 rpcz_store.cc:276] 0709 13:03:25.201505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.202581 (+  1076us) service_pool.cc:224] Handling call
0709 13:03:26.299674 (+1097093us) ts_tablet_manager.cc:1151] Deleting tablet 0ab7f997f0f54ab6bfebcbbd01c136d6
0709 13:03:26.299834 (+   160us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:03:28.295556 (+1995722us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:29.012963 (+717407us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:29.121322 (+108359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":377778,"tablet-delete.queue_time_us":1086812,"thread_start_us":687805,"threads_started":1}
I20260709 13:03:29.124042  1445 catalog_manager.cc:5028] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): tablet 0ab7f997f0f54ab6bfebcbbd01c136d6 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
I20260709 13:03:29.130385  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.131173  2292 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:29.131206  7690 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:29.132725  7690 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:29.133736  7690 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.053329  8031 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.139349  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.139412  7777 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.141067  1708 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:29.141366  8031 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:29.141983  8005 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
W20260709 13:03:29.142886  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.143409  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.145802  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2940) took 1932 ms. Trace:
I20260709 13:03:29.097697  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:29.150444  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52868.9us
I20260709 13:03:29.152035  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:29.153224  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.154143  1555 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.142552  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:29.146385  1709 rpcz_store.cc:276] 0709 13:03:27.213838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.750332 (+536494us) service_pool.cc:224] Handling call
0709 13:03:27.750785 (+   453us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:27.773484 (+ 22699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.773528 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:28.333381 (+559853us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b2c000b1c58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:28.334080 (+   699us) log.cc:844] Serialized 78 byte log entry
0709 13:03:28.484646 (+150566us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.511243 (+ 26597us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.511474 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.511559 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.019050 (+507491us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:29.145485 (+126435us) raft_consensus.cc:1685] finished
0709 13:03:29.145516 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.145724 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26380,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":27616,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":180,"spinlock_wait_cycles":187828480,"thread_start_us":150216,"threads_started":1,"wal-append.queue_time_us":399851}
I20260709 13:03:29.170086  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.172950  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.174152  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.175165  2294 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:29.175282  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:29.176855  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:29.178205  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:29.067312  7976 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.186640  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2860) took 1555 ms. Trace:
I20260709 13:03:29.181005  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.182580  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.183709  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:29.141366  7889 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.001s
I20260709 13:03:29.187150  1558 rpcz_store.cc:276] 0709 13:03:27.632265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.893753 (+261488us) service_pool.cc:224] Handling call
0709 13:03:27.894127 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.894341 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.894365 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.937466 (+ 43101us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.070881 (+133415us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:28.070905 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.070952 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.976688 (+905736us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:29.167654 (+190966us) raft_consensus.cc:1685] finished
0709 13:03:29.167676 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.186551 (+ 18875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7796480,"thread_start_us":79814,"threads_started":1,"wal-append.queue_time_us":183288}
I20260709 13:03:29.090232  7854 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:29.089471  7861 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:29.195116  1560 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:29.194139  1709 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:29.192235  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:29.143193  7773 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:29.213372  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.213711  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.205106  1633 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:29.205570  1634 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:29.207984  1559 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:03:29.215986  1559 pending_rounds.cc:85] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:29.216343  1559 pending_rounds.cc:107] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:03:29.216207  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.216883  1559 raft_consensus.cc:2889] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.184062  7819 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:29.183336  8031 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:29.219193  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.213541  1555 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:29.186116  8043 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:29.184185  8006 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:29.179751  7832 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:29.174340  7865 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.204242  7981 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:29.198195  8017 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:29.192350  7976 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.220423  7952 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:03:29.142464  8005 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.192238  8020 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:29.236384  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.238327  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:29.195947  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.270190  2292 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.210867  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.194743  7690 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:29.232795  8003 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.234355  8031 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.217422  7942 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.231871  7855 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.006s
I20260709 13:03:29.217859  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1024) took 1168 ms. Trace:
I20260709 13:03:29.223974  7802 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:29.228854  7787 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.266495  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:29.299213  1559 rpcz_store.cc:276] 0709 13:03:28.049456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.074635 (+ 25179us) service_pool.cc:224] Handling call
0709 13:03:28.074977 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.112891 (+1037914us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5800771ed0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9400aa 00007f70ce9c2447 00007f70ce9dafe7 00007f70ce9dc7de 00007f70ce9e6b49 00007f70ce9e7416 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4
0709 13:03:29.217553 (+104662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.217789 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136336768}
I20260709 13:03:29.261552  8006 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.227197  7860 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.253368  7870 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:29.250206  7854 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:29.257756  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:29.253142  7773 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.198160  7654 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.271863  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61057.3us
I20260709 13:03:29.345575  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.255808  7316 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:29.298290  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.258625  8050 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.301121  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.299727  7853 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.353062  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.353866  1559 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:29.301599  2288 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:29.302147  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.356659  1708 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:29.260926  7819 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.267160  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.355440  1559 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:29.307067  2289 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:29.258880  8043 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.233439  7578 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:03:29.380446  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.274415  7958 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.280826  7772 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.271100  8017 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.306358  7787 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:29.337141  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 79454.2us
I20260709 13:03:29.297875  8031 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.336609  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.298853  7814 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:29.343161  2292 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:29.402190  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.349740  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2866) took 1372 ms. Trace:
I20260709 13:03:29.358848  7855 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:29.307847  8006 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:29.338042  7854 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.311132  7870 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:29.382887  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.268944  7861 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:29.431905  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.387822  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.388085  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.355473  7802 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:29.341485  8013 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.358588  7975 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:29.353462  7858 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.369035  8050 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.402977  1551 rpcz_store.cc:276] 0709 13:03:27.977267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.977450 (+   183us) service_pool.cc:224] Handling call
0709 13:03:27.977867 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.978139 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:27.978634 (+   495us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.114312 (+1135678us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18003aa6e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:03:29.349235 (+234923us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.349314 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:29.349377 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.349493 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.349658 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":6663,"raft.queue_time_us":68424,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8860,"spinlock_wait_cycles":6291840,"thread_start_us":234755,"threads_started":1}
I20260709 13:03:29.404888  2290 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:29.435992  8043 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:29.456739  7801 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 13:03:29.456991  8050 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.429071  8054 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:03:29.261385  7865 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.458797  7649 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.460383  7865 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.461361  2293 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:29.353098  7773 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.442378  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2397) took 1891 ms. Trace:
I20260709 13:03:29.444407  1635 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:29.302981  7942 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.359736  7654 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.444950  1708 raft_consensus.cc:2376] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 16. Re-sending same reply.
W20260709 13:03:29.412182  8030 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.873s	user 0.003s	sys 0.000s
I20260709 13:03:29.438422  8007 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:29.452183  7861 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.438140  8045 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:03:29.453116  8066 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:03:29.454352  7893 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.002s
I20260709 13:03:29.390750  7787 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:29.462507  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 193992us
I20260709 13:03:29.417470  7854 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:29.416832  7860 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.401443  7814 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.494248  8003 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:29.495321  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.495461  7654 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.465600  1702 rpcz_store.cc:276] 0709 13:03:27.550722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.825776 (+275054us) service_pool.cc:224] Handling call
0709 13:03:27.826092 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.827734 (+  1642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.827787 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.037702 (+1209915us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b2c000b1c58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:29.042717 (+  5015us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.277206 (+234489us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.277249 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.277438 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.442016 (+164578us) raft_consensus.cc:1685] finished
0709 13:03:29.442043 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.442305 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1270,"raft.queue_time_us":1396,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8257,"spinlock_wait_cycles":7581056,"thread_start_us":220829,"threads_started":1,"wal-append.queue_time_us":389645}
I20260709 13:03:29.497402  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.498795  1702 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:29.492053  1558 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:29.500752  8017 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.504853  7870 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.506007  7870 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.506450  7980 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:29.488983  8020 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:29.507843  8020 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.510246  8020 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.457139  7801 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:29.513912  8057 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.413s	user 0.007s	sys 0.000s
I20260709 13:03:29.514844  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2862) took 1833 ms. Trace:
I20260709 13:03:29.515139  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2410) took 1042 ms. Trace:
I20260709 13:03:29.515241  1557 rpcz_store.cc:276] 0709 13:03:27.681250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.913645 (+232395us) service_pool.cc:224] Handling call
0709 13:03:27.914074 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.924782 (+ 10708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.924812 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.925273 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.036569 (+1111296us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b1800313720. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:29.352704 (+316135us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:29.352747 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.352953 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.514553 (+161600us) raft_consensus.cc:1685] finished
0709 13:03:29.514573 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.514783 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14705,"raft.queue_time_us":58062,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1934,"spinlock_wait_cycles":19903872,"thread_start_us":315893,"threads_started":1,"wal-append.queue_time_us":1550628}
I20260709 13:03:29.518981  2292 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:29.519397  7773 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.397840  7958 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.520099  1702 pending_rounds.cc:85] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:29.520879  7773 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.521396  7773 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.521978  7958 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.522349  7813 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:29.523511  7773 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.523959  7860 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.492532  7787 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.525967  7860 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:29.528297  8062 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:03:29.528542  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.529241  1557 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.530329  1705 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:29.531000  7814 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.532064  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.490543  8045 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.534229  8045 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.535413  8045 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:29.520411  1702 pending_rounds.cc:107] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 12.13
I20260709 13:03:29.535991  1702 raft_consensus.cc:2889] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.536137  1632 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:29.536520  7957 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.462339  7871 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.515535  1701 rpcz_store.cc:276] 0709 13:03:28.472965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.816827 (+343862us) service_pool.cc:224] Handling call
0709 13:03:28.848597 (+ 31770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.848845 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:28.848872 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.862103 (+ 13231us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.038657 (+176554us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.038698 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.038782 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.513650 (+474868us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b24003e64c8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb93609d 00007f70ce56810e 00007f70ce57f88c 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221
0709 13:03:29.514875 (+  1225us) raft_consensus.cc:1685] finished
0709 13:03:29.514911 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.515085 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92783360,"thread_start_us":138558,"threads_started":1,"wal-append.queue_time_us":232763}
I20260709 13:03:29.531688  1705 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:29.550141  1558 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:29.551338  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.369586  7819 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.553447  1557 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:29.554280  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.557798  1556 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:29.471524  7802 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.561333  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.563074  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.565044  2297 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:29.555601  1673 connection.cc:256] RPC call timeout handler was delayed by -1.0383107509999263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 13:03:29.568043  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.569387  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.486011  8007 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:29.574044  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.577009  1556 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:29.578186  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:29.554517  7959 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.773s	user 0.007s	sys 0.004s
I20260709 13:03:29.579471  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.577960  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:29.504956  8005 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.596455  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.597191  2291 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:29.598927  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.579020  1556 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:29.601823  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.581353  2297 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:29.572656  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:29.580166  1560 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:03:29.553095  7981 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:29.635370  2297 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:29.611332  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:29.639092  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66521us
I20260709 13:03:29.584488  7853 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.640260  6983 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:29.647600  7853 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.640837  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.642299  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.643396  2291 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:29.635166  7801 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.654253  6983 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:29.644011  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:29.644937  7290 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:29.645089  7981 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.656244  7958 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:29.651703  7853 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.654712  6983 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.657096  6983 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:29.657277  7719 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.004s
I20260709 13:03:29.657449  7772 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:29.658049  7813 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:29.658331  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.585121  7871 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:03:29.661185  8031 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:29.662506  1551 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:29.638686  2297 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.656025  7802 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.666706  6983 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.664534  7942 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:29.668598  7986 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.955s	user 0.005s	sys 0.000s
I20260709 13:03:29.670959  2297 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.671742  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.673250  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.673691  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:29.675395  1628 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:29.676373  8007 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.679418  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1181) took 2867 ms. Trace:
I20260709 13:03:29.682847  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.683071  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.686862  7787 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.592648  7847 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.675136  1627 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:29.714913  2295 rpcz_store.cc:276] 0709 13:03:26.812232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.812956 (+   724us) service_pool.cc:224] Handling call
0709 13:03:26.834313 (+ 21357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.834761 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.834799 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.835376 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.287060 (+451684us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:27.287136 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.287311 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.668005 (+2380694us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:29.679033 (+ 11028us) raft_consensus.cc:1685] finished
0709 13:03:29.679058 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.679341 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145669,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9641,"spinlock_wait_cycles":25050496,"thread_start_us":371335,"threads_started":1,"wal-append.queue_time_us":788846}
W20260709 13:03:29.689901  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.720692  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.691123  8005 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:29.691954  8047 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.911s	user 0.000s	sys 0.005s
I20260709 13:03:29.694270  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:29.698280  7944 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.701042  2292 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:29.706705  1554 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:29.706452  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.691169  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:29.708071  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.727586  1555 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:29.730020  1555 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:29.709208  2291 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:29.710332  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.717224  2289 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:29.723477  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:29.732908  1707 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:29.733279  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.734035  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.738745  2297 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:03:29.739413  2297 pending_rounds.cc:85] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 12
I20260709 13:03:29.739877  1707 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:29.714298  7719 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.717833  7871 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.761611  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.762444  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.764515  1707 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:29.746099  8001 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.007s
I20260709 13:03:29.725737  7861 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.739887  2297 pending_rounds.cc:107] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:03:29.670864  8006 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.593106  7959 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.772097  1560 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:03:29.788888  2297 raft_consensus.cc:2889] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:29.808235  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.810295  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.814569  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.771123  7847 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.748123  7772 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:29.743441  8005 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:29.770882  8001 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.013s	sys 0.007s
I20260709 13:03:29.858373  1709 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:29.853909  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.862402  2289 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:29.819590  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.835633  1707 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:29.851719  7842 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:03:29.867614  1560 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:29.742621  7814 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.873973  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:29.875278  7787 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.877838  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.879343  7959 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:29.857430  7847 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.880362  8061 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:29.881228  8006 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:29.819006  7861 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.771355  7967 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:03:29.882156  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:29.836620  8075 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.883538  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:29.886489  1598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.205s (SENT) [suppressed 23 similar messages]
I20260709 13:03:29.766116  7719 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:29.888979  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.771822  7871 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.857913  7958 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:29.946928  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3010) took 1070 ms. Trace:
I20260709 13:03:29.898562  2291 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:29.881060  7942 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:29.909482  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:29.918455  1560 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:29.919694  2290 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:29.940062  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.941926  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:29.948421  1634 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:29.947510  1631 rpcz_store.cc:276] 0709 13:03:28.876227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.876404 (+   177us) service_pool.cc:224] Handling call
0709 13:03:29.946662 (+1070258us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000dded4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:29.946854 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509424512}
I20260709 13:03:29.954856  1707 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.955420  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:29.963245  6983 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:29.970369  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:30.057130  2292 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:29.971294  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:29.974428  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:29.978397  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:30.058368  2295 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:30.058734  1560 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:29.945078  7870 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:30.002977  7976 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.000s
W20260709 13:03:29.973567  7831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.148s	user 0.029s	sys 0.036s
I20260709 13:03:29.964066  8045 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.015261  7871 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:29.940641  7959 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:03:29.972731  7994 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:29.968390  7913 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:29.945468  7854 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:30.022547  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 444678us
I20260709 13:03:30.082336  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:29.979044  7942 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:29.964514  7958 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.009330  7860 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:30.022441  7813 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:29.944679  8075 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.952117  8005 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:29.994988  8078 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:03:30.053447  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.014s	sys 0.048s
W20260709 13:03:30.102864  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.014s	sys 0.048s
W20260709 13:03:30.098075  8043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.016s	sys 0.053s
I20260709 13:03:30.074733  2292 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
W20260709 13:03:30.065110  7831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.029s	sys 0.036s
I20260709 13:03:30.081168  7994 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:30.089612  7870 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:30.086135  6983 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:30.077638  8045 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.077135  7976 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.092185  7860 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.109305  6983 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:30.109889  7870 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:30.116206  7847 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.101089  7861 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.103686  7719 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:30.121657  8048 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.274s	user 0.007s	sys 0.000s
W20260709 13:03:30.107316  8043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.016s	sys 0.053s
I20260709 13:03:30.134888  7870 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:30.142861  6983 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.147567  6983 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:30.148524  8082 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:03:30.154460  7958 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.156838  7976 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.158258  7994 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.159943  7994 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:30.182610  7957 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.004s
I20260709 13:03:30.223857  8045 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.202324  7913 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.249598  8080 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 13:03:30.286118  8086 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:03:30.234964  7913 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.168813  7854 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.301579  8031 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:30.312940  7959 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:30.313623  7959 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:30.314674  7959 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:30.326617  8098 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.176s	user 0.001s	sys 0.002s
W20260709 13:03:30.333691  8089 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.181s	user 0.002s	sys 0.000s
W20260709 13:03:30.397471  8017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.007s	sys 0.018s
W20260709 13:03:30.398267  8017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.008s	sys 0.018s
W20260709 13:03:30.466360  1552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.008s
W20260709 13:03:30.467387  1552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.000s	sys 0.009s
W20260709 13:03:30.468133  1598 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.004s	sys 0.017s
W20260709 13:03:30.468680  1598 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.004s	sys 0.017s
W20260709 13:03:30.469336  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 582816us
I20260709 13:03:30.472689  1559 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:03:30.473556  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.008s	sys 0.006s
W20260709 13:03:30.062186  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 87861.3us
W20260709 13:03:30.474479  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.000s	sys 0.016s
W20260709 13:03:30.474947  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.000s	sys 0.016s
I20260709 13:03:30.476730  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3023) took 1245 ms. Trace:
W20260709 13:03:30.477806  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.467425  1557 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:30.478986  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.479130  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.049376  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.481608  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.084666  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:30.483394  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.474009  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.008s	sys 0.007s
I20260709 13:03:30.485270  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1057) took 1348 ms. Trace:
I20260709 13:03:30.485745  1626 rpcz_store.cc:276] 0709 13:03:29.137069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.523674 (+386605us) service_pool.cc:224] Handling call
0709 13:03:29.524037 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.550895 (+ 26858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.550924 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.551322 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.484649 (+933327us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:30.484762 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.484902 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.484959 (+    57us) raft_consensus.cc:1685] finished
0709 13:03:30.484975 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.485199 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169292,"raft.queue_time_us":27236,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":145605,"thread_start_us":889731,"threads_started":1,"wal-append.queue_time_us":661068}
I20260709 13:03:30.470781  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:30.488790  1553 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:30.491847  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2962) took 1031 ms. Trace:
I20260709 13:03:30.494660  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.494040  1701 rpcz_store.cc:276] 0709 13:03:29.460166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.538888 (+ 78722us) service_pool.cc:224] Handling call
0709 13:03:29.539179 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.539486 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.539512 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.540006 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.031715 (+491709us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:30.041350 (+  9635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.041454 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.491162 (+449708us) raft_consensus.cc:1685] finished
0709 13:03:30.491206 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.491546 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2469,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5984,"thread_start_us":447151,"threads_started":1,"wal-append.queue_time_us":560562}
I20260709 13:03:30.503561  1704 raft_consensus.cc:2410] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:30.507150  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:30.508263  1701 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:30.514209  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.017s	sys 0.040s
W20260709 13:03:30.514895  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.017s	sys 0.040s
I20260709 13:03:30.516733  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2961) took 1122 ms. Trace:
I20260709 13:03:30.517393  1708 rpcz_store.cc:276] 0709 13:03:29.394678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.532891 (+138213us) service_pool.cc:224] Handling call
0709 13:03:29.533175 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.542081 (+  8906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.542125 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.557278 (+ 15153us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.515807 (+958529us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:30.515966 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.516165 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.516267 (+   102us) raft_consensus.cc:1685] finished
0709 13:03:30.516285 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.516638 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8523,"raft.queue_time_us":9112,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":957735,"threads_started":1,"wal-append.queue_time_us":708374}
W20260709 13:03:30.519631  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.477253  1632 rpcz_store.cc:276] 0709 13:03:29.231112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.536958 (+305846us) service_pool.cc:224] Handling call
0709 13:03:29.537247 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.553946 (+ 16699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.553996 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.554623 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.475908 (+921285us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:30.476044 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.476224 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.476296 (+    72us) raft_consensus.cc:1685] finished
0709 13:03:30.476308 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.476560 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156564,"raft.queue_time_us":21814,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":134671,"thread_start_us":920374,"threads_started":1,"wal-append.queue_time_us":784151}
I20260709 13:03:30.526419  2289 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:30.524195  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:30.479624  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.554041  1708 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:30.558331  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2939) took 1432 ms. Trace:
W20260709 13:03:30.561425  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2399) took 3009 ms (client timeout 2205 ms). Trace:
I20260709 13:03:30.577467  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:30.577759  1704 rpcz_store.cc:269] 0709 13:03:27.551643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.685000 (+133357us) service_pool.cc:224] Handling call
0709 13:03:30.560954 (+2875954us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:30.561234 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6287233920}
I20260709 13:03:30.577033  1703 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:03:30.563413  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2372) took 1017 ms. Trace:
I20260709 13:03:30.592480  2296 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:03:30.592970  7967 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:30.591570  1703 pending_rounds.cc:85] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 5
I20260709 13:03:30.593823  1703 pending_rounds.cc:107] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:30.569444  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1054) took 1433 ms. Trace:
I20260709 13:03:30.570390  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2875) took 1368 ms. Trace:
I20260709 13:03:30.574134  1553 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:30.584882  7823 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.585235  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2371) took 1039 ms. Trace:
I20260709 13:03:30.586298  7860 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.586604  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1190) took 1242 ms. Trace:
I20260709 13:03:30.592085  1557 rpcz_store.cc:276] 0709 13:03:29.546087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.637084 (+ 90997us) service_pool.cc:224] Handling call
0709 13:03:29.637384 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.478483 (+841099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.563049 (+ 84566us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:30.563339 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907242880}
I20260709 13:03:30.594256  1703 raft_consensus.cc:2889] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:30.558849  1634 rpcz_store.cc:276] 0709 13:03:29.125362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.439169 (+313807us) service_pool.cc:224] Handling call
0709 13:03:29.948158 (+508989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.953905 (+  5747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.557796 (+603891us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54000dded0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:30.558096 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1169565440}
I20260709 13:03:30.588025  7814 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.582793  7842 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.594175  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1414) took 1465 ms. Trace:
I20260709 13:03:30.594909  1559 rpcz_store.cc:276] 0709 13:03:29.201821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.359019 (+157198us) service_pool.cc:224] Handling call
0709 13:03:29.359340 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.473646 (+1114306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.570005 (+ 96359us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002f9c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:30.570286 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2558467456}
I20260709 13:03:30.598492  2293 rpcz_store.cc:276] 0709 13:03:29.344160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.344345 (+   185us) service_pool.cc:224] Handling call
0709 13:03:29.344720 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.461875 (+117155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.585945 (+1124070us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54007bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:30.586307 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267313408}
I20260709 13:03:30.603829  7843 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:30.604369  7975 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:30.604652  7843 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.605149  7975 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.602391  2296 rpcz_store.cc:276] 0709 13:03:29.128971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.145426 (+ 16455us) service_pool.cc:224] Handling call
0709 13:03:29.145767 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.593023 (+1447256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.593834 (+   811us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54005a0fd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:30.594106 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3203972864}
W20260709 13:03:30.167876  7802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.024s	sys 0.047s
I20260709 13:03:30.593864  7967 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.614405  7861 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:30.596489  1551 rpcz_store.cc:276] 0709 13:03:29.545788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.598417 (+ 52629us) service_pool.cc:224] Handling call
0709 13:03:30.584674 (+986257us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b54000e8154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:30.585004 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16331520}
I20260709 13:03:30.619946  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 1036) took 1449 ms. Trace:
I20260709 13:03:30.620404  1558 rpcz_store.cc:276] 0709 13:03:29.170221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.204893 (+ 34672us) service_pool.cc:224] Handling call
0709 13:03:30.619495 (+1414602us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54000e9554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:30.619724 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510750592}
I20260709 13:03:30.621409  7719 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.594238  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:30.622123  7719 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.622542  1556 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 10.
I20260709 13:03:30.623533  7847 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.629194  7823 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:30.608247  7802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.024s	sys 0.048s
I20260709 13:03:30.638285  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:30.562861  7316 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:03:30.597783  7942 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.563267  7649 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.644047  7854 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.645984  7957 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:30.649119  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:30.650602  7861 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.635442  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:30.651273  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:30.570565  7913 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:30.616858  7975 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.607730  7843 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:30.594487  1635 rpcz_store.cc:276] 0709 13:03:29.136234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.471073 (+334839us) service_pool.cc:224] Handling call
0709 13:03:29.471496 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:29.550323 (+ 78827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.550358 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:29.557450 (+  7092us) log.cc:844] Serialized 52 byte log entry
0709 13:03:30.054819 (+497369us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:30.054929 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.568552 (+513623us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:30.568621 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.568697 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:30.568710 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.569225 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184632,"raft.queue_time_us":55136,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":142530,"spinlock_wait_cycles":56344320,"thread_start_us":454328,"threads_started":1,"wal-append.queue_time_us":618730}
I20260709 13:03:30.607473  7842 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.106642  7813 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.701956  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:30.704164  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55513.6us
I20260709 13:03:30.706408  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:30.707219  1557 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:30.709641  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.710631  1704 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:30.712093  7787 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:30.712620  7787 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:30.714064  7787 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:30.714780  1705 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.004s	sys 0.012s
W20260709 13:03:30.716590  1705 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.004s	sys 0.012s
I20260709 13:03:30.720295  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2963) took 1181 ms. Trace:
I20260709 13:03:30.720929  7843 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:30.720850  1705 rpcz_store.cc:276] 0709 13:03:29.539177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.564917 (+ 25740us) service_pool.cc:224] Handling call
0709 13:03:29.565326 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.565639 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:29.977106 (+411467us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.979160 (+  2054us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.719458 (+740298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.719615 (+   157us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:30.719735 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.719899 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.720194 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25204,"prepare.queue_time_us":758002,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":26072,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":22766,"thread_start_us":1124444,"threads_started":2}
I20260709 13:03:30.724926  7959 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.714038  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:30.725970  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89912.7us
I20260709 13:03:30.726126  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.726501  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:30.727160  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:30.727975  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.728654  7654 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:30.729641  1704 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.736649  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.738395  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2878) took 1050 ms. Trace:
W20260709 13:03:30.745137  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 107549us
I20260709 13:03:30.745886  1700 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:30.744810  7823 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.746155  7842 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:30.746364  7952 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:30.765021  7952 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.751968  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2433) took 1015 ms. Trace:
I20260709 13:03:30.747311  7787 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.752909  8078 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:30.753453  7813 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.753479  7654 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.751603  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.739787  7942 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.777148  8076 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:30.777915  8076 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:30.778738  7654 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.778954  7942 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:30.759819  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 124507us
I20260709 13:03:30.768292  1635 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:30.769016  8075 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:30.753525  2294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.010s	sys 0.005s
I20260709 13:03:30.766503  1709 rpcz_store.cc:276] 0709 13:03:29.676628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.706214 (+ 29586us) service_pool.cc:224] Handling call
0709 13:03:30.679840 (+973626us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400165b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:30.691909 (+ 12069us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300963584}
I20260709 13:03:30.775189  1708 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:30.775956  1701 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
W20260709 13:03:30.774537  7722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.008s	sys 0.007s
I20260709 13:03:30.694654  7854 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.743357  7967 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:30.781072  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:30.774560  8092 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.785717  7958 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.769896  7957 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:30.779860  8061 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.788170  7854 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.751098  1552 rpcz_store.cc:276] 0709 13:03:29.688278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.768864 (+ 80586us) service_pool.cc:224] Handling call
0709 13:03:29.769300 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.769537 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:30.468228 (+698691us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.680985 (+212757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.687405 (+  6420us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:30.696723 (+  9318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.730732 (+ 34009us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.738325 (+  7593us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1607,"prepare.queue_time_us":524043,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":6335,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":13408,"thread_start_us":809542,"threads_started":2}
I20260709 13:03:30.795857  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:30.797264  2294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.011s	sys 0.006s
I20260709 13:03:30.809639  1626 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:30.810652  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:30.815567  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71307us
I20260709 13:03:30.779101  8076 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.791188  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.782589  8078 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:30.807200  7722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.009s	sys 0.007s
I20260709 13:03:30.820767  7861 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:30.820993  8072 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:03:30.804467  8003 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.015s	sys 0.004s
W20260709 13:03:30.818706  1702 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.005s	sys 0.009s
I20260709 13:03:30.834436  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.819717  7958 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:30.840550  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:30.846231  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:30.849573  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:30.850613  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:30.828893  7967 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:30.841816  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:30.855960  1702 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.006s	sys 0.009s
W20260709 13:03:30.852466  1706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.003s	sys 0.023s
W20260709 13:03:30.869359  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52408.8us
W20260709 13:03:30.861780  1707 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.777s	user 0.016s	sys 0.006s
W20260709 13:03:30.881481  1706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.004s	sys 0.024s
W20260709 13:03:30.884058  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72851.7us
I20260709 13:03:30.855544  7967 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.891892  7957 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:30.892433  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 145664us
I20260709 13:03:30.893529  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:30.861634  7854 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:30.894556  7957 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:30.886389  1705 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:30.892352  7967 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:30.862082  8003 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.016s	sys 0.005s
I20260709 13:03:30.901671  1707 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
W20260709 13:03:30.895882  6983 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.695s	user 0.006s	sys 0.009s
I20260709 13:03:30.897364  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1187) took 1596 ms. Trace:
I20260709 13:03:30.880084  8061 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:30.928632  8061 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.929385  7942 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:30.929939  7942 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.930361  7942 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:30.896623  1560 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.837s	user 0.016s	sys 0.008s
I20260709 13:03:30.884028  7861 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:03:30.932157  1560 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:30.913168  1627 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:30.933777  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2439) took 1053 ms. Trace:
I20260709 13:03:30.934266  1707 rpcz_store.cc:276] 0709 13:03:29.880652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.909141 (+ 28489us) service_pool.cc:224] Handling call
0709 13:03:30.933684 (+1024543us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.935089  7722 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:30.910281  7813 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:30.936131  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2435) took 1259 ms. Trace:
I20260709 13:03:30.895726  8078 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:30.936542  1707 rpcz_store.cc:276] 0709 13:03:29.677167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.935104 (+1257937us) service_pool.cc:224] Handling call
0709 13:03:30.935533 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.935757 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.935800 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.935870 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:30.935908 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.935961 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.936059 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.924047  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:30.940804  7870 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.795s	user 0.017s	sys 0.002s
I20260709 13:03:30.914603  7977 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:03:30.915251  7823 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:30.916103  8075 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:30.918463  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 144855us
I20260709 13:03:30.906966  1632 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:30.919943  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:30.921159  2293 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:30.921916  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:30.922851  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:30.922926  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:30.924137  8107 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:30.924619  8070 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 13:03:30.944164  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.945138  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.945720  7958 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:30.947257  7994 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.787s	user 0.012s	sys 0.031s
I20260709 13:03:30.913647  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:30.954142  1704 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:30.958578  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:30.931600  8061 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:30.977907  1635 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:30.935529  7722 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:30.982340  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2431) took 1306 ms. Trace:
I20260709 13:03:30.983139  7854 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.973409  1626 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:30.907423  2294 rpcz_store.cc:276] 0709 13:03:29.301052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.302617 (+  1565us) service_pool.cc:224] Handling call
0709 13:03:29.338686 (+ 36069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.338904 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:29.593177 (+254273us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.874050 (+1280873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.885685 (+ 11635us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.897029 (+ 11344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.897134 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.897311 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":551902,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":6171,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":32121,"thread_start_us":1443512,"threads_started":2,"wal-append.queue_time_us":1247996}
W20260709 13:03:30.992905  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.987388  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:30.976855  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:30.995434  1705 rpcz_store.cc:276] 0709 13:03:29.675978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.911555 (+1235577us) service_pool.cc:224] Handling call
0709 13:03:30.981638 (+ 70083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.981942 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.981967 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.982007 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.982064 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.982123 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.982246 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.004004  1628 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:31.004683  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47689.2us
I20260709 13:03:31.012979  2296 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:03:31.013371  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90738.2us
I20260709 13:03:30.979398  8107 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:30.897646  8092 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:31.017392  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 111397us
I20260709 13:03:31.023910  2290 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.038364  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 2957) took 1808 ms. Trace:
I20260709 13:03:31.038822  1706 rpcz_store.cc:276] 0709 13:03:29.229687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.520171 (+290484us) service_pool.cc:224] Handling call
0709 13:03:29.520522 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.520737 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.520762 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.521239 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.103351 (+582112us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.979198 (+875847us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.037950 (+ 58752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.038054 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.038136 (+    82us) raft_consensus.cc:1685] finished
0709 13:03:31.038146 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.038306 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4748,"prepare.queue_time_us":48043,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":1358926,"threads_started":2,"wal-append.queue_time_us":710376}
I20260709 13:03:31.040117  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.041044  1631 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:31.042042  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.042559  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:31.044176  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 891979us
I20260709 13:03:31.031699  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.043625  1629 raft_consensus.cc:2410] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:31.056610  2288 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:31.056967  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:03:31.058029  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164643us
I20260709 13:03:31.058831  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.059544  7854 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.060674  1630 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:31.061998  7854 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.062633  7854 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:31.052497  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.064191  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2428) took 1949 ms. Trace:
I20260709 13:03:31.064549  1702 rpcz_store.cc:276] 0709 13:03:29.114778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.499509 (+384731us) service_pool.cc:224] Handling call
0709 13:03:29.519880 (+ 20371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:29.536703 (+ 16823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.536734 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:29.537274 (+   540us) log.cc:844] Serialized 52 byte log entry
0709 13:03:30.024462 (+487188us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:30.325858 (+301396us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:30.873509 (+547651us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.005258 (+131749us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.034547 (+ 29289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.048009 (+ 13462us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.063971 (+ 15962us) raft_consensus.cc:1685] finished
0709 13:03:31.063986 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.064118 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6614,"prepare.queue_time_us":147245,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":3669,"raft.queue_time_us":23896,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":21133,"spinlock_wait_cycles":39722240,"thread_start_us":1294821,"threads_started":2,"wal-append.queue_time_us":612930}
I20260709 13:03:31.067911  8005 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:31.068614  8005 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:31.069883  8005 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:31.071919  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.074199  8092 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.075001  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:31.075994  8098 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:31.077104  7975 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:31.080389  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 173346us
I20260709 13:03:31.021248  8107 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:31.082541  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.978119  7870 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.032182  7842 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:31.067615  1705 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:31.067586  1702 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:03:31.095458  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.062961  7854 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.114284  7854 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:31.116025  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2883) took 1426 ms. Trace:
I20260709 13:03:31.096894  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:31.117354  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84773.3us
I20260709 13:03:31.036289  7994 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:31.118201  7994 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:31.118371  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:31.118705  7913 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:31.119287  7913 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:31.094743  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1101) took 1008 ms. Trace:
I20260709 13:03:31.130649  1705 rpcz_store.cc:276] 0709 13:03:30.086076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.023979 (+937903us) service_pool.cc:224] Handling call
0709 13:03:31.067141 (+ 43162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.094459 (+ 27318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.094675 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42240}
I20260709 13:03:31.098757  7968 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:31.103875  8061 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:31.145303  8061 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:31.146384  8061 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.148080  7813 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.148770  7813 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.149215  7719 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.150990  7813 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.151417  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1041) took 1410 ms. Trace:
I20260709 13:03:31.098815  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.104923  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:31.105496  8006 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.107195  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2425) took 2085 ms. Trace:
I20260709 13:03:31.112706  8007 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:31.113263  7842 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.116477  1555 rpcz_store.cc:276] 0709 13:03:29.689599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.497248 (+807649us) service_pool.cc:224] Handling call
0709 13:03:30.498035 (+   787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.498920 (+   885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.498963 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.537056 (+ 38093us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.678269 (+141213us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.678444 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.678557 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.115603 (+437046us) raft_consensus.cc:1685] finished
0709 13:03:31.115625 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.115835 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105111,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":237,"thread_start_us":109786,"threads_started":1,"wal-append.queue_time_us":529273}
I20260709 13:03:31.119340  7994 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.091984  1707 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:03:31.096294  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.179216  1627 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:31.120656  8031 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:31.121424  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:31.123209  8117 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.152s	user 0.002s	sys 0.002s
I20260709 13:03:31.124231  7814 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.124846  8116 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 13:03:31.126405  7722 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.129869  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1097) took 1045 ms. Trace:
I20260709 13:03:31.131562  2290 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:31.132268  1628 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:31.141678  7942 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.143185  7975 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:31.143616  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.202035  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.159622  1703 rpcz_store.cc:276] 0709 13:03:29.022269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.108274 (+ 86005us) service_pool.cc:224] Handling call
0709 13:03:30.563648 (+1455374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.675308 (+111660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.106954 (+431646us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001b6a50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:31.107133 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345919360}
I20260709 13:03:31.151861  1554 rpcz_store.cc:276] 0709 13:03:29.713821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.528460 (+814639us) service_pool.cc:224] Handling call
0709 13:03:30.529138 (+   678us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:30.529789 (+   651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.529820 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:30.530597 (+   777us) log.cc:844] Serialized 52 byte log entry
0709 13:03:30.688448 (+157851us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.688513 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.688604 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.123511 (+434907us) raft_consensus.cc:1685] finished
0709 13:03:31.123532 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.123703 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84759,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":157299,"threads_started":1,"wal-append.queue_time_us":589756}
I20260709 13:03:31.188503  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2437) took 1455 ms. Trace:
I20260709 13:03:31.202975  1707 rpcz_store.cc:276] 0709 13:03:29.723663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.971687 (+1248024us) service_pool.cc:224] Handling call
0709 13:03:31.019527 (+ 47840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.179167 (+159640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.179422 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.196411  1700 rpcz_store.cc:276] 0709 13:03:30.084867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.493995 (+409128us) service_pool.cc:224] Handling call
0709 13:03:31.129614 (+635619us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400133dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:31.129769 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547452288}
W20260709 13:03:31.189332  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 129950us
I20260709 13:03:31.182116  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:31.153009  7843 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:31.139009  7847 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:31.144304  7968 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.181275  1702 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
W20260709 13:03:31.206183  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 144461us
I20260709 13:03:31.152510  7854 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.158783  7823 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:31.207543  1632 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:31.218732  1706 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:31.199959  6983 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:03:31.213752  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:31.221019  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.222227  1635 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:31.224941  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.225589  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:31.223943  7861 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:31.248193  7861 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:31.248548  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 188964us
I20260709 13:03:31.249187  7861 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.250316  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:31.250752  7861 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:31.251909  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 162860us
I20260709 13:03:31.253091  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:31.254947  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.227058  7952 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.255722  7843 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:31.198699  7814 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.257073  7777 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:31.258971  7814 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.208736  8006 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.262385  8006 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.264057  8001 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.264703  8001 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.267443  8001 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.224170  7870 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.232203  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2434) took 1552 ms. Trace:
I20260709 13:03:31.233117  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1100) took 1147 ms. Trace:
I20260709 13:03:31.228914  7958 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.270816  7958 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.208002  7842 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:31.271900  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 118299us
I20260709 13:03:31.272832  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.273819  7842 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.273901  7958 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.228477  1626 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.274868  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.276178  7942 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.276921  7854 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.279721  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:31.280526  1627 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.248184  8111 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.281908  8111 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.282662  1700 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:31.283252  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.284085  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:31.284220  7975 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:31.284157  1630 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:31.285130  7975 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:31.285440  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.235459  7722 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.286527  1634 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:31.236982  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:31.287609  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50749.3us
I20260709 13:03:31.288095  1632 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:31.237457  7847 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.290123  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.238510  1673 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:31.291270  1673 leader_election.cc:403] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:31.291973  7777 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:31.292584  7854 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:31.292738  7772 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17.
I20260709 13:03:31.293203  7854 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:31.294453  7777 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:31.294323  7854 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.268286  1708 rpcz_store.cc:276] 0709 13:03:29.676884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.911641 (+1234757us) service_pool.cc:224] Handling call
0709 13:03:31.129816 (+218175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.143061 (+ 13245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.143099 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.143153 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:31.143187 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.143253 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.229306 (+ 86053us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400133dd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:31.229436 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325176064}
I20260709 13:03:31.295471  7777 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.297796  7967 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.299002  1700 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:03:31.299629  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47919.7us
I20260709 13:03:31.241174  7968 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.241622  8075 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:31.244060  7719 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:31.237509  2261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.189s	user 0.011s	sys 0.018s
I20260709 13:03:31.245049  7823 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.208345  7787 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:31.245568  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.247668  7813 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.143215  7913 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.268505  1704 rpcz_store.cc:276] 0709 13:03:30.085782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.014117 (+928335us) service_pool.cc:224] Handling call
0709 13:03:31.079923 (+ 65806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.080343 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:03:31.080717 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.232710 (+151993us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.232774 (+    64us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:31.232856 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.232944 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.233061 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38087,"prepare.queue_time_us":204,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":46570,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":107842,"threads_started":1,"wal-append.queue_time_us":165204}
I20260709 13:03:31.235234  8078 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.249191  1707 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:31.309218  1709 raft_consensus.cc:2368] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11.
W20260709 13:03:31.311174  2261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.263s	user 0.012s	sys 0.018s
I20260709 13:03:31.312418  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:31.312739  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57821.3us
I20260709 13:03:31.317005  1635 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:31.290529  7847 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.323520  1627 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:31.309672  8075 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.325325  1705 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:31.312973  7870 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:31.325793  1703 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:31.336437  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.336517  1626 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:31.348692  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
W20260709 13:03:31.348943  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 135875us
I20260709 13:03:31.344563  1557 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:31.351398  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.332841  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.348202  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:31.367702  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.369529  1553 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:31.370889  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.365110  1559 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:31.373116  1560 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:31.330276  8075 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:31.380126  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.382284  1555 raft_consensus.cc:2410] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:31.354599  7913 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:31.350407  8111 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.324741  8092 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:03:31.328588  7813 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:31.385576  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 165217us
W20260709 13:03:31.386165  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69718.4us
I20260709 13:03:31.386075  8118 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:03:31.386847  8133 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.332630  8006 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:31.355311  7787 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:31.390843  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:31.380432  7847 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.403157  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:31.393710  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48340.5us
I20260709 13:03:31.405128  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:31.355993  7870 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:31.396127  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 166112us
I20260709 13:03:31.406200  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.377945  8001 raft_consensus.cc:2764] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:31.333284  8078 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.388506  7967 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.401327  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:31.414299  1556 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.398432  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.404043  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:31.416738  7870 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.412492  8111 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:31.419224  8111 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.419584  8111 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.420161  7957 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.407235  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.420995  1553 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:31.422222  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.428252  1557 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:31.429594  1559 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:31.430528  7865 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.431061  7865 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:31.431481  7865 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.404641  7847 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:31.414214  7958 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.436194  7847 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.436530  7958 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.436996  7958 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.437959  7968 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:31.407722  8133 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.453682  1601 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:31.448865  7860 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:31.473094  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74761.7us
I20260709 13:03:31.478448  7942 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:31.478325  7637 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:03:31.478739  8133 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.479292  7637 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:31.479396  8133 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.479897  8133 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.480245  8133 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.481287  7831 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.403249  8107 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.481848  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:31.481808  7831 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.483852  8107 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.485383  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.486334  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:31.487977  7977 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:31.488488  7977 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:31.489390  7977 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.490095  8075 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.491532  1552 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:31.492202  1708 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:31.475710  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71739.2us
I20260709 13:03:31.492820  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.493620  1626 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:31.497588  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 75439.1us
I20260709 13:03:31.495323  1556 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11.
I20260709 13:03:31.471205  7814 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:31.503459  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:31.479715  7637 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:31.491869  7975 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:31.495684  8118 raft_consensus.cc:2764] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 13:03:31.520362  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.502511  7814 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.474937  7772 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:03:31.492756  7967 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:03:31.507879  8107 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:31.546084  7814 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:31.545773  7959 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.586998  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 170913us
I20260709 13:03:31.573490  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.574404  8045 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.549935  7975 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:31.600054  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 98430.7us
I20260709 13:03:31.601078  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:31.600925  7975 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:31.602970  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.603613  1704 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:31.600054  8045 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.606901  8045 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.548817  8007 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:31.607548  8007 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:31.607913  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.608001  8007 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.608870  7959 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.610128  7813 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.610560  7831 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.610956  7959 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:31.611114  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.581506  7637 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.612704  7637 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:31.613250  7823 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.615003  7719 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:31.615404  7719 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:31.618371  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2441) took 1069 ms. Trace:
I20260709 13:03:31.618795  1709 rpcz_store.cc:276] 0709 13:03:30.560007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.824252 (+264245us) service_pool.cc:224] Handling call
0709 13:03:31.618165 (+793913us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:31.618307 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022496640}
I20260709 13:03:31.619889  8076 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.619957  7860 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.616336  7719 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.620709  8076 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.620749  7860 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.622025  7719 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.623100  7860 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:31.624259  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 115159us
W20260709 13:03:31.629819  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.623817  7860 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.540530  8111 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:31.637888  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 64528us
W20260709 13:03:31.656966  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.623080  8076 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.588744  8075 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.664855  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.671859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.637607  7860 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.676390  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:31.677232  1704 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:31.672855  1709 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:31.676827  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:31.682471  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57494.4us
I20260709 13:03:31.664013  7942 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.689154  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 88184.4us
I20260709 13:03:31.653918  8045 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:03:31.690723  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.691417  1707 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:31.678550  7722 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.694633  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.693356  1703 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:03:31.696079  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.684187  8118 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:31.698737  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.687438  7823 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:31.699735  8001 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:03:31.690972  7959 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:31.708362  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.679312  7847 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.724579  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.725869  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.726363  7847 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.718413  8061 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:31.727377  8061 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:31.728411  8061 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.730018  8061 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.733170  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:31.736408  8118 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:31.737228  8118 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:31.739646  7654 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:31.692577  7942 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.735878  8133 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.710984  7842 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:31.711923  8149 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:31.697289  7722 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.674299  7968 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:03:31.704988  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.758037  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:31.697034  7831 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:31.671466  8007 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.721462  7860 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.768842  7722 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.721387  8144 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.769833  8144 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.722819  8001 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.778031  8045 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:31.778654  8045 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:31.778362  8001 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.779718  8045 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.781014  8001 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.781203  8045 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.783274  8144 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.724293  1626 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:31.761005  7823 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.751329  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:31.784961  8132 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.385s	user 0.001s	sys 0.000s
I20260709 13:03:31.785118  1632 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:31.786281  7823 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.786430  8007 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.787060  7957 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:31.761148  7959 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.755079  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:31.789781  7959 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.787581  7957 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.759822  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:31.758504  2296 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:31.764115  8149 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.765175  7831 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:31.768981  8135 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.284s	user 0.003s	sys 0.000s
I20260709 13:03:31.774259  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:31.722138  8111 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.801306  1707 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:03:31.799003  2293 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:31.805759  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50754us
I20260709 13:03:31.808284  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.804929  7831 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.815876  8092 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:31.816076  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:31.816432  8092 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.816797  8092 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.817692  1703 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:03:31.820343  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84551.1us
W20260709 13:03:31.821267  8139 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:03:31.825802  8092 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:31.828630  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68964.3us
I20260709 13:03:31.805217  8001 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:31.832386  8149 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.833087  7814 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.837529  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:31.838001  1598 leader_election.cc:400] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 10 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11.
I20260709 13:03:31.838075  8118 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:31.838809  7823 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:31.840224  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1104) took 1358 ms. Trace:
I20260709 13:03:31.895365  1706 rpcz_store.cc:276] 0709 13:03:30.481744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.040512 (+558768us) service_pool.cc:224] Handling call
0709 13:03:31.040879 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.224450 (+183571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.839964 (+615514us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54003e1f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:31.840157 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404509440}
W20260709 13:03:31.842700  8137 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 13:03:31.844439  6983 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:31.844095  7654 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:31.898900  7654 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:31.899318  7654 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:31.899809  2288 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:03:31.899999  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62193.2us
I20260709 13:03:31.900338  7654 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.901656  7654 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.904137  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1098) took 1819 ms. Trace:
I20260709 13:03:31.904541  1701 rpcz_store.cc:276] 0709 13:03:30.085170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.005705 (+920535us) service_pool.cc:224] Handling call
0709 13:03:31.072543 (+ 66838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.213656 (+141113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.213686 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.228020 (+ 14334us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.414582 (+186562us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:31.414618 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.414697 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.841877 (+427180us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:31.903833 (+ 61956us) raft_consensus.cc:1685] finished
0709 13:03:31.903861 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.904078 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85692,"raft.queue_time_us":104345,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4062,"spinlock_wait_cycles":47187840,"thread_start_us":126717,"threads_started":1,"wal-append.queue_time_us":187843}
I20260709 13:03:31.910513  7831 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.845140  7942 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:31.847149  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.913455  1552 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:31.848928  1554 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:31.849021  8148 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.174s	user 0.000s	sys 0.002s
I20260709 13:03:31.849634  1556 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:31.851097  8078 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.856613  1555 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:31.910944  7865 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.858446  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.922093  1551 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:31.859014  7959 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.923173  7959 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.854153  7842 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:31.859591  8151 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:31.860354  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.939980  1704 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:31.940203  1707 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.882148  2296 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:31.896776  6983 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:31.858268  8144 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:31.935868  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.936427  1709 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:31.939116  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.880916  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:31.935369  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:31.951220  1702 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:31.951251  1558 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.914300  7823 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:31.959455  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.961095  2289 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:31.956728  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:31.964819  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:31.960186  7958 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.523s	user 0.004s	sys 0.009s
I20260709 13:03:31.875440  7957 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.970767  2297 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:31.972311  7870 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.972793  8001 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.972858  7870 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.975235  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:31.967978  8140 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:03:31.962950  7823 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.973515  8001 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.980384  2289 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:31.973915  7958 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:31.993010  1558 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:31.993563  7958 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:31.996712  7814 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.963688  8151 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:31.999524  8151 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:31.984691  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:31.980325  6983 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:32.003552  2288 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:31.976552  8156 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.028855  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:31.954844  8144 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.077065  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.078855  6983 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.078045  2294 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:32.086948  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.090063  1707 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:32.094485  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:32.095485  1558 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:32.101539  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.101817  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.102516  2289 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:32.104460  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.106336  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:32.107156  2291 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:32.111029  8092 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.085253  8156 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.119999  8133 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:32.122277  7865 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:32.124318  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.124953  7842 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.129124  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:32.129993  1558 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.130867  8144 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:32.132687  1551 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:03:32.134162  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:32.134776  1551 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.136234  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:32.137566  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:32.138187  1555 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:32.138676  1558 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:32.139053  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:32.140796  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:32.142009  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.142697  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:32.142726  7994 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:32.143167  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.143294  7994 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.142377  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.144574  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.149215  1632 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:03:32.142792  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.145274  1630 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:32.145752  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:32.146886  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.164124  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.165099  1631 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:32.165642  8156 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:32.166299  8156 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.166379  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.166996  8156 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.167595  2289 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:32.168205  1635 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:32.168787  7994 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.169291  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.172518  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:32.174522  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.178182  7870 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.180001  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.180914  1628 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:32.181758  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:32.184645  1558 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:32.145274  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:32.190505  8156 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.211872  1626 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:32.213836  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:32.214952  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:32.215348  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:32.192194  2295 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:32.195592  7977 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:32.232900  1558 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:32.233246  7977 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:32.196328  7722 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:32.236112  7722 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:32.239872  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.197679  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:32.241255  8031 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:32.241828  8031 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.242215  8031 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.244163  7977 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.243655  7722 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.204896  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:32.252344  1555 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:32.192167  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:32.256371  2294 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:32.218029  2288 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:32.147199  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.219173  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:32.220927  7787 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:32.260416  2289 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:32.260818  7787 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:32.261216  7787 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:32.262019  7787 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.221251  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:32.264890  7958 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:32.265108  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:32.266112  8150 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:32.267665  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:32.197108  7719 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.241153  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43576.1us
I20260709 13:03:32.268720  7719 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.269832  7722 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:32.270681  7722 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:32.270721  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:32.271100  7722 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:32.271395  7719 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.272079  8107 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:32.271997  7722 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:32.201857  7994 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.273202  7722 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.273913  8107 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:32.274857  8150 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:32.275206  8150 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:32.275288  7994 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.221696  8163 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:32.275951  7977 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.276535  7977 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:32.276126  8150 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.276978  7977 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.278872  7719 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:32.263484  7787 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:32.298635  1551 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:32.304821  7823 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.309801  1558 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:32.218272  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:32.403601  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 185370us
I20260709 13:03:32.368180  2291 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:32.417943  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:32.428812  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:32.429226  1558 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:32.430694  1558 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:32.432009  1558 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:03:32.432322  1558 pending_rounds.cc:85] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:03:32.432557  1558 pending_rounds.cc:107] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:03:32.432775  1558 raft_consensus.cc:2889] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:32.434129  1557 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:03:32.534716  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.541914  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200023us
I20260709 13:03:32.542824  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:32.625149  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 207282us
W20260709 13:03:32.626081  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.627156  1551 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:03:32.427453  8061 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:32.635821  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 93110.7us
I20260709 13:03:32.629596  8163 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:03:32.671495  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.640568  8163 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:32.677079  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.027s	sys 0.056s
W20260709 13:03:32.678817  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.027s	sys 0.056s
I20260709 13:03:32.673223  7814 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:32.627846  7802 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:03:32.599274  7977 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.637396  8061 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:32.700476  2292 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.027s	sys 0.051s
I20260709 13:03:32.639299  8031 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:32.709300  2292 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.027s	sys 0.051s
I20260709 13:03:32.694617  7654 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:32.695035  7814 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:32.703644  7968 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.716995  7814 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:32.710793  8061 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:32.750073  7968 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.741736  7654 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:32.833747  7654 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.861496  7968 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:32.863304  8031 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.842921  7823 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.832007  8163 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:32.892683  7722 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.003s
W20260709 13:03:32.922127  2294 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.004s
I20260709 13:03:32.922832  2294 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
W20260709 13:03:32.936836  2260 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.003s	sys 0.007s
W20260709 13:03:32.937568  2260 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.003s	sys 0.007s
W20260709 13:03:32.938072  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 611365us
I20260709 13:03:32.969035  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:32.970463  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2444) took 1891 ms. Trace:
I20260709 13:03:32.970868  1708 rpcz_store.cc:276] 0709 13:03:31.079004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.296432 (+217428us) service_pool.cc:224] Handling call
0709 13:03:31.296746 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.492597 (+195851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.970194 (+1477597us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400167954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:32.970399 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449040128}
I20260709 13:03:32.969945  8031 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.979463  7870 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:32.976081  7654 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.982728  7870 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.983476  7654 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.985628  7870 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:32.972824  8031 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:32.985752  7654 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:32.988576  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.048s	sys 0.074s
W20260709 13:03:32.989244  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.048s	sys 0.074s
W20260709 13:03:32.989770  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 716661us
I20260709 13:03:32.990976  1525 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:32.999994  1525 leader_election.cc:403] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:03:32.998845  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.023116  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.028223  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.009227  7977 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.015935  7654 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:33.017092  7722 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:33.047256  7654 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:33.051802  7977 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.078037  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.077936  7654 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.078886  1708 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:33.080220  2297 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:33.079989  7977 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.081185  1708 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:03:33.081543  1708 pending_rounds.cc:85] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:33.081775  1708 pending_rounds.cc:107] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:03:33.082003  1708 raft_consensus.cc:2889] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.082240  7977 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.085595  1627 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:33.086762  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.088183  8171 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:33.085426  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.091246  1708 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:33.091392  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:33.094000  2260 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.549s (SENT)
I20260709 13:03:33.094537  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.095688  8005 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:33.097739  8141 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.377s	user 0.033s	sys 0.047s
I20260709 13:03:33.100486  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1200) took 1224 ms. Trace:
I20260709 13:03:33.100916  2290 rpcz_store.cc:276] 0709 13:03:31.876253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.876381 (+   128us) service_pool.cc:224] Handling call
0709 13:03:31.898019 (+ 21638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.000094 (+102075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.000124 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.000167 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:32.000199 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.100216 (+1100017us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400720954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:33.100281 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.100419 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234199296}
I20260709 13:03:33.102923  8005 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:33.069060  7787 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.000s
W20260709 13:03:33.121039  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.123260  8007 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:33.124419  8007 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:33.125470  8007 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.126181  7831 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.126657  8007 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:33.145200  1627 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:33.064779  7814 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.148092  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.146104  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:33.151489  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.152894  8031 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.144640  8005 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:03:33.155249  7814 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.157182  8144 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:33.161043  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.161559  1702 raft_consensus.cc:2410] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:03:33.161873  8150 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.162274  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.203259  1626 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:33.226323  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 2981) took 1304 ms. Trace:
I20260709 13:03:33.226806  1702 rpcz_store.cc:276] 0709 13:03:31.922019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.128796 (+1206777us) service_pool.cc:224] Handling call
0709 13:03:33.226239 (+ 97443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.233743  6983 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:33.236052  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.228431  7787 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.234354  6983 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.240736  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:33.241153  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134671us
I20260709 13:03:33.233525  7719 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.241946  1555 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:33.249521  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.233461  8031 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:33.255395  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 220441us
I20260709 13:03:33.258251  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.259019  1626 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:33.261529  6983 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:33.243353  1523 connection.cc:256] RPC call timeout handler was delayed by -1.2350401499999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 13:03:33.272578  1523 connection.cc:256] RPC call timeout handler was delayed by -1.164840149999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 13:03:33.276158  1523 connection.cc:256] RPC call timeout handler was delayed by -1.1531401500000129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 13:03:33.276750  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.277742  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.280665  6983 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.285684  6983 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.275184  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.286826  1626 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:03:33.288868  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.267005  7831 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:33.295653  7831 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.298066  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.299258  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.280640  7847 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:33.254056  7787 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.290485  7823 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.315574  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.316850  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.259717  7719 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.319381  7787 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.320115  7719 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.320799  1557 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:33.321851  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.322794  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.323467  2295 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
W20260709 13:03:33.325580  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.323889  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.324158  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.327073  1553 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:33.327280  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.327075  1559 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:33.302403  8107 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.330143  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.273325  8061 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.334897  6983 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:33.336601  2260 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.748s (SENT)
I20260709 13:03:33.335697  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.338361  1554 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:33.334831  1552 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:33.341109  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:33.290251  8031 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.343513  1555 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:03:33.311443  7722 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.346151  1559 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:33.312125  7847 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.353307  8179 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:33.353588  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.353928  8179 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.354724  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.355260  7847 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.355333  1704 raft_consensus.cc:2376] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16. Re-sending same reply.
I20260709 13:03:33.356256  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.357052  1707 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 12.
I20260709 13:03:33.357745  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.358022  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.358367  1704 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:33.358700  1709 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:33.359210  8031 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.360451  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.360634  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.314450  7823 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.382004  1704 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:33.361631  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.362619  1702 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:33.351939  7722 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.352499  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:33.364701  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.365756  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.365926  8187 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:33.373606  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:33.346689  8173 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:03:33.401274  8187 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.402989  7847 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:33.403674  7847 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.385528  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.404153  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:33.405012  1559 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.405561  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.406489  1704 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:33.385772  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.345083  7977 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:33.408569  1709 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:33.408452  7977 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.409736  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.410746  7977 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.386776  1708 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:33.412468  7831 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.413080  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.414263  1708 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:33.414546  8061 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.387308  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.387971  1558 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:33.388197  7722 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.388814  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:33.389611  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.417016  1701 raft_consensus.cc:2410] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:03:33.417856  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.418005  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.418458  1701 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:33.419332  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.419693  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:33.392016  7823 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:33.393492  1630 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:03:33.420514  2290 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:33.385528  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.394423  1552 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:33.392738  8179 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.397387  8176 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:33.395114  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.424765  8176 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:33.425227  8176 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:33.426179  8176 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.400499  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.419879  8061 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.383905  1634 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:33.399576  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.446530  1555 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:33.411250  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.407684  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.448429  1709 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
W20260709 13:03:33.434587  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.427614  8176 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.444245  1626 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:33.448195  1704 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:33.341678  8107 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.453734  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.454949  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.415323  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.404198  7847 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:33.454421  1634 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:33.456962  1708 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:33.458441  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.459923  1551 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:03:33.461009  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.461905  2290 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:33.465827  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:33.454835  8186 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.003s
I20260709 13:03:33.467664  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.466017  8031 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:33.473727  8001 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.475190  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:33.451663  8179 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.476058  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.480185  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.481635  1703 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:33.455865  7722 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.455842  7787 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.485630  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.493361  8031 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.444276  7994 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:33.465500  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.488351  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:33.490958  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.475270  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.606791  1705 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.492533  1707 raft_consensus.cc:2376] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17. Re-sending same reply.
I20260709 13:03:33.493121  1559 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:33.493304  7719 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.493517  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:33.494628  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.461328  8188 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.601969  8001 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:03:33.611523  8188 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.612442  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1208) took 1584 ms. Trace:
I20260709 13:03:33.612732  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.613807  8188 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.497419  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.614022  1628 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
W20260709 13:03:33.614799  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 117477us
I20260709 13:03:33.612895  2292 rpcz_store.cc:276] 0709 13:03:32.028054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.094570 (+ 66516us) service_pool.cc:224] Handling call
0709 13:03:32.095061 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.095719 (+   658us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:32.711800 (+616081us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.250543 (+538743us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.254426 (+  3883us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.264672 (+ 10246us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.264834 (+   162us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:33.265047 (+   213us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.265875 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.265966 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.469138 (+203172us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b1800127420. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:33.612176 (+143038us) raft_consensus.cc:1685] finished
0709 13:03:33.612202 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.612365 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7698,"prepare.queue_time_us":986961,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":17391,"raft.queue_time_us":20372,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":20741,"spinlock_wait_cycles":22502528,"thread_start_us":1035441,"threads_started":3,"wal-append.queue_time_us":394481}
I20260709 13:03:33.497865  8107 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.615794  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.500478  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1442) took 1783 ms. Trace:
I20260709 13:03:33.502629  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.616823  1628 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:33.617271  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.617837  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.618445  1700 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:33.616657  2293 rpcz_store.cc:276] 0709 13:03:31.717373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.717530 (+   157us) service_pool.cc:224] Handling call
0709 13:03:31.718688 (+  1158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.802701 (+ 84013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.500230 (+1697529us) spinlock_profiling.cc:243] Waited 80 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:33.500413 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184107776}
I20260709 13:03:33.483592  7957 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.488857  8187 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:33.452127  8061 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.622818  8061 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.588775  1708 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:33.623934  8187 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:33.624583  1524 connection.cc:256] RPC call timeout handler was delayed by -1.4874588909999602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 13:03:33.624925  8187 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:33.625559  8187 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:33.604439  7722 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.626561  8187 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.486850  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:33.628155  1631 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.605975  7787 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.628314  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.629776  1708 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:33.630368  7787 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:33.608760  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 143329us
W20260709 13:03:33.596468  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106914us
I20260709 13:03:33.618191  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.497980  7823 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:33.624998  1524 connection.cc:256] RPC call timeout handler was delayed by -1.0311588909999614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
I20260709 13:03:33.636879  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:33.616123  8107 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.632606  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.639592  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.691722  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:33.692723  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.654381  1626 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
W20260709 13:03:33.696220  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 59437.5us
I20260709 13:03:33.695539  1629 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:33.457933  7977 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.696563  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64641.6us
I20260709 13:03:33.697052  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:33.693926  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54449.3us
I20260709 13:03:33.691722  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:33.704154  1628 raft_consensus.cc:2410] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:03:33.694126  1634 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:33.627486  7722 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.709160  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:33.712756  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.602044  8179 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.710119  7977 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.719082  7977 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.703058  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.721192  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:33.721692  1626 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:33.723150  1626 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:33.610594  7994 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.724053  8150 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:33.724591  8190 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:33.724828  8169 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:33.725113  8190 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.725256  8169 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:33.725504  8190 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:33.725539  7967 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:33.726181  7722 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:33.726711  7722 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:33.727177  7967 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:33.729969  7967 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:33.730365  7831 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:33.730674  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.731531  1707 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:33.634848  7719 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:33.732252  1701 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:33.732441  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.730811  7831 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:33.733620  7831 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.733840  7719 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.734551  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.734969  8061 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:33.735853  8061 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:33.736186  8061 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:33.736992  8061 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.738222  8061 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:33.723483  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.740231  1627 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:03:33.728446  2292 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:33.741175  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:33.742256  1708 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:03:33.724098  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 233005us
I20260709 13:03:33.743333  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.724151  8188 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:33.742339  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:33.752200  1631 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:33.746004  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.728881  8179 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.768734  1559 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:33.768842  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.753237  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.770241  1556 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:33.643244  8007 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:33.759012  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.634172  7823 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:33.762229  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.698864  8187 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.763473  1558 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:33.725672  8169 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.761842  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:33.749361  8188 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.764616  7831 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:33.835726  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 488130us
I20260709 13:03:33.778726  1707 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:33.790601  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.835676  8001 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:33.744544  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.846132  1560 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
W20260709 13:03:33.841684  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.765223  7975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.016s	user 0.059s	sys 0.105s
I20260709 13:03:33.727571  7722 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.780229  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.846136  1708 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
W20260709 13:03:33.851742  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 151446us
W20260709 13:03:33.854425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 116112us
W20260709 13:03:33.856194  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83957.5us
I20260709 13:03:33.857383  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.803095  8179 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.847162  8188 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:33.848556  7975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.100s	user 0.066s	sys 0.108s
I20260709 13:03:33.861379  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.862133  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.849423  7823 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.850369  8007 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:33.870118  1675 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.870026  1708 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:03:33.870850  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 114483us
I20260709 13:03:33.853873  7787 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:33.863547  7719 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:33.871031  8150 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:33.886690  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 189712us
I20260709 13:03:33.884171  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:33.902872  1708 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:03:33.885509  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.876621  7994 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:33.879648  7719 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.929612  1556 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:33.932282  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:33.933007  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.886396  7957 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:33.886840  7975 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:33.938876  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:33.885321  7967 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:33.938295  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.950675  1557 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:33.953548  1552 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:33.949200  1553 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:33.992085  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.993008  1556 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:33.994732  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:33.995580  1559 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:33.914104  8031 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:33.930907  7977 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:33.934355  7719 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.003247  1708 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
W20260709 13:03:34.049471  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 179431us
I20260709 13:03:34.060156  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:34.068068  1558 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.060537  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:34.069433  1556 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.062510  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:34.063215  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:34.059160  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.952400  7975 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.074306  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:34.080722  1557 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:33.951413  8200 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.086783  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.048122  8001 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:34.099176  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221275us
W20260709 13:03:34.099610  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.105996  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173767us
I20260709 13:03:34.085992  7975 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.084816  8031 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:34.132352  1706 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:34.097551  8200 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:34.115716  7847 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.001s
I20260709 13:03:34.175410  1556 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:34.194397  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:34.195384  1552 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:34.128456  8078 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:34.190526  8192 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:03:34.315483  1705 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
W20260709 13:03:34.140471  8193 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:03:34.143532  8194 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 13:03:34.144326  8197 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
W20260709 13:03:34.145207  8196 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:34.152084  8210 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:34.164152  1553 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:34.178030  1558 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:34.136736  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:34.311686  8078 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.118991  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:34.316174  1705 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:34.115355  8150 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.133535  8200 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.088263  8201 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:03:34.323686  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:34.192868  7977 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:34.115377  8205 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.198s
I20260709 13:03:34.328085  1706 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:03:34.329604  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 194415us
I20260709 13:03:34.131620  7831 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:34.356835  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:34.340170  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2458) took 2443 ms (client timeout 2228 ms). Trace:
I20260709 13:03:34.075316  8007 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.341612  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:34.343969  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 207314us
I20260709 13:03:34.193696  7847 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:34.347091  8189 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 13:03:34.192147  7975 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:34.352767  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 233832us
I20260709 13:03:34.388576  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:34.351420  1552 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:34.179224  8169 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:34.358992  1705 rpcz_store.cc:269] 0709 13:03:31.896701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.474983 (+1578282us) service_pool.cc:224] Handling call
0709 13:03:34.340091 (+865108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:34.398186  8207 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:03:34.399365  8207 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:34.391868  7959 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.408423  6983 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:34.410815  6983 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.356346  7994 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.382144  1558 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:34.342902  8078 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.393690  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.413045  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1130) took 1048 ms. Trace:
I20260709 13:03:34.421995  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:34.381793  1523 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:34.395591  7831 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.421665  1702 rpcz_store.cc:276] 0709 13:03:33.364982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.424859 (+ 59877us) service_pool.cc:224] Handling call
0709 13:03:33.425221 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:33.425728 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.425757 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:33.426333 (+   576us) log.cc:844] Serialized 52 byte log entry
0709 13:03:33.816190 (+389857us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:33.816253 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.816358 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.412850 (+596492us) raft_consensus.cc:1685] finished
0709 13:03:34.412866 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.412998 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":125,"thread_start_us":389369,"threads_started":1,"wal-append.queue_time_us":452229}
I20260709 13:03:34.416931  8169 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.415539  1553 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:34.425529  6983 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.425946  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.430940  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.433296  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:34.436295  8218 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.436996  8218 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:34.439235  8218 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.409157  7847 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.439841  8177 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.440442  8177 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.442795  8177 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:34.420668  8061 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.005s
I20260709 13:03:34.443470  8215 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.447664  8213 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:03:34.359720  7977 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.451437  1627 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:03:34.451886  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:34.454352  8198 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:03:34.455318  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.455714  7654 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:34.399736  8207 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:34.421073  8107 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.713s	user 0.006s	sys 0.005s
I20260709 13:03:34.418154  7967 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.439895  8208 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:34.354054  8210 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:03:34.467923  1635 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:34.469056  1634 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:34.351668  8001 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:34.480767  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 363643us
W20260709 13:03:34.481700  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50854.8us
I20260709 13:03:34.498986  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.499188  1632 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:34.482590  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.430485  8180 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:34.485132  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:34.503789  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.431216  8205 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:34.496850  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.506143  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.503216  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.469051  7654 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:34.488574  7959 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.406641  8214 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:03:34.495078  8061 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.466203  7831 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:34.499920  1631 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:34.430694  8163 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 13:03:34.464356  8218 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:34.481027  8169 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:34.486259  7975 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:34.503449  6983 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:34.496987  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.515216  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.510087  8180 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:34.493710  7847 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.521270  8031 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:34.523991  8219 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.521476  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:34.583860  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62504.4us
I20260709 13:03:34.585939  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.518992  8177 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.527546  8207 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.529102  7787 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:34.506067  8205 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:34.532346  1635 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:34.516037  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:34.521099  8190 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.795s	user 0.007s	sys 0.006s
W20260709 13:03:34.598192  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 82284.3us
I20260709 13:03:34.531682  1633 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:34.519927  7959 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.533624  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.600211  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1113) took 1242 ms. Trace:
I20260709 13:03:34.600591  1631 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:34.518810  7722 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:34.534154  7831 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:34.524333  8007 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 13:03:34.537978  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 1.14916e+06us
I20260709 13:03:34.538528  8005 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:34.536067  7654 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.588064  8177 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.539938  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1086) took 1071 ms. Trace:
I20260709 13:03:34.542958  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.544694  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.545682  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:34.605119  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59524.4us
I20260709 13:03:34.592444  7843 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.549294  8078 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.554502  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:34.551321  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.577730  7847 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:34.522244  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 89034us
I20260709 13:03:34.610982  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:34.612591  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:34.613655  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:34.573737  8180 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:34.616014  8208 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.616438  8208 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.616813  8208 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.594579  8214 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.621135  8214 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.522603  8061 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.622586  8078 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.603850  7975 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:34.606915  7814 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:34.624477  7977 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.624591  7814 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.602387  7831 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:34.536579  7719 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.638763  7719 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:34.538980  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 1.06405e+06us
I20260709 13:03:34.639743  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.640460  8200 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.642407  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:34.642751  8200 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.643399  8144 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:34.607151  7843 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:34.644042  7843 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.609337  8169 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.644925  8169 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.645292  8169 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.647990  7968 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:34.648428  8007 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:34.648974  8007 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.649355  8007 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:34.650180  7968 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:34.650460  8188 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.773s	user 0.007s	sys 0.004s
I20260709 13:03:34.650633  7968 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:34.651064  7968 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.650940  8188 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:34.602872  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:34.604447  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 61592.1us
I20260709 13:03:34.653142  8188 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.654433  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.655964  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.656126  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.656451  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.592676  7967 raft_consensus.cc:2764] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:03:34.563366  8218 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.657522  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.657728  2297 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:34.658113  2291 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.656068  1700 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:34.659157  8218 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.547713  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:34.660167  1634 raft_consensus.cc:2410] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:03:34.661092  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.663108  7831 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.605988  1525 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:34.665649  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 59745.3us
W20260709 13:03:34.604728  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60122.7us
I20260709 13:03:34.595937  7787 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:34.683429  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.599846  8205 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.604118  1629 rpcz_store.cc:276] 0709 13:03:33.468093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.710221 (+242128us) service_pool.cc:224] Handling call
0709 13:03:33.710560 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.710800 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.710826 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.711322 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.849977 (+138655us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:33.850017 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.852693 (+  2676us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.453717 (+601024us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:34.526381 (+ 72664us) raft_consensus.cc:1685] finished
0709 13:03:34.526428 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.539865 (+ 13437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5708544,"thread_start_us":138193,"threads_started":1,"wal-append.queue_time_us":416278}
I20260709 13:03:34.498323  8107 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:34.692718  2291 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:34.601696  7722 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.633361  7654 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.600734  1703 rpcz_store.cc:276] 0709 13:03:33.353158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.353296 (+   138us) service_pool.cc:224] Handling call
0709 13:03:34.595427 (+1242131us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001342d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:34.595543 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292985088}
I20260709 13:03:34.606143  8177 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.603545  8005 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.601665  8190 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.704133  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.704325  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.636737  8226 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.632966  7814 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:34.668479  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65694.8us
I20260709 13:03:34.707921  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.708480  2295 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.709550  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:34.669335  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:34.711016  8171 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:34.712404  8171 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:34.713661  8171 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:34.714238  8171 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:34.714566  8171 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:34.714895  8171 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:34.672361  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.685204  7719 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.666363  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.684130  2292 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:34.692538  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.608690  1627 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:34.694461  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.602895  8225 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.724255  8225 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:34.700587  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:34.673082  8187 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.000s
I20260709 13:03:34.700678  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:34.703086  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 89503.1us
I20260709 13:03:34.700415  7722 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:34.601397  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 165360us
I20260709 13:03:34.685544  7865 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:34.740041  1706 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: [18.8-18.8]
I20260709 13:03:34.740309  2296 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:34.609087  8219 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:34.718730  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:34.718245  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 51979.5us
I20260709 13:03:34.745165  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.745599  1705 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:34.685425  7802 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:03:34.746538  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:34.728175  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.747531  2297 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:34.748543  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 79337.2us
I20260709 13:03:34.736366  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:34.737105  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:34.750765  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:34.741838  1559 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:34.706691  8005 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.743037  1701 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:34.743196  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.750897  1707 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.752923  1708 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:34.741694  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.715498  7831 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:34.753554  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:34.760630  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.762089  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:34.762948  1703 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:34.765383  2297 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:34.742681  7865 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.729135  7994 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.775754  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:34.776523  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.732448  8003 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.767673  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:34.733520  8201 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:34.784982  2292 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:34.788002  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:34.860268  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 118641us
I20260709 13:03:34.855250  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.872332  2294 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:34.854348  7802 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:34.874109  1707 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:34.870119  2295 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:34.756081  7977 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:34.776764  8219 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:34.889914  8224 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.799661  8234 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:34.969084  2290 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:34.783033  7865 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:34.761758  7719 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:03:34.891141  8225 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:34.922649  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 213140us
I20260709 13:03:34.973803  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:34.922794  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:34.925032  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:34.974586  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 375397us
I20260709 13:03:34.975337  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:34.780385  7843 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:34.935662  1628 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
W20260709 13:03:34.925333  8179 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.059s	user 0.011s	sys 0.011s
W20260709 13:03:34.936715  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 161032us
W20260709 13:03:34.939199  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 151289us
I20260709 13:03:34.946702  8236 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:34.956139  8224 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:34.855975  8191 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:34.900483  7994 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:34.854976  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:34.974803  2297 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:34.932583  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:34.947754  7847 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.942720  8214 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:34.978614  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53676.7us
I20260709 13:03:34.981662  8225 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:34.777120  8226 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.156s
I20260709 13:03:34.996434  8238 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:34.997236  7975 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:34.997941  8243 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:34.967020  7967 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.020162  7967 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.021055  8201 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:35.023077  7967 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:34.924953  8003 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.011106  1632 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:35.022156  8201 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.026203  8003 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:34.977270  7842 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.027176  7842 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.029876  7842 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:34.977789  8234 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:35.031898  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 123598us
I20260709 13:03:35.031438  8234 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.032655  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:35.019691  8239 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:35.036227  8239 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:35.037480  8239 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.039212  8239 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.019482  8243 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.001475  8179 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:34.978161  8237 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:34.948159  8231 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:03:35.010825  7994 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.044750  8238 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.004153  8224 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:35.017293  7975 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:35.063810  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90065.1us
I20260709 13:03:35.046324  8176 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.046037  8191 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.048074  8243 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:35.070194  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:35.061239  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81539.2us
I20260709 13:03:35.056833  7994 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.054888  8190 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:35.084962  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52385.2us
I20260709 13:03:35.060256  8237 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:35.056444  8238 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.061082  8208 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.067663  7975 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:35.086769  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 71838.3us
I20260709 13:03:35.061005  7977 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:35.113832  8241 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:03:35.115123  8180 consensus_meta.cc:287] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.500s	user 0.007s	sys 0.001s
I20260709 13:03:35.068452  8179 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:35.082657  7823 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.198s	user 0.009s	sys 0.004s
I20260709 13:03:35.003393  8236 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:35.129269  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59144.9us
I20260709 13:03:35.164640  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:35.082942  7842 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.081415  8176 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.168921  7842 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.171427  7842 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.171869  8204 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:35.172106  8216 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.121922  8234 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.177381  8208 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.080210  8191 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.178275  7823 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.165995  8180 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.181433  8176 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.166216  8179 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.195736  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3059) took 1716 ms. Trace:
W20260709 13:03:35.196226  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 414274us
W20260709 13:03:35.195593  8007 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.003s
I20260709 13:03:35.130636  8241 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:35.202171  7968 consensus_meta.cc:287] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.007s
W20260709 13:03:35.212272  8230 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:03:35.217626  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 131211us
W20260709 13:03:35.215381  2262 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 2.165s (SENT) [suppressed 13 similar messages]
I20260709 13:03:35.215063  1626 rpcz_store.cc:276] 0709 13:03:33.479613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.749819 (+270206us) service_pool.cc:224] Handling call
0709 13:03:34.404944 (+655125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.405153 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.405180 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.405223 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:34.405262 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.405319 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.195482 (+790163us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54001e9450. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:35.195660 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505717632}
I20260709 13:03:35.067699  8219 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.226075  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.203118  7802 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:35.208844  8226 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:35.211946  7842 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:35.234395  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 108628us
W20260709 13:03:35.239560  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 264275us
I20260709 13:03:35.250895  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.257560  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:35.225551  7968 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.257006  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:35.256362  8204 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:35.270769  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:35.270920  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:35.272056  1707 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:35.271282  8204 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.272825  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:35.257097  7831 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:35.257165  7814 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.001s
I20260709 13:03:35.223878  8007 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.253047  7843 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.210825  8216 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.256949  8201 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.285178  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:35.233371  8234 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:35.256681  7842 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.248936  7847 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:35.297631  8258 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:35.298235  8258 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:35.290159  2289 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:35.286747  7814 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.301043  7968 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.301772  7814 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:35.302026  7842 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.292044  7831 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:35.292555  8232 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
W20260709 13:03:35.288826  8144 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:03:35.296588  8179 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.307832  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:35.305505  2293 raft_consensus.cc:2368] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
W20260709 13:03:35.304782  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53975us
I20260709 13:03:35.304783  2292 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:35.308832  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 541242us
I20260709 13:03:35.259132  8003 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 13:03:35.292893  8191 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:35.289285  7722 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.001s
I20260709 13:03:35.299093  8258 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.321271  1629 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:35.303478  7787 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.001s
W20260709 13:03:35.316563  8247 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:35.383594  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:35.370929  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:35.320874  8248 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:03:35.311473  8205 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 13:03:35.297649  8234 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:35.317011  8244 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:35.317231  8005 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.000s
I20260709 13:03:35.380995  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:35.361948  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:35.417542  2289 raft_consensus.cc:2410] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:35.299121  7847 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.279275  8240 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.303s
I20260709 13:03:35.305786  8061 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.243144  7977 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.322763  8007 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.432094  8216 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:35.434621  8220 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.890s	user 0.000s	sys 0.011s
I20260709 13:03:35.292903  8251 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:03:35.317256  8250 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.242s
I20260709 13:03:35.387653  2297 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:35.372237  8144 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:35.317374  7654 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.002s
I20260709 13:03:35.387542  7787 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.412334  8264 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:03:35.414676  7865 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.311136  8204 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:03:35.380282  7722 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.419044  6983 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.242476  8237 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.317202  8253 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.303326  8201 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:35.415864  8177 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
I20260709 13:03:35.378036  8226 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:35.309927  8241 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.431134  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:35.470562  1703 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:35.319888  8249 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.193s
I20260709 13:03:35.438344  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.416846  8244 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.317126  7994 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.369700  7842 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:03:35.425503  7847 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:35.318118  7831 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.418215  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:35.460772  1707 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:35.431670  8061 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.436981  7823 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.460773  6983 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.416658  8205 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.427235  8005 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.481972  2296 raft_consensus.cc:2368] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
I20260709 13:03:35.475694  8177 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.483219  8236 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.485776  8224 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.486747  7959 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:35.486944  8201 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:35.491361  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.425048  8238 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.497390  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:35.469172  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 181607us
I20260709 13:03:35.501490  8264 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:35.475967  8244 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:35.532660  7994 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.483745  8252 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:03:35.535218  7787 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.473065  8226 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:35.481619  8144 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:03:35.476758  8219 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:35.540918  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.542492  2291 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:35.544515  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47194.8us
I20260709 13:03:35.388968  7802 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.493165  7847 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.556332  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:35.556986  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:35.517354  6983 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.498023  8225 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:35.509760  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.496002  8061 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.498139  8241 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:35.520318  8171 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.805s	user 0.005s	sys 0.009s
I20260709 13:03:35.524098  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.524766  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:35.577698  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 269941us
I20260709 13:03:35.578562  1558 raft_consensus.cc:2376] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 11. Re-sending same reply.
I20260709 13:03:35.525370  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.578944  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:35.525484  8238 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.526675  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.529412  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.530092  8177 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.530332  1554 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:35.583977  8201 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:35.453393  8257 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:03:35.531752  7865 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.584832  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.532087  1708 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:35.510960  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.537761  1703 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:35.536020  8226 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:35.541595  1555 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:35.592321  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.474970  7975 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:35.551537  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.555035  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:35.555608  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.557022  2297 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:35.548282  8219 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:35.601130  8219 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.602896  8236 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:35.566264  8223 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.009s	user 0.003s	sys 0.024s
I20260709 13:03:35.603613  8236 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.568482  8005 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:35.568730  8222 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.971s	user 0.002s	sys 0.009s
I20260709 13:03:35.558051  8266 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.579967  1551 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
W20260709 13:03:35.580201  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53596.2us
I20260709 13:03:35.581457  1557 raft_consensus.cc:2368] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:03:35.606247  8266 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.606592  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.607052  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.607267  1551 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:35.582343  7722 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.607684  1557 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:35.607740  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.608263  7722 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:35.609058  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:35.576363  8169 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.931s	user 0.013s	sys 0.000s
I20260709 13:03:35.609472  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.609598  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.610038  1551 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.609927  8169 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.611160  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.612185  8169 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.576666  8061 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.613669  8007 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.614354  8007 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.616727  8191 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.617213  8007 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.617591  1551 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:35.617591  8191 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:35.619275  8191 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.619686  8191 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:35.639928  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.641319  6983 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:35.641883  6983 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:35.642735  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:35.642707  6983 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.586695  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.585006  8171 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.645929  7787 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.646469  7787 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.647210  8171 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:35.593236  1554 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:35.648937  7787 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.650801  1554 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:35.652019  2288 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:35.594237  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.606752  8266 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.581992  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.582733  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.645107  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.653810  1558 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:35.654702  1675 leader_election.cc:400] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:03:35.654460  1556 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:35.655079  1675 leader_election.cc:403] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:35.654484  8266 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.655809  7957 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:03:35.656188  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:35.656983  7823 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:35.657347  7823 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.657670  7823 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.655432  8177 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:35.661770  8177 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:35.662393  8177 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:35.663897  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:35.664754  8216 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:35.665124  8216 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.665450  8216 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.673291  1554 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:35.674232  7957 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:03:35.559988  7994 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.131s
I20260709 13:03:35.693303  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:35.703577  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.704479  1555 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:35.706574  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 213830us
I20260709 13:03:35.707085  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:35.709628  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 66965.3us
W20260709 13:03:35.711755  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.717181  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.736356  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.732988  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:35.740953  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.741917  1558 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:35.738428  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.744486  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.751648  1556 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.751227  1553 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:35.757123  7958 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:35.757673  7958 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:35.758080  7958 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.733487  7957 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:35.732187  7959 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:35.770701  2263 leader_election.cc:400] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:35.771113  2263 leader_election.cc:403] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:35.745476  7865 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:34.561954   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:35.777993  8235 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:35.778354  8061 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.738248  7843 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:03:35.738354  8210 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:03:35.780043  8251 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:35.780447  8251 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:35.775058  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.760182  8171 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.768692  8236 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:35.765923  8179 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.777151  7865 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.782828  1558 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:35.770305  7959 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.793756  1556 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:35.781162  8144 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:35.786034  8243 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
I20260709 13:03:35.774878  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.798691  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.815281  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:35.853510  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.854889  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:35.865525  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50392.8us
I20260709 13:03:35.867538  1601 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:35.780915  8216 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.794749  7865 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:35.796588  7959 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:35.883684  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:35.821483  7814 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.006s
I20260709 13:03:35.772801  8225 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:35.889600  1700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2465) took 2404 ms (client timeout 1827 ms). Trace:
W20260709 13:03:35.890731  1700 rpcz_store.cc:269] 0709 13:03:33.485005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.712498 (+227493us) service_pool.cc:224] Handling call
0709 13:03:35.889351 (+2176853us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54001b6f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:35.889517 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167597824}
I20260709 13:03:35.789055  8171 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.874188  8236 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:35.901181  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.057s
W20260709 13:03:35.902375  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.006s	sys 0.057s
I20260709 13:03:35.793421  8179 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:35.902966  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 520787us
W20260709 13:03:35.890808  7814 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.006s	sys 0.008s
I20260709 13:03:35.873685  8205 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.905448  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:35.839411  8177 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.895843  8224 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:35.892083  8272 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:35.894415  8235 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:35.926662  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.815783  8251 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:35.962695  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.963198  8251 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.963567  1557 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:35.964389  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.965852  1634 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:03:35.966041  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:35.966571  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:35.966828  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:35.967181  1628 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:35.967254  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:35.967643  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:35.872777  8243 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.968861  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:35.936299  8275 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:03:35.936723  7957 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:35.938498  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.942281  8216 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:35.896067  7823 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:35.948165  7831 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:35.987885  1552 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:35.958254  8272 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:35.961401  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.927052  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:35.972074  1633 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:35.984149  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:35.987493  1628 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:36.005733  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 78765.6us
I20260709 13:03:36.005918  1635 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.924502  8005 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:35.923700  7975 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.003418  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.000844  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:35.965615  8244 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.009164  1557 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:36.013495  1627 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:36.014447  1557 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:03:36.016481  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.009773  1634 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:36.009837  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:35.973377  8171 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.010301  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.012506  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.021667  1560 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:36.024830  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.026567  7975 raft_consensus.cc:2764] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:03:36.028599  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.029506  2289 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:36.030767  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.036556  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:36.038229  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.002084  7831 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.035264  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.009442  7977 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:36.050904  1560 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:36.052834  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.055881  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.060412  1635 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:36.046748  8216 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:36.062232  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.064649  1635 raft_consensus.cc:2393] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:36.046309  8289 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:36.065435  8144 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:36.061514  7957 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.067533  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.035386  7958 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.068923  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.070096  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.050093  7977 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:36.071693  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.073061  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.073626  1635 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:36.073843  2289 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:36.074466  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.057916  8211 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:36.039829  7847 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.078681  1627 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:36.080330  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1091) took 1173 ms. Trace:
I20260709 13:03:36.079088  8219 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.063652  8216 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:36.082412  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.102299  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:36.099109  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60921us
I20260709 13:03:36.098524  1629 rpcz_store.cc:276] 0709 13:03:34.907293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.907428 (+   135us) service_pool.cc:224] Handling call
0709 13:03:35.321044 (+413616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.381058 (+ 60014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.080121 (+699063us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54002968d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.080264 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908731648}
I20260709 13:03:36.103701  8216 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.103900  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:36.104648  1703 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:36.106501  1734 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37587 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37587 timed out after 15.000s (SENT)
I20260709 13:03:36.107702  8243 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.061707  8191 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.110514  8191 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.073226  7977 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:36.072880  7831 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.075619  8289 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.083113  8211 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.112766  8244 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.100391  8272 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.113636  8211 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.116282  1634 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:36.117738  7959 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.119112  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.120342  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:36.123410  8179 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.101759  7843 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:36.124469  2295 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:36.126855  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.127112  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.135121  1703 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:36.141359  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.142248  1627 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:36.141992  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:36.133421  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:36.147928  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:36.150575  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.152930  1703 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:36.154654  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.158607  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:36.160549  1702 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: [18.8-18.8]
I20260709 13:03:36.161397  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.162927  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.166855  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:36.168383  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:36.167701  1702 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:36.170866  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.154731  8241 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:36.180922  8241 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:36.181389  8241 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.181737  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1216) took 1482 ms. Trace:
I20260709 13:03:36.182116  2292 rpcz_store.cc:276] 0709 13:03:34.699272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.934207 (+234935us) service_pool.cc:224] Handling call
0709 13:03:35.304512 (+370305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.359695 (+ 55183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.181510 (+821815us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400771ad0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.181669 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850832256}
I20260709 13:03:36.165931  8224 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:36.187094  8224 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:36.187996  8224 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.192235  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1061) took 2278 ms. Trace:
I20260709 13:03:36.192679  1559 rpcz_store.cc:276] 0709 13:03:33.914036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.996327 (+ 82291us) service_pool.cc:224] Handling call
0709 13:03:34.059789 (+ 63462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.751449 (+691660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.191969 (+1440520us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54002f9754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:36.192151 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568089984}
I20260709 13:03:36.194741  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:36.195894  8252 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:36.196281  8252 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:36.196660  8252 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:36.197815  8219 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.176088  8061 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.178596  8208 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:36.164134  8290 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:36.183652  2292 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:36.201232  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:36.156777  7847 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.157845  7959 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.160377  7975 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:36.164884  7843 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:36.145069  7865 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.156543  8191 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:03:36.162737  8179 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:03:36.210197  8219 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:36.212276  8226 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.005s
I20260709 13:03:36.227713  7977 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.212001  1709 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:36.153921  7719 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:36.166152  8295 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:36.236899  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1446) took 1543 ms. Trace:
I20260709 13:03:36.252562  2291 rpcz_store.cc:276] 0709 13:03:34.693772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.693892 (+   120us) service_pool.cc:224] Handling call
0709 13:03:34.694170 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.543073 (+848903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.236596 (+693523us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54004c1654. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:36.236823 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950331648}
I20260709 13:03:36.222730  8216 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.230027  7865 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.232252  7958 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.234019  2289 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:36.234702  8191 consensus_queue.cc:1243] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:36.254164  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.242794  1626 raft_consensus.cc:2410] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 18 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:03:36.257889  7977 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:36.258723  8216 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.192309  8289 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.260053  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:36.262681  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52039us
I20260709 13:03:36.210862  1702 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:36.220696  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.211249  8290 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.204274  8208 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.282008  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.282425  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:36.283056  1703 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.285820  1702 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:36.281334  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.272995  1633 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:36.244792  8236 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.297508  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:36.299270  2291 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:36.279872  8290 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.302773  1708 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:36.303531  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.306413  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.222980  8300 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:03:36.323915  8208 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.325215  8208 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:36.316669  8289 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.328720  8241 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.329361  8289 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.331012  8241 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.331243  8289 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.245507  8226 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.266057  8252 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.333196  8226 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.335301  8252 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:36.336318  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2470) took 2849 ms (client timeout 2466 ms). Trace:
I20260709 13:03:36.332988  2295 raft_consensus.cc:1217] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Deduplicated request from leader. Original: 15.11->[16.12-18.13]   Dedup: 16.12->[18.13-18.13]
W20260709 13:03:36.336735  1701 rpcz_store.cc:269] 0709 13:03:33.486599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.744659 (+258060us) service_pool.cc:224] Handling call
0709 13:03:36.316642 (+2571983us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400096d54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.336232 (+ 19590us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073756032}
I20260709 13:03:36.337203  2295 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:03:36.333182  1551 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:36.338549  1551 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:36.339985  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:36.340600  1559 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:36.278594  8216 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:03:36.310384  1702 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:36.343205  1628 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:36.342545  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.307410  8031 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:36.345196  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 60745us
W20260709 13:03:36.348227  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 153420us
W20260709 13:03:36.348905  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.349287  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.349654  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.341723  8300 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.352769  8300 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:36.353286  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.353709  8300 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.355314  8300 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.358018  8207 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:36.360390  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.362136  2295 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:36.364001  1627 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:36.364269  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.364953  1631 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:36.365182  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.365805  1627 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.365844  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.366469  7959 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:36.366990  7959 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:36.346580  1551 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.372759  1551 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:36.363514  2295 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:36.366580  1631 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.380095  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.393626  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:36.399296  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.406410  1558 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:36.352157  8031 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:36.415632  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2934786129999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:03:36.410312  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:36.423885  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1220) took 1676 ms (client timeout 1805 ms). Trace:
I20260709 13:03:36.422711  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1435) took 1768 ms. Trace:
W20260709 13:03:36.424429  2296 rpcz_store.cc:269] 0709 13:03:34.730010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.760179 (+ 30169us) service_pool.cc:224] Handling call
0709 13:03:36.405836 (+1645657us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400233054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.405981 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093972224}
I20260709 13:03:36.424564  2293 rpcz_store.cc:276] 0709 13:03:34.654047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.654165 (+   118us) service_pool.cc:224] Handling call
0709 13:03:36.422454 (+1768289us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540031fbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.422625 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495383296}
I20260709 13:03:36.390065  7959 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.425041  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.423309  2297 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:36.426285  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.412125  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.428375  1703 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:36.380021  8180 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.436081  1554 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:36.431209  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.440057  1559 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:36.394671  8272 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:36.442500  1627 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:36.386740  7843 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:36.423398  8211 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.415884  7865 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.420725  7977 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:36.415956  7958 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.463580  1558 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [16.11-17.12]
I20260709 13:03:36.464776  7957 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.387068  8252 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:36.429829  8295 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:36.429628  8176 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.468631  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:36.436059  7847 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:36.472999  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.443200  8078 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:36.420300  8251 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:03:36.457249  8249 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:36.459206  8001 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:36.443702  7719 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:36.415674  8289 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:03:36.480825  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:36.496866  2288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1447) took 1802 ms (client timeout 2025 ms). Trace:
I20260709 13:03:36.441488  8226 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:03:36.497377  2288 rpcz_store.cc:269] 0709 13:03:34.694110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.694226 (+   116us) service_pool.cc:224] Handling call
0709 13:03:36.496650 (+1802424us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540027f454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:36.496775 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145211648}
I20260709 13:03:36.503161  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:36.504132  1556 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:36.505728  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.507962  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.511024  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.511513  2294 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:36.512329  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.513367  2295 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:36.515578  8240 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:36.516546  8240 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:36.484661  1551 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:36.517009  8240 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.468881  7958 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.495675  7842 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:36.518020  8240 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.518750  7842 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:36.519147  7842 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:36.519258  7958 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.519822  1554 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:36.495947  7847 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.520913  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.521546  7847 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.522559  1626 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:36.524226  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:36.520028  7842 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.475461  8176 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:36.526882  8169 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:03:36.520442  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.527428  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.528205  2292 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:36.528510  1627 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:36.483429  1631 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:36.516615  2288 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:03:36.533304  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.533879  1524 leader_election.cc:400] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15.
I20260709 13:03:36.534713  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:36.535962  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46830.2us
I20260709 13:03:36.469377  8272 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.472549  7654 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:03:36.540110  8233 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:36.540246  8169 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:36.496444  8078 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:36.541615  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.472074  7977 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:36.548702  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67934.3us
I20260709 13:03:36.557581  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.458655  8213 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:03:36.552923  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.554915  1626 raft_consensus.cc:2410] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:36.558883  1631 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.554060  1627 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:36.470194  7865 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.537580  7842 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.553007  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.484155  8001 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.547554  7823 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.581633  1556 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:36.550165  8240 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.586707  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.554481  7787 raft_consensus.cc:2764] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:36.467650  7843 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.594960  8272 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.596393  7977 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:36.597690  8308 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:03:36.475301  8295 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:36.598515  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45589.5us
I20260709 13:03:36.587165  8268 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:36.635300  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.598694  8251 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:36.553885  8200 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:03:36.574849  8235 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:36.600078  6983 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:36.602051  8213 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:36.602499  8203 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.651531  8203 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.651886  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.653335  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.653772  8203 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:36.597723  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.614264  1634 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:36.654568  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.655148  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.612802  8231 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:03:36.655642  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.655921  1634 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:36.655727  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.626868  7865 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.627593  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.671020  1626 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.588459  8001 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:36.555064  7722 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:03:36.554612  7719 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
W20260709 13:03:36.641038  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.620040  1631 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:36.646936  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.676971  1628 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.647135  8268 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.649911  7823 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:36.654085  7843 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.657250  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.660615  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.660547  8208 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.661556  8240 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:36.655934  6983 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:36.674733  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.634295  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:36.628141  8226 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:36.682843  2262 leader_election.cc:400] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:36.656140  7831 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.660070  8295 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.690086  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:36.697901  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 266748us
I20260709 13:03:36.698679  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.698971  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:36.686510  1629 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:36.671862  7957 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:36.693058  1628 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:36.694595  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.698823  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:36.702785  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:36.700731  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66528.1us
I20260709 13:03:36.704237  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:36.692963  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:36.704399  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.706037  1628 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:36.699844  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:36.704341  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.704869  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.684443  7823 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:36.720891  1631 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:36.713181  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:36.669654  1633 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:36.716833  1702 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:03:36.717660  1704 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:36.718871  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.706562  6983 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.723523  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.724011  1633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:36.725220  1633 raft_consensus.cc:606] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:36.725638  1633 raft_consensus.cc:614] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:36.720216  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:36.726312  1634 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:36.729663  8225 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:36.730616  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.731515  8264 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:36.732954  8264 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:36.733168  8007 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:36.733444  8264 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:36.734555  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:36.735198  1556 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:36.738803  1553 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:36.702930  7722 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:36.700191  8213 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.742693  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:36.706127  7958 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:36.672235  8078 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.753722  1626 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:36.711951  8295 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.705547  8251 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:36.706036  8179 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:36.705869  8231 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.758896  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:36.696583  8208 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.739843  8007 consensus_queue.cc:1243] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:36.738327  8258 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:36.750715  8061 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.702908  8235 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:36.800825  1631 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:03:36.738327  8163 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:36.803689  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.754702  8213 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:36.737636  8268 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.761119  8261 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:36.754346  8285 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:36.791884  8179 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:36.860591  1626 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:36.800428  7843 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:36.791715  8251 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:36.868422  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 168668us
I20260709 13:03:36.750655  7722 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:36.859537  8224 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:36.813263  8290 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:36.870164  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111389us
W20260709 13:03:36.856139  8315 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:03:36.922626  8208 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.761248  8295 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.892225  8211 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:03:36.926003  8207 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.001s
I20260709 13:03:36.926096  8295 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:36.926641  1630 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:36.893738  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:36.912348  1556 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:36.869293  8239 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.914067  8213 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:36.917389  8318 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:36.918416  8251 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.919699  8179 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:36.944211  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50590.4us
I20260709 13:03:36.885120  7847 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:03:36.948743  7722 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.951301  8290 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:36.901307  7957 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:03:36.896282  7843 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.952059  8328 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:37.002317  7843 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:37.004273  7722 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:36.954039  1703 raft_consensus.cc:2368] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20.
I20260709 13:03:36.881601  8317 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.870237  8264 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:36.901029  8261 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:36.964082  8300 consensus_meta.cc:287] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.608s	user 0.006s	sys 0.006s
I20260709 13:03:36.866237  8061 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.026999  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.028717  8261 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.024552  8317 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:36.995916  8213 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.057116  8207 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:37.057788  8207 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.058205  8207 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:37.064054  2260 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.518s (SENT)
I20260709 13:03:36.983228  8239 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.068480  8203 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:37.100840  2292 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:03:37.101642  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.100716  1630 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:37.113165  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.114133  1556 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:37.116761  1556 pending_rounds.cc:85] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:37.117359  1556 pending_rounds.cc:107] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:03:37.117786  1556 raft_consensus.cc:2889] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.129691  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:37.132699  8328 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:37.133944  8328 consensus_queue.cc:1243] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:37.135040  7957 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:37.135785  7957 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.137076  7957 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.143946  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.146961  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:37.110041  8226 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.112310  8290 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:37.108870  8007 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:03:37.163723  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.170224  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:37.147909  8163 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:37.148790  8328 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:03:37.123329  8241 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:03:37.150319  8226 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.186399  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:37.177601  7842 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.007s
I20260709 13:03:37.188915  1632 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:37.157002  8329 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:37.178730  8031 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.011s
I20260709 13:03:37.192155  6983 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:37.194123  2292 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:37.163089  8224 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:37.176512  8163 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.173846  8295 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:37.171917  8290 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.179040  8335 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:37.170671  8300 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:03:37.204751  7958 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:37.212788  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:37.221187  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.218943  1555 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 11.
I20260709 13:03:37.208135  8163 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:37.205900  8317 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:37.209491  7994 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:37.225371  7958 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.237624  7994 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:37.238348  7958 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.238646  7994 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.243119  8224 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.244385  8290 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.227963  8191 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:37.250733  7975 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:37.251250  7975 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:37.253151  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.204578  8031 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.264048  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:37.264797  1633 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:37.263278  8007 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:37.271253  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.217447  8295 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.273705  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.274569  2297 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:37.256274  7994 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.284811  1633 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:37.266067  8061 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.212411  8321 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:03:37.216616  8335 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:37.288726  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 93538.8us
I20260709 13:03:37.270305  8007 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.294049  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.294211  1701 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:37.294795  1702 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:37.296598  1702 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:37.302037  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.302565  1633 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:03:37.304332  1702 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:37.288200  8239 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.291913  8031 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.297724  8007 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.309480  1702 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:03:37.312368  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.313467  2290 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:03:37.313835  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:37.314751  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.314956  1633 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:03:37.315625  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:37.362054  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.372804  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.377588  1704 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:37.381870  1708 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:03:37.384801  2262 leader_election.cc:400] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:37.381815  1633 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:37.385788  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.386004  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.388373  1598 leader_election.cc:400] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20.
I20260709 13:03:37.388705  1598 leader_election.cc:403] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:03:37.392623  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.392916  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.394549  1705 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:03:37.393641  2294 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:37.315627  8210 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:37.412422  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.398571  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.414593  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.416409  2297 raft_consensus.cc:2368] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
I20260709 13:03:37.417660  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:37.419049  1673 leader_election.cc:400] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19.
I20260709 13:03:37.420089  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.420786  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.428591  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.432497  1552 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:03:37.391512  8208 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:37.433864  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.436935  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.437693  1705 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:37.395740  8261 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:03:37.304908  7975 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.395638  8317 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.446287  8261 consensus_queue.cc:1243] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:03:37.446650  1554 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:37.446856  8261 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:37.447223  8261 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:37.448175  1554 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:37.448984  1560 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:37.449961  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.451087  1559 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:37.394795  8239 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:37.460366  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.461112  2297 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
W20260709 13:03:37.440568  8344 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:03:37.383004  7843 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:37.413008  8233 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:37.466835  1552 raft_consensus.cc:1240] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:37.394627  7842 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:37.430996  8321 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:37.423503  8210 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.475517  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.496815  2297 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:37.460803  8345 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:03:37.501689  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.502767  1552 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:37.463037  1555 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:37.476168  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:37.481930  8343 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:03:37.445111  8207 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.468082  8317 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.509495  8207 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.510567  8317 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.497963  7975 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:37.487697  1705 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:37.520557  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.521148  1705 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:37.394906  7865 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:03:37.416977  8251 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:03:37.526047  8251 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.526518  8251 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:37.527997  8350 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:03:37.315460  8295 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.505853  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.530575  1557 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:37.524116  8031 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.532537  8031 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.448068  8261 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.542631  1633 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:37.432740  8285 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:03:37.399063  8180 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:03:37.548321  1709 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15.
I20260709 13:03:37.544873  2297 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:37.580451  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:37.491685  8001 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.812s	user 0.011s	sys 0.011s
I20260709 13:03:37.568423  1708 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:37.535650  8031 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.496416  8321 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.496224  7842 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.588493  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.562760  1705 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:37.598160  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.600231  1709 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:37.574805  8213 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:37.593266  7842 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:37.608625  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.608625  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.523630  8163 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.610456  2297 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:37.613178  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.614188  1708 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:37.611367  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:37.605082  7831 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.905s	user 0.013s	sys 0.009s
I20260709 13:03:37.593186  8321 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:37.618321  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.622385  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.625571  1705 raft_consensus.cc:2368] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20.
I20260709 13:03:37.628712  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.628901  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.627633  1709 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:37.640480  1708 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 15: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:37.642995  1635 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:37.607367  8239 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:37.612058  8031 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:37.592087  8001 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.678063  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.675165  8251 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.656872  8239 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.680979  1708 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:37.679106  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 207263us
I20260709 13:03:37.612795  8213 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.735219  1628 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:37.697237  8357 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:03:37.746666  1709 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:37.738312  1708 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:37.746790  2297 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:37.747962  2297 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:37.736097  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.703378  8335 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.698756  8163 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.746881  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1453) took 1735 ms. Trace:
I20260709 13:03:37.752507  1709 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:03:37.755589  1709 pending_rounds.cc:85] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 9
I20260709 13:03:37.756343  1709 pending_rounds.cc:107] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:37.756742  1709 raft_consensus.cc:2889] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.755239  2289 rpcz_store.cc:276] 0709 13:03:36.011817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.114739 (+102922us) service_pool.cc:224] Handling call
0709 13:03:37.746651 (+1631912us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:37.746817 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244127616}
I20260709 13:03:37.751401  2297 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.761700  1705 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:37.760068  2297 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:37.765645  1629 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:37.755021  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.679463  8031 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.701790  8007 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.745405  8233 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:37.636029  8210 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.759829  7865 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:37.782585  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.814307  2289 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:37.752519  8364 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:37.776407  1553 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:37.792958  8358 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:03:37.780787  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:37.794025  8355 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:03:37.851882  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:37.815831  2291 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:03:37.852614  1705 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17. Ops: [14.7-16.9]
I20260709 13:03:37.852632  2289 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:37.813288  7865 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:37.845786  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.847121  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 119571us
W20260709 13:03:37.791627  8351 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:03:37.854152  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.857643  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:37.860586  1706 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:37.818094  8364 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:37.863755  1705 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:03:37.864758  2297 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:37.866671  8007 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.864086  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:37.868824  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:37.869426  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.870129  1706 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:37.871332  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 2984) took 1907 ms. Trace:
I20260709 13:03:37.875147  7959 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:37.876343  8210 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:37.879422  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.882500  8264 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.883129  8264 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.884071  8200 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:37.884573  8200 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:37.884225  7865 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.885748  8264 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.885542  8200 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.883536  8231 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:37.887010  8364 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.887699  8231 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:37.888481  8364 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:37.891530  8231 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:37.886993  8268 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:37.892575  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:37.893296  1627 raft_consensus.cc:606] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:37.893705  1627 raft_consensus.cc:614] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:37.895295  1627 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:37.898551  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3001) took 1368 ms. Trace:
I20260709 13:03:37.898823  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:37.899045  1631 rpcz_store.cc:276] 0709 13:03:36.530548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.724377 (+193829us) service_pool.cc:224] Handling call
0709 13:03:36.724671 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.864749 (+140078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.898274 (+1033525us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b54001428d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:37.898480 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174615424}
I20260709 13:03:37.900763  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.902259  7823 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:37.904836  8213 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.905009  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1104) took 1296 ms. Trace:
I20260709 13:03:37.905529  1628 rpcz_store.cc:276] 0709 13:03:36.608734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.712840 (+104106us) service_pool.cc:224] Handling call
0709 13:03:37.904740 (+1191900us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:37.904914 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2327745920}
I20260709 13:03:37.907609  8177 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:37.908434  1632 rpcz_store.cc:276] 0709 13:03:35.964308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.967921 (+  3613us) service_pool.cc:224] Handling call
0709 13:03:36.122774 (+154853us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b58001f6898. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfe9c 00007f70cb8bfa5b 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:03:37.030781 (+908007us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b58001f6898. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:03:37.030982 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.195101 (+164119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.871068 (+675967us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003435d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:37.871263 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375112192}
I20260709 13:03:37.915867  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.858434  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.916564  1552 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:37.916851  1709 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:37.918344  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1112) took 1306 ms. Trace:
I20260709 13:03:37.919193  1627 rpcz_store.cc:276] 0709 13:03:36.611461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.894592 (+1283131us) service_pool.cc:224] Handling call
0709 13:03:37.894927 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.895674 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.918252 (+ 22578us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.913221  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:37.914255  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55817.7us
I20260709 13:03:37.914951  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.927623  1554 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:37.926992  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.936064  1552 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:37.885768  7865 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:37.928017  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.937454  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.940109  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:37.946800  1555 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:37.941727  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.943358  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1463) took 1061 ms. Trace:
I20260709 13:03:37.948114  2290 rpcz_store.cc:276] 0709 13:03:36.881795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.881967 (+   172us) service_pool.cc:224] Handling call
0709 13:03:36.900239 (+ 18272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.313946 (+413707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.943088 (+629142us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540031fbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:37.943293 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949463936}
W20260709 13:03:37.950759  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53474.9us
W20260709 13:03:37.955835  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.951619  2263 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:37.886988  8200 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:37.861624  7843 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:37.919907  7823 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:37.966212  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.972301  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.973555  1556 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:37.971057  1706 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:37.963515  1708 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:37.973757  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.937620  8264 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:38.002986  8264 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.011683  1709 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:37.983395  1552 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:37.980491  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:37.984359  8001 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:37.984748  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.986706  8367 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:37.976481  8373 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:37.968555  8240 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:03:37.993114  8031 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:37.993575  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:38.047376  2290 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:37.996537  1551 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:37.998299  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.013536  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.017282  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:37.971843  8251 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.982143  1554 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:37.937681  8365 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:03:38.027061  1706 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:38.035072  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9744e1e31604277953315481c56b171"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:38.053179  1627 raft_consensus.cc:606] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:38.053678  1627 raft_consensus.cc:614] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:38.053774  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.987468  8233 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:38.038268  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:38.058632  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.059741  1701 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:38.039618  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.060835  1556 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:38.042871  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.062069  1552 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:38.066458  8226 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.072309  1675 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:38.067692  8375 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:38.003520  7865 consensus_queue.cc:1059] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:03:38.068738  7994 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.784s	user 0.012s	sys 0.001s
I20260709 13:03:38.072969  2289 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:38.098978  1555 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:38.074000  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:38.068354  8208 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 13:03:38.082285  8367 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:38.083920  8365 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:38.088045  8375 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.091563  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.093828  8264 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.095304  8239 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.099960  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.107002  1555 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:38.102676  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:38.104671  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.070793  8007 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.110431  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:38.111089  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:38.103456  8365 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.113018  8365 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:38.115849  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:38.117808  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.114094  8375 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.099690  7959 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:38.103598  8367 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:38.092270  8321 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.103292  8208 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20.
I20260709 13:03:38.110174  8373 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:38.119491  8235 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:03:38.127696  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:38.119590  7865 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:38.119587  8219 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:38.126272  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.104089  7958 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:38.070950  8251 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.114953  8176 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.152763  1707 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:38.133025  7959 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:38.123299  8177 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:38.045012  1705 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:38.124372  8233 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:03:38.098191  7842 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.123432  8264 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:38.153040  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.145360  7865 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.139321  7843 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.124913  8364 raft_consensus.cc:3055] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:38.156554  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 221922us
I20260709 13:03:38.119935  8007 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.140246  8373 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.160567  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.196707  1705 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:38.197125  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.172163  1703 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:03:38.142303  1634 raft_consensus.cc:606] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:38.144142  1555 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:38.198407  1634 raft_consensus.cc:614] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:38.198251  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:38.198226  1703 pending_rounds.cc:85] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 11
I20260709 13:03:38.199913  1703 pending_rounds.cc:107] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:03:38.200222  1703 raft_consensus.cc:2889] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:38.158315  8266 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:38.201366  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 249829us
I20260709 13:03:38.197800  1701 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:38.210145  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.208863  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:38.210861  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:38.210899  1701 raft_consensus.cc:2376] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19. Re-sending same reply.
I20260709 13:03:38.215067  1555 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:38.218737  1557 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:38.179864  8364 raft_consensus.cc:740] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.223336  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.207883  8264 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:38.233139  1555 raft_consensus.cc:1217] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Deduplicated request from leader. Original: 14.10->[16.11-17.12]   Dedup: 16.11->[17.12-17.12]
I20260709 13:03:38.233932  1555 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [16.11-17.12]
I20260709 13:03:38.214193  8240 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:38.238867  1557 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:03:38.237975  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 147276us
I20260709 13:03:38.239598  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.203045  7843 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.222563  7958 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.228343  8285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.011s	sys 0.008s
W20260709 13:03:38.244895  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.228613  7787 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:38.232149  8264 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.238240  8208 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.254305  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.252465  8383 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:38.241060  8240 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.229903  8233 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:03:38.235459  7865 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.301831  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3006) took 1597 ms. Trace:
I20260709 13:03:38.312282  1631 rpcz_store.cc:276] 0709 13:03:36.704740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.900183 (+1195443us) service_pool.cc:224] Handling call
0709 13:03:37.900481 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:37.912973 (+ 12492us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:37.913034 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:37.929427 (+ 16393us) log.cc:844] Serialized 52 byte log entry
0709 13:03:38.156259 (+226832us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:38.156320 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.157213 (+   893us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.301517 (+144304us) raft_consensus.cc:1685] finished
0709 13:03:38.301546 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.301752 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13870,"raft.queue_time_us":7574,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16476,"thread_start_us":159086,"threads_started":1,"wal-append.queue_time_us":256305}
I20260709 13:03:38.314107  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:38.314793  2290 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:38.315634  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2511) took 1597 ms. Trace:
W20260709 13:03:38.316985  8348 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.938s	user 0.006s	sys 0.000s
I20260709 13:03:38.316102  1700 rpcz_store.cc:276] 0709 13:03:36.718321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.718447 (+   126us) service_pool.cc:224] Handling call
0709 13:03:36.718751 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.719077 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.719104 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.721584 (+  2480us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.070505 (+1348921us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b18003af480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:38.220874 (+150369us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:38.220930 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.221017 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.315328 (+ 94311us) raft_consensus.cc:1685] finished
0709 13:03:38.315351 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.315569 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11967,"raft.queue_time_us":43988,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":50645,"spinlock_wait_cycles":65927936,"thread_start_us":150142,"threads_started":1,"wal-append.queue_time_us":1560311}
I20260709 13:03:38.318706  1700 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:38.305346  8251 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.307969  8328 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:38.308667  8321 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.335135  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:38.335858  1555 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:38.244268  8373 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.259778  8264 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:38.310213  2296 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:03:38.256294  8377 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:38.255749  7958 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.304428  7842 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:38.321291  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.248915  8285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.012s	sys 0.008s
I20260709 13:03:38.282860  1701 raft_consensus.cc:2376] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 15. Re-sending same reply.
I20260709 13:03:38.335008  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:38.405776  1634 raft_consensus.cc:606] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:38.406296  1634 raft_consensus.cc:614] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:38.309062  7843 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:38.359858  2296 pending_rounds.cc:85] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 10
I20260709 13:03:38.424772  2296 pending_rounds.cc:107] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:03:38.425143  2296 raft_consensus.cc:2889] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:38.358829  1702 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:03:38.240846  7959 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.319938  8240 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:38.361318  2290 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:38.433974  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.434784  1559 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:38.435722  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.436039  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1483) took 2245 ms. Trace:
I20260709 13:03:38.436805  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:38.437386  1559 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.438980  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2518) took 1062 ms. Trace:
I20260709 13:03:38.439487  1703 rpcz_store.cc:276] 0709 13:03:37.376588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.394242 (+ 17654us) service_pool.cc:224] Handling call
0709 13:03:37.425729 (+ 31487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.200737 (+775008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.438758 (+238021us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400166054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:38.438917 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715673856}
I20260709 13:03:38.436576  2291 rpcz_store.cc:276] 0709 13:03:36.191018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.331854 (+140836us) service_pool.cc:224] Handling call
0709 13:03:37.747097 (+1415243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.852841 (+105744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.434857 (+582016us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b5400057154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:38.435750 (+   893us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400057150. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:38.435937 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411083392}
I20260709 13:03:38.276127  8208 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.273043  7787 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.462131  1559 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:38.335685  8235 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:03:38.423950  8211 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.333176  8328 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:38.282842  8364 consensus_queue.cc:260] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.467805  1555 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:38.431066  8226 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.467542  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:38.476545  1559 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:38.480933  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:38.486447  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.487517  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:38.499001  1559 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:38.503283  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.510857  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:38.607599  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 96855.8us
I20260709 13:03:38.616119  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.617110  1555 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:38.618654  1555 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:38.619174  1555 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:38.623255  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.629962  8007 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.689934  1559 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:38.631141  8385 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:38.633544  7787 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.489228  8285 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:38.584044  8200 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
W20260709 13:03:38.692286  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81578.1us
I20260709 13:03:38.723404  2260 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:38.468865  8373 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:38.679478  1555 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.725015  1555 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:38.728312  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:38.729115  1559 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:38.729115  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.730774  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:38.475044  8211 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.746816  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:38.718374  8364 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:38.491969  8226 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.790688  8211 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.805683  8226 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:38.806810  8365 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 13:03:38.807412  8365 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:38.808350  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.808845  1523 connection.cc:256] RPC call timeout handler was delayed by -1.7558211879999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 13:03:38.809631  1523 connection.cc:256] RPC call timeout handler was delayed by -1.7545627220000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 13:03:38.811830  8240 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:38.814589  8240 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.729072  8385 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.815990  8385 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.819695  8208 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.821169  1555 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:38.761433  8285 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:38.611784  8377 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.172s
I20260709 13:03:38.797542  7958 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.761554  8233 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:38.832621  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.844265  1523 connection.cc:256] RPC call timeout handler was delayed by -1.7013597389999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:03:38.844975  1523 connection.cc:256] RPC call timeout handler was delayed by -1.6847211879999691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:03:38.845831  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.843535  8177 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.002s
W20260709 13:03:38.868626  1523 connection.cc:256] RPC call timeout handler was delayed by -1.6379211880000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:03:38.869506  1523 connection.cc:256] RPC call timeout handler was delayed by -1.6167373619999807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 13:03:38.866600  8397 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:03:38.868180  8001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.015s
I20260709 13:03:38.867538  8177 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:38.876773  8393 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:38.892201  8001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.000s	sys 0.019s
W20260709 13:03:38.898875  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.898443  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.757411  8328 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.894244  8383 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:38.913576  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.915396  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.917284  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:38.917868  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 161558us
I20260709 13:03:38.913170  8393 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.919013  8377 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.906723  8211 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.933629  1701 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:38.934000  8061 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.933873  8211 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.934823  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.934628  8061 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.936045  8211 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.937179  8061 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:38.937747  8061 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.931314  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:38.932626  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.938524  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.939522  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:38.939859  8007 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:38.940445  1708 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:38.940817  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:38.941504  7787 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:38.942000  8211 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:38.942150  7787 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:38.942855  8211 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:38.932967  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.944164  1702 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:38.945254  1701 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:38.947621  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.949538  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.944761  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:38.946290  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.952122  1706 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:38.952847  1701 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:38.906330  8233 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.954519  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:38.915243  8383 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:38.928248  8396 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:38.932936  8328 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:38.959939  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:38.965960  1703 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:38.929085  8251 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.944808  8365 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:38.948493  8407 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:03:38.919535  8393 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:38.964757  8400 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:03:38.959038  7994 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:38.988723  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:38.973603  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3073) took 2892 ms (client timeout 1674 ms). Trace:
W20260709 13:03:38.989863  1635 rpcz_store.cc:269] 0709 13:03:36.081665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.102937 (+ 21272us) service_pool.cc:224] Handling call
0709 13:03:38.973386 (+2870449us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001e8a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:38.973500 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3441159936}
I20260709 13:03:38.974323  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:38.974575  8233 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.976787  8366 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:38.978875  8264 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.000s
I20260709 13:03:38.985234  1556 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:03:38.994719  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.920501  8377 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.998461  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:39.012176  8406 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:38.969878  8251 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.012972  8208 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:39.013366  7994 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:38.946895  8061 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.014036  8383 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.966995  8396 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:38.997089  8366 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:39.021574  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.953348  7787 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.020112  8007 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:39.030447  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.024653  8407 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:39.040613  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.048180  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.049096  2297 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:39.049717  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.032236  8231 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:39.028139  8365 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:39.023810  8264 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.047645  7957 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.099745  8369 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.199s	user 0.010s	sys 0.000s
W20260709 13:03:39.101780  8354 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.482s	user 0.009s	sys 0.004s
I20260709 13:03:39.103312  8396 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.105716  8377 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:39.112298  8061 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.113209  8239 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.113560  8251 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.055153  8405 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:39.024309  8211 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.114112  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.129724  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.061342  8398 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.055675  8231 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:39.135409  1708 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:03:39.136677  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.136911  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:39.145808  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.147902  1700 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:39.148334  2294 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:03:39.064309  8365 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.177824  1704 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:03:39.127947  7957 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:39.210095  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.069007  7994 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.160053  8376 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:39.273654  8424 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:03:39.287786  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2452) took 2061 ms. Trace:
I20260709 13:03:39.288358  1558 rpcz_store.cc:276] 0709 13:03:37.226918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.227146 (+   228us) service_pool.cc:224] Handling call
0709 13:03:37.244553 (+ 17407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.252093 (+  7540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.252128 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.263295 (+ 11167us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.495220 (+231925us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:37.495267 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.495364 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.101133 (+1605769us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:39.287400 (+186267us) raft_consensus.cc:1685] finished
0709 13:03:39.287432 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.287684 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":31876480,"thread_start_us":231528,"threads_started":1,"wal-append.queue_time_us":329802}
W20260709 13:03:39.292115  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.062274  8366 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.150369  8405 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.300383  8233 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.300952  8233 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.301517  8405 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.303071  8233 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:39.291191  8376 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:39.160508  8398 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.335450  2294 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:03:39.340517  7957 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:39.346112  8422 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:39.318712  8231 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:39.351441  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.375088  8007 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:39.376087  8007 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:39.376442  8007 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:39.376933  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1491) took 2776 ms. Trace:
I20260709 13:03:39.377516  2288 rpcz_store.cc:276] 0709 13:03:36.600691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.600854 (+   163us) service_pool.cc:224] Handling call
0709 13:03:36.601192 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:36.632393 (+ 31201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.632443 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:36.658247 (+ 25804us) log.cc:844] Serialized 52 byte log entry
0709 13:03:36.870729 (+212482us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:36.870756 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.376397 (+2505641us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b54006c02d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.376466 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.376533 (+    67us) raft_consensus.cc:1685] finished
0709 13:03:39.376551 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.376859 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6942,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5011,"spinlock_wait_cycles":46406016,"thread_start_us":212121,"threads_started":1,"wal-append.queue_time_us":270365,"wal-append.run_cpu_time_us":1854,"wal-append.run_wall_time_us":1009165}
I20260709 13:03:39.379993  8398 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:39.380941  8376 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.382010  8376 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.383298  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2519) took 2006 ms. Trace:
I20260709 13:03:39.383858  1704 rpcz_store.cc:276] 0709 13:03:37.376814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.397171 (+ 20357us) service_pool.cc:224] Handling call
0709 13:03:37.397454 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.178500 (+1781046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.382947 (+204447us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54003dc554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.383210 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4093594368}
I20260709 13:03:39.385438  8334 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.385982  8334 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.388024  8406 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.388618  8406 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.390743  8208 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:39.391355  8208 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:39.392272  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1090) took 1929 ms. Trace:
I20260709 13:03:39.393150  8396 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.393312  7847 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.393690  8396 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.393893  7847 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.354435  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:39.359309  8264 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.395920  8396 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.367395  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1100) took 2760 ms. Trace:
I20260709 13:03:39.397081  1626 rpcz_store.cc:276] 0709 13:03:36.606829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.760297 (+153468us) service_pool.cc:224] Handling call
0709 13:03:36.839052 (+ 78755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.893050 (+ 53998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.367141 (+2474091us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b54000ccbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.367311 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49065856}
W20260709 13:03:39.397795  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.379392  8353 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 1.883s	user 0.000s	sys 0.009s
I20260709 13:03:39.398876  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:39.361374  8365 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.419092  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:39.420586  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.401015  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.403112  1631 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:39.403563  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.421664  1702 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
W20260709 13:03:39.421903  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.404139  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.422217  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.422577  1627 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:39.423432  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:39.405817  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.424059  1627 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:39.410821  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:39.408147  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:39.409149  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 2992) took 3442 ms (client timeout 1750 ms). Trace:
I20260709 13:03:39.392812  1560 rpcz_store.cc:276] 0709 13:03:37.463186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.464037 (+   851us) service_pool.cc:224] Handling call
0709 13:03:37.464539 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.465043 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.465088 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.480393 (+ 15305us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.987436 (+1507043us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b1800322120. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:39.055405 (+ 67969us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:39.055468 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.055544 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.391845 (+336301us) raft_consensus.cc:1685] finished
0709 13:03:39.391872 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.392188 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15598,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":2709,"spinlock_wait_cycles":48728832,"thread_start_us":67673,"threads_started":1,"wal-append.queue_time_us":1749909}
I20260709 13:03:39.416355  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.428473  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1467) took 2176 ms. Trace:
I20260709 13:03:39.428879  2292 rpcz_store.cc:276] 0709 13:03:37.252457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.252621 (+   164us) service_pool.cc:224] Handling call
0709 13:03:37.252969 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.261170 (+  8201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.261199 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.261685 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.390317 (+128632us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:37.390365 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.390441 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.378868 (+1988427us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:39.420804 (+ 41936us) raft_consensus.cc:1685] finished
0709 13:03:39.420826 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.428402 (+  7576us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3988,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2726,"spinlock_wait_cycles":31567232,"thread_start_us":128272,"threads_started":1,"wal-append.queue_time_us":180407}
I20260709 13:03:39.432737  8402 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:39.433540  8402 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:39.425408  1630 rpcz_store.cc:269] 0709 13:03:35.970637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.976216 (+  5579us) service_pool.cc:224] Handling call
0709 13:03:39.408931 (+3432715us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54002756d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:39.409077 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111238912}
I20260709 13:03:39.431341  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.439481  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.440423  1627 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21.
I20260709 13:03:39.445269  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:39.450798  1709 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:39.454417  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2464) took 1021 ms. Trace:
I20260709 13:03:39.454849  1559 rpcz_store.cc:276] 0709 13:03:38.433168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.730257 (+297089us) service_pool.cc:224] Handling call
0709 13:03:38.765684 (+ 35427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.765899 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:38.895906 (+130007us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.453963 (+558057us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.454124 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.454163 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:39.454203 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.454263 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.454359 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":306461,"prepare.queue_time_us":174048,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":358698,"threads_started":2,"wal-append.queue_time_us":353155}
I20260709 13:03:39.456532  1627 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:39.454417  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:39.424074  7654 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.463729  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.477722  1626 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:39.472671  1627 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:39.446884  7957 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:39.435704  8402 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.481824  6983 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:39.484269  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.488297  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1080) took 2063 ms. Trace:
I20260709 13:03:39.461717  8233 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:39.463490  7959 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:39.488735  1553 rpcz_store.cc:276] 0709 13:03:37.424988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.425122 (+   134us) service_pool.cc:224] Handling call
0709 13:03:37.425463 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.848078 (+422615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.488060 (+1639982us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54002ba2d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.488239 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796743040}
I20260709 13:03:39.477545  8393 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:39.463706  7847 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.477746  8398 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:39.489403  1627 raft_consensus.cc:2376] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18. Re-sending same reply.
I20260709 13:03:39.377243  8007 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.486552  7975 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.503715  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.507036  1626 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:03:39.480293  8365 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:03:39.506819  1627 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:39.508291  8433 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.522024  8433 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.496599  7959 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.491460  8233 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:39.498950  8393 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:39.547339  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.525354  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.495112  8334 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.483984  8405 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:03:39.484180  7654 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.504271  8398 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:39.503126  8225 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:39.549820  1627 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:39.527829  1632 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
W20260709 13:03:39.550006  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.525054  7975 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.550645  1626 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:39.620774  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:39.550302  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 151609us
W20260709 13:03:39.615617  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.547446  8233 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:39.663961  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.579991  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:39.587785  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.722903  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.535845  7959 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:39.549289  8393 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:39.728128  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.729027  2290 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:39.718811  8398 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.663466  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.728191  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.731613  1628 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:39.662619  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1150) took 2168 ms (client timeout 2080 ms). Trace:
I20260709 13:03:39.583978   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:39.576853  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1109) took 2966 ms. Trace:
I20260709 13:03:39.722257  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.643723  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1110) took 3032 ms. Trace:
I20260709 13:03:39.591698  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:39.589344  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:39.584488  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.586961  8367 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:39.729231  8406 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.723344  1702 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:39.754498  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 174604us
W20260709 13:03:39.732944  1707 rpcz_store.cc:269] 0709 13:03:37.494527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.594816 (+100289us) service_pool.cc:224] Handling call
0709 13:03:39.662362 (+2067546us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54003e1f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:39.662532 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247092480}
I20260709 13:03:39.735451  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:39.754506  1523 connection.cc:256] RPC call timeout handler was delayed by -1.27724051499996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:03:39.661227  8231 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:39.743137  1629 rpcz_store.cc:276] 0709 13:03:36.610895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.423422 (+812527us) service_pool.cc:224] Handling call
0709 13:03:37.439853 (+ 16431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.770732 (+330879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.643388 (+1872656us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.643637 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748687744}
I20260709 13:03:39.548589  8007 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:39.745060  6983 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.740845  1633 rpcz_store.cc:276] 0709 13:03:36.610616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.394057 (+783441us) service_pool.cc:224] Handling call
0709 13:03:37.394472 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.555184 (+160712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.576517 (+2021333us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001e8554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.576772 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340095872}
I20260709 13:03:39.749135  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:39.749840  8435 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.745047  8404 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.751423  8236 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.751930  8179 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:39.752357  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.844094  1703 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:39.844287  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.734543  8225 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.568629  7823 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.184s
W20260709 13:03:39.584477  8428 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:39.580997  8163 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.172s
I20260709 13:03:39.753086  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:39.755884  1708 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:39.756394  1523 connection.cc:256] RPC call timeout handler was delayed by -1.2459789339999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
W20260709 13:03:39.855967  1523 connection.cc:256] RPC call timeout handler was delayed by -1.1287477169999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:03:39.856454  1523 connection.cc:256] RPC call timeout handler was delayed by -1.1004546479999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
W20260709 13:03:39.856845  1523 connection.cc:256] RPC call timeout handler was delayed by -1.0781787279999548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 13:03:39.737663  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2453) took 1482 ms. Trace:
W20260709 13:03:39.756577  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.756583  1630 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:39.742826  1627 raft_consensus.cc:2368] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24.
I20260709 13:03:39.861928  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.550056  8433 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.753923  8334 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:03:39.758642  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.889688  1702 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:39.759940  6983 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.765756  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.893419  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.895080  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.754099  8180 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.896154  8180 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.845137  1629 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:39.843680  8435 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.901989  6983 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.902292  8435 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:39.899317  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.848778  8404 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.905260  8180 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.745110  8367 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:39.848021  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 366557us
I20260709 13:03:39.906917  8367 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:39.908146  8367 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:39.908991  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.849769  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:39.909662  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.910646  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.910835  8437 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:39.911302  1633 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.759914  7994 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:39.864148  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.857854  1557 rpcz_store.cc:276] 0709 13:03:38.255482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.258043 (+  2561us) service_pool.cc:224] Handling call
0709 13:03:38.258449 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.258663 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:38.293837 (+ 35174us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.737243 (+1443406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.737295 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:39.737369 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.737456 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.737596 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":733120,"prepare.queue_time_us":41077,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":70,"raft.queue_time_us":775525,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":715524,"threads_started":2,"wal-append.queue_time_us":596276}
I20260709 13:03:39.862717  1627 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:39.923990  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.758057  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:39.857508  8321 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:39.880268  8398 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.905501  8404 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:39.762710  8264 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
W20260709 13:03:39.847462  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 98410.6us
I20260709 13:03:39.937853  6983 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:39.938745  6983 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:39.912487  1626 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.926486  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:39.942379  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.943415  1556 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:39.947876  1554 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:39.949195  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.951495  1560 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:39.952494  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:39.957167  1560 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:39.956354  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.959468  1553 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:03:39.961484  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:39.962091  1554 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.968580  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.969744  1555 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:39.970551  1707 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:03:39.971304  1555 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:03:39.972095  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:39.972467  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.972725  1558 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:39.972929  1629 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:39.968611  6983 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:39.972095  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.960623  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:39.971776  1555 pending_rounds.cc:85] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:39.979403  1555 pending_rounds.cc:107] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:03:39.979701  1555 raft_consensus.cc:2889] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.981869  1557 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:39.981756  1626 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:39.982904  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.983882  1626 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:39.920819  7994 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.989444  1702 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:39.928334  8435 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:39.852974  8236 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:39.991137  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 233122us
I20260709 13:03:39.922683  8433 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:39.992867  1708 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:39.864503  8211 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:03:40.003274  1633 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
W20260709 13:03:40.008260  1631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.027s	sys 0.047s
W20260709 13:03:40.009671  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.027s	sys 0.047s
I20260709 13:03:40.010011  1626 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:40.010999  7975 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.011073  1556 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:03:40.016237  1556 pending_rounds.cc:85] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:40.016716  1556 pending_rounds.cc:107] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:03:40.017277  1556 raft_consensus.cc:2889] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:40.007328  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.011466  1626 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:40.005003  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.012012  7975 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.013669  1629 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:40.014603  8236 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.024895  1633 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:40.029110  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.029315  1555 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:40.029808  1557 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:39.856091  8179 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.037108  1633 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:40.044714  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:40.045562  2291 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:40.046566  6983 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:40.046923  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:39.995891  7994 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:40.031838  7957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.013s	sys 0.079s
I20260709 13:03:40.068809  6983 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:40.070658  6983 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:40.072047  6983 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:40.005708  8404 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:40.084190  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.088030  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:40.068732  7957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.080s
I20260709 13:03:40.031028  7975 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:40.038313  8251 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:40.108319  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.034s	sys 0.058s
I20260709 13:03:40.006242  8393 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.133442  1626 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:03:40.122978  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.038s	sys 0.058s
I20260709 13:03:40.121152  8251 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:40.120673  7957 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:40.137511  8251 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.138118  7957 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.138832  8213 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.124127  8266 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:40.140228  8266 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:40.120945  7975 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.144760  1458 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 10 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:40.146536  1458 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 10 to 13, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 13 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.127467  8429 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.149339  8429 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:40.151736  1452 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 212) took 28352 ms (client timeout 15000 ms). Trace:
I20260709 13:03:40.153303  8233 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:40.155438  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.160594   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:40.156544  8398 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:40.065948  8376 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:03:40.028298  8433 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.157457  8251 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.158949  1635 raft_consensus.cc:2410] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:03:40.159363  8429 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:40.135946  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 597484us
I20260709 13:03:40.084607  8437 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:40.188545  1700 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:40.191851  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.078800  8440 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.160207  8393 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.219704  8393 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:40.220659  8393 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:40.224349  8443 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.225106  8443 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.226372  1700 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:03:40.178094  7787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.018s	sys 0.024s
I20260709 13:03:40.216785  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.244864  1674 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21.
I20260709 13:03:40.139349  8213 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.174600  7654 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:40.227695  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.221473  8393 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.262013  8213 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.269556  8433 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.227566  8443 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.272194  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:40.243216  7787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.018s	sys 0.024s
I20260709 13:03:40.278885  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1505) took 1250 ms. Trace:
I20260709 13:03:40.188177  8398 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:40.185950  8437 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:40.279316  2296 rpcz_store.cc:276] 0709 13:03:39.028528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.031609 (+  3081us) service_pool.cc:224] Handling call
0709 13:03:39.034213 (+  2604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.273078 (+238865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.273112 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.273156 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:39.273190 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.278602 (+1005412us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400350054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:40.278664 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.278819 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548824832}
I20260709 13:03:40.248556  8429 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.251080  8295 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:40.304977  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2465) took 1871 ms. Trace:
I20260709 13:03:40.315991  1634 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:40.314808  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.317324  1554 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:40.313645  1551 rpcz_store.cc:276] 0709 13:03:38.433583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.762472 (+328889us) service_pool.cc:224] Handling call
0709 13:03:38.815414 (+ 52942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.815643 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:38.929479 (+113836us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.761096 (+831617us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.302774 (+541678us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.302831 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:40.302884 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.302956 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.304908 (+  1952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495391,"prepare.queue_time_us":126542,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":710508,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":5143,"thread_start_us":982819,"threads_started":4,"wal-append.queue_time_us":133851}
W20260709 13:03:40.318262  2292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.064s	sys 0.092s
W20260709 13:03:40.318696  2292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.064s	sys 0.093s
I20260709 13:03:40.319286  1556 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: [15.12-16.13]
I20260709 13:03:40.312036  1458 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 12, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 12 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:03:40.323860  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:40.324535  2295 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
W20260709 13:03:40.341871  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.303404  8439 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:40.358589  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.311362  8398 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.328801  8179 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.311743  8349 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:03:40.303393  8445 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:03:40.321880  8429 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.329685  8295 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.243848  8440 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:40.373301  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.008s	sys 0.026s
W20260709 13:03:40.359122  8169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.016s	sys 0.015s
W20260709 13:03:40.377028  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.368117  8438 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:03:40.360424  8439 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:40.380568  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.008s
W20260709 13:03:40.422914  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.009s	sys 0.008s
W20260709 13:03:40.423002  1673 connection.cc:256] RPC call timeout handler was delayed by -1.6782772490000184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 13:03:40.423652  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:40.429674  1523 leader_election.cc:400] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20.
I20260709 13:03:40.427906  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:40.393193  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.008s	sys 0.026s
W20260709 13:03:40.380050  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 295934us
I20260709 13:03:40.392760  8449 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.400120  2295 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:40.410125  7865 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.412418  7958 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:40.412580  8458 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:40.414177  8251 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.416050  8440 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.417347  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:40.419319  8377 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:03:40.421614  7959 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.001s
I20260709 13:03:40.421533  8398 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.368376  7975 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:40.445353  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 692264us
I20260709 13:03:40.433184  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.434109  8396 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.378644  7994 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.355911  8437 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.430373  8378 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.374789  8233 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.436893  8402 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.433274  1458 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 4 to 11, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.439170  8450 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.450876  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.452503  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.480219  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.481027  1551 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:40.481420  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:40.482030  1554 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:40.452831  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.484071  1552 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:40.484913  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.485467  1552 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:40.486194  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.486899  1552 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:03:40.439409  7958 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:40.454826  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.455950  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:40.489511  1555 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:40.489945  1560 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:40.489346  1633 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:03:40.490631  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.492612  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.498690  1554 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:03:40.470337  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 375473us
I20260709 13:03:40.455696  8377 consensus_queue.cc:1243] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:40.500334  8233 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:03:40.499609  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 195064us
I20260709 13:03:40.501690  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:40.508301  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.510080  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.510259  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.510803  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.512310  1555 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.514442  1554 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:03:40.515398  1554 pending_rounds.cc:85] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:40.515683  1554 pending_rounds.cc:107] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:03:40.516018  1554 raft_consensus.cc:2889] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:40.447428  7959 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.507580  1674 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.511668  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.520117  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.522346  1551 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:40.523416  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.524056  1551 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:40.447952  8406 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:03:40.530435  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:40.499514  8231 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.001s
I20260709 13:03:40.491362  8450 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.480675  8393 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.480726  8460 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:40.537667  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86897.8us
I20260709 13:03:40.548310  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:40.549341  1627 raft_consensus.cc:2410] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:03:40.550180  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.545150  1551 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:40.520536  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:40.552249  1554 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.549347  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.547073  1560 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:40.556051  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.556977  1551 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:40.560791  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:40.442487  8458 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.464983  8396 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.563519  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.525251  7843 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:40.568782  8376 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.576690  1555 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:40.580397  1554 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
W20260709 13:03:40.596580  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 149605us
W20260709 13:03:40.603631  1673 connection.cc:256] RPC call timeout handler was delayed by -1.179677249000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260709 13:03:40.443840  8449 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:40.613404  1673 connection.cc:256] RPC call timeout handler was delayed by -1.0770772490000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 13:03:40.550190  8450 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.614245  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:40.608822  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45409.4us
W20260709 13:03:40.615664  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65558.4us
I20260709 13:03:40.623955  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:40.552060  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:40.625048  1701 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:40.617695  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 110174us
I20260709 13:03:40.619978  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.621722  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:40.622031  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 190286us
I20260709 13:03:40.558961  8169 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.635475  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.570993  8236 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.642854  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:40.637135  1458 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 12 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.650316  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.569501  8231 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:40.469760  8378 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.659986  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:40.623181  8376 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.528877  8266 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.657377  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:40.629848  8461 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.676083  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:40.614916  7654 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.553221  8377 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:03:40.663105  1554 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:03:40.664299  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.622895  8007 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:03:40.677276  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.774s (SENT) [suppressed 21 similar messages]
W20260709 13:03:40.649341  8367 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.741s	user 0.005s	sys 0.009s
W20260709 13:03:40.678260  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51732.2us
I20260709 13:03:40.552953  8466 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
I20260709 13:03:40.684896  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:40.695178  8451 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.696278  8007 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:40.653240  8236 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.674309  8376 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.692389  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:40.715797  1707 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.669852  8231 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.700462  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:40.654234  8453 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.634022  8031 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:03:40.691201  2289 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:40.726680  8208 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.677278  8480 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:40.677995  8454 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:40.697278  8007 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.647881  8393 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:03:40.646785  7958 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.712160  8367 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:40.723871  8401 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.827s	user 0.009s	sys 0.001s
W20260709 13:03:40.722623  1525 connection.cc:256] RPC call timeout handler was delayed by -1.2581241589999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 13:03:40.633701  8459 consensus_queue.cc:1059] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:03:40.710633  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.674849  8429 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.525823  8439 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.694236  8452 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
W20260709 13:03:40.738724  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54667.5us
W20260709 13:03:40.744849  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67606.8us
I20260709 13:03:40.690446  8478 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:40.682380  8233 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:03:40.717401  8236 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.707177  8456 raft_consensus.cc:2764] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 13:03:40.719729  8169 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:40.729622  8231 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:40.762617  1525 connection.cc:256] RPC call timeout handler was delayed by -1.0097241589999726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:03:40.746891  8480 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.737013  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:40.762583  7975 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:40.797510  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.763219  8485 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:40.801215  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.784066  7994 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:40.172812  1452 rpcz_store.cc:269] 0709 13:03:11.798941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.799296 (+   355us) service_pool.cc:224] Handling call
0709 13:03:40.151496 (+28352200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:28.800382 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:28.916419 (+116037us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:28.916476 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:03:28.916490 (+    14us) tablet.cc:662] Decoding operations
0709 13:03:29.028980 (+112490us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:29.029125 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:29.029157 (+    32us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:03:29.030683 (+  1526us) tablet.cc:701] Row locks acquired
0709 13:03:29.030744 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:03:29.031070 (+   326us) write_op.cc:270] Start()
0709 13:03:29.031251 (+   181us) write_op.cc:276] Timestamp: P: 1783602209030987 usec, L: 0
0709 13:03:29.031278 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:03:29.032109 (+   831us) log.cc:844] Serialized 6665 byte log entry
0709 13:03:30.044231 (+1012122us) op_driver.cc:464] REPLICATION: finished
0709 13:03:30.286096 (+241865us) write_op.cc:301] APPLY: starting
0709 13:03:30.286217 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:30.403548 (+117331us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:30.403694 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:31.108175 (+704481us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:31.108731 (+   556us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:03:31.108905 (+   174us) write_op.cc:312] APPLY: finished
0709 13:03:31.109610 (+   705us) log.cc:844] Serialized 156 byte log entry
0709 13:03:31.215691 (+106081us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:31.216189 (+   498us) write_op.cc:454] Released schema lock
0709 13:03:31.216267 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:31.260222 (+ 43955us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:31.260260 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":721942,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":241492,"apply.run_cpu_time_us":41826,"apply.run_wall_time_us":974032,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75255,"num_ops":17,"prepare.queue_time_us":662949,"prepare.run_cpu_time_us":30996,"prepare.run_wall_time_us":550299,"raft.queue_time_us":582526,"raft.run_cpu_time_us":27002,"raft.run_wall_time_us":348951,"replication_time_us":1012763,"thread_start_us":1570466,"threads_started":5,"wal-append.queue_time_us":552900,"wal-append.run_cpu_time_us":45120,"wal-append.run_wall_time_us":2556759}]]}
W20260709 13:03:40.788108  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77544.8us
W20260709 13:03:41.080695  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 210359us
I20260709 13:03:41.081782  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:41.087361  1525 leader_election.cc:400] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24.
I20260709 13:03:41.093621  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1096) took 1147 ms. Trace:
I20260709 13:03:40.894527  1552 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:41.094388  1558 rpcz_store.cc:276] 0709 13:03:39.946233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.973467 (+ 27234us) service_pool.cc:224] Handling call
0709 13:03:39.973744 (+   277us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:40.361013 (+387269us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:40.361051 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:40.375030 (+ 13979us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.523816 (+148786us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:40.523852 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.523925 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.092708 (+568783us) raft_consensus.cc:1685] finished
0709 13:03:41.092799 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.093303 (+   504us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153137,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":532514,"threads_started":2,"wal-append.queue_time_us":621739}
I20260709 13:03:41.098445  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:41.109102  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2978) took 1017 ms. Trace:
I20260709 13:03:41.109825  1556 rpcz_store.cc:276] 0709 13:03:40.091935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.092077 (+   142us) service_pool.cc:224] Handling call
0709 13:03:40.319036 (+226959us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:40.319686 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.108452 (+788766us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000767d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.108824 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521047808}
I20260709 13:03:41.116483  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:40.803514  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:40.969003  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3090) took 1568 ms. Trace:
W20260709 13:03:40.866487  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135472us
I20260709 13:03:41.146642  1635 rpcz_store.cc:276] 0709 13:03:39.400578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.400724 (+   146us) service_pool.cc:224] Handling call
0709 13:03:40.951616 (+1550892us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54003449d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:40.968889 (+ 17273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695899520}
I20260709 13:03:41.147948  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1098) took 1172 ms. Trace:
I20260709 13:03:41.148658  1559 rpcz_store.cc:276] 0709 13:03:39.946780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.973564 (+ 26784us) service_pool.cc:224] Handling call
0709 13:03:39.977290 (+  3726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.977490 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.977527 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.977584 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:39.977628 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.977699 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.118574 (+1140875us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b54000776d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.118859 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7757568}
W20260709 13:03:40.859153  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68033.2us
I20260709 13:03:40.959560  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.153149  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:41.153908  1700 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:40.954196  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2542) took 1367 ms. Trace:
I20260709 13:03:41.156234  1702 rpcz_store.cc:276] 0709 13:03:39.587242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.890778 (+303536us) service_pool.cc:224] Handling call
0709 13:03:39.891217 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.992408 (+101191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.953911 (+961503us) spinlock_profiling.cc:243] Waited 98 ms on lock 0x7b54001b7e54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:40.954117 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225298688}
I20260709 13:03:40.813872  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2523) took 2788 ms. Trace:
I20260709 13:03:40.882277  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:40.857873  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 197974us
I20260709 13:03:41.165165  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:40.988557  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:40.995498  2295 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:41.137555  8007 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.172222  1630 raft_consensus.cc:1240] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:41.173552  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.174587  1632 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:41.174705  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.001135  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:41.172281  8007 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:41.177299  8476 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.453s	user 0.000s	sys 0.003s
I20260709 13:03:41.178931  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3091) took 1777 ms. Trace:
I20260709 13:03:41.160086  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:40.815863  7654 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:41.180071  8454 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:41.179958  8451 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:41.003716  7823 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:40.945494  7865 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.002s
I20260709 13:03:41.182159  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1469) took 1769 ms. Trace:
I20260709 13:03:40.799453  7959 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.190205  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:40.786116  8176 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:41.191628  8176 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:41.192350  8176 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:41.012573  8231 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.201586   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:41.201825  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:40.943130  8233 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.204692  8233 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.025801  8236 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:40.956991  8261 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:41.208359  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.209812  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.210564  8236 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.809686  8266 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.947316  8366 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:41.212507  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.182163  7823 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.214229  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:40.990722  8334 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.215925  7654 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:41.216295  7843 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:41.216511  8334 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:41.218716  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.219954  8452 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.220939  8452 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.221349  8487 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:41.223052  8487 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.990466  8376 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:40.878440  8377 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.033389  8383 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:41.264088  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 148017us
I20260709 13:03:41.264394  8233 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.265178  8233 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:41.019251  8402 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 13:03:41.271991  2291 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:41.272241  8402 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.276026  8452 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.276298  8377 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.268539  8007 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.252192  8487 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:41.280413  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52077.1us
I20260709 13:03:41.282444  8487 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:40.932600  8435 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.927s	user 0.013s	sys 0.000s
I20260709 13:03:40.821834  8449 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.936782  8450 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.285480  1632 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:41.286772  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:40.976100  8453 raft_consensus.cc:3055] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:40.943992  8458 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:40.793147  8429 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:40.970729  8463 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260709 13:03:40.980682  8464 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.285s	user 0.002s	sys 0.000s
W20260709 13:03:40.982154  8467 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:03:40.977964  8460 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:40.851944  8469 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:03:40.844074  8473 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.298306  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1171) took 1773 ms. Trace:
I20260709 13:03:41.291069  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:40.844738  8479 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:03:41.308894  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:41.287629  8450 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:41.309036  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54842.7us
I20260709 13:03:41.288295  8453 raft_consensus.cc:740] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:40.934724  8480 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:41.309491  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 505921us
I20260709 13:03:41.312312  8453 consensus_queue.cc:260] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:41.314625  8453 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.989205  8485 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:41.019194  8489 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:41.309031  8509 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.316143  8489 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.316937  8485 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.132328  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1515) took 1701 ms. Trace:
I20260709 13:03:41.284029  8435 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:41.319260  8489 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.009684  8481 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:03:41.318341  2292 rpcz_store.cc:276] 0709 13:03:39.430944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.431103 (+   159us) service_pool.cc:224] Handling call
0709 13:03:39.431385 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.431601 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:39.529417 (+ 97816us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.855492 (+1326075us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b180045d300. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:03:41.130787 (+275295us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.131289 (+   502us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.131380 (+    91us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:41.131494 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.131638 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.131969 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":830415,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":1766,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":5049,"spinlock_wait_cycles":28651904,"thread_start_us":1160806,"threads_started":3,"wal-append.queue_time_us":886920}
I20260709 13:03:41.325234  8225 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:40.931164  8492 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:41.327128  8492 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.329835  8492 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.145299  7975 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.176031  1707 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:40.949970  2297 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:40.780014  7958 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:41.172222  1631 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:41.189756  8349 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:41.175433  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1118) took 1440 ms. Trace:
I20260709 13:03:41.179479  1634 rpcz_store.cc:276] 0709 13:03:39.401822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.401971 (+   149us) service_pool.cc:224] Handling call
0709 13:03:39.430956 (+ 28985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.316473 (+885517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.178259 (+861786us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b540014f5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:41.178684 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034772864}
I20260709 13:03:41.212028  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.212524  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:40.984614  8367 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:41.226780  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62140.1us
I20260709 13:03:41.163048  1705 rpcz_store.cc:276] 0709 13:03:38.025714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.207252 (+181538us) service_pool.cc:224] Handling call
0709 13:03:38.207579 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.281103 (+ 73524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.281144 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.293602 (+ 12458us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.737816 (+444214us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:38.765703 (+ 27887us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.765935 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.981636 (+215701us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b18003af660. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:39.667628 (+685992us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:39.667698 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.723146 (+1055448us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:40.813615 (+ 90469us) raft_consensus.cc:1685] finished
0709 13:03:40.813638 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.813781 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41422,"prepare.queue_time_us":50327,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":373,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":17834,"spinlock_wait_cycles":260232448,"thread_start_us":402755,"threads_started":1,"wal-append.queue_time_us":581904}
I20260709 13:03:41.219554  8334 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.203544  8231 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.216609  7654 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.224292  7823 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.254534  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:41.254853  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.257172  8376 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.331998  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:41.265375  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:40.846897  8425 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.318s	user 0.003s	sys 0.000s
I20260709 13:03:41.266677  1635 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:41.182818  2294 rpcz_store.cc:276] 0709 13:03:39.412185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.412318 (+   133us) service_pool.cc:224] Handling call
0709 13:03:39.412718 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.419549 (+  6831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.419586 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.420237 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.418468 (+998231us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b18003f3520. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:40.524587 (+106119us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:40.524636 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.524712 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.038789 (+514077us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:41.181556 (+142767us) raft_consensus.cc:1685] finished
0709 13:03:41.181586 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.181901 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16219,"raft.queue_time_us":165,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":7487,"spinlock_wait_cycles":52034688,"thread_start_us":105914,"threads_started":1,"wal-append.queue_time_us":1257687}
W20260709 13:03:41.270563  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 56588.1us
I20260709 13:03:41.270790  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.271031  7959 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.279619  8483 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:41.284016  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:40.869999  8471 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.287755  1632 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:40.792855  8478 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.303634  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:41.388885  8478 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:41.389092  1702 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:41.389983  1628 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:41.391296  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.291728  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:41.306466  1559 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:41.392215  8507 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:41.392241  1555 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:41.393639  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.395063  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:41.298852  1709 rpcz_store.cc:276] 0709 13:03:39.525122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.525330 (+   208us) service_pool.cc:224] Handling call
0709 13:03:39.528799 (+  3469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.859268 (+330469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.859325 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.992012 (+132687us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.479240 (+487228us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:40.505575 (+ 26335us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.505842 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.505940 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.969895 (+463955us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:41.029208 (+ 59313us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b18003538c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:41.292857 (+263649us) raft_consensus.cc:1685] finished
0709 13:03:41.292911 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.298084 (+  5173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22348,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":5423,"raft.queue_time_us":161917,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":183,"spinlock_wait_cycles":114530560,"thread_start_us":728082,"threads_started":2,"wal-append.queue_time_us":536201}
I20260709 13:03:41.314342  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:41.292569  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2977) took 1203 ms. Trace:
I20260709 13:03:41.132468  8031 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:41.399070  8031 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.133912  8169 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:41.374156  8509 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:41.378275  8483 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:41.333477  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.338965  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.314740  8211 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:41.383598  7959 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.396862  1557 rpcz_store.cc:276] 0709 13:03:40.089131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.089299 (+   168us) service_pool.cc:224] Handling call
0709 13:03:40.089659 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.090057 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.090093 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.090662 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.447620 (+356958us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:40.447702 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.448507 (+   805us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.979957 (+531450us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:41.291903 (+311946us) raft_consensus.cc:1685] finished
0709 13:03:41.291954 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.292354 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87702,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9801,"spinlock_wait_cycles":100182400,"thread_start_us":313907,"threads_started":1,"wal-append.queue_time_us":604251}
I20260709 13:03:41.400594  8471 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.336411  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:41.347812  1633 rpcz_store.cc:276] 0709 13:03:39.735008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.132698 (+397690us) service_pool.cc:224] Handling call
0709 13:03:40.185141 (+ 52443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:40.490574 (+305433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.174631 (+684057us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54000cd0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:41.175166 (+   535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580751616}
I20260709 13:03:41.364476  2296 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:03:41.331501  8295 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:41.331609  2297 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:41.332190  2293 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:41.350308  7654 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:41.337095  8378 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.338708  8396 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.372550  1703 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:41.368096  1629 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:03:41.428546  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.429661  1629 raft_consensus.cc:2410] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:03:41.377686  2295 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:41.430611  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:41.395391  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.423480  8295 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:41.406230  2292 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:41.431533  1705 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:41.431969  8269 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.370023  8349 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:41.432641  8269 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.432719  8295 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.433954  8349 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.410696  8225 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:41.435201  8225 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.435591  8225 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:41.398252  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.436586  1633 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:41.436903  1709 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:41.410183  8334 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:41.382994  8377 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.410959  8456 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:03:41.415184  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.439414  2297 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.440723  2297 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:41.405588  8474 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.784s	user 0.002s	sys 0.001s
I20260709 13:03:41.444911  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 2979) took 1352 ms. Trace:
I20260709 13:03:41.445318  1553 rpcz_store.cc:276] 0709 13:03:40.092889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.093393 (+   504us) service_pool.cc:224] Handling call
0709 13:03:40.093658 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.396672 (+303014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.396719 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.400994 (+  4275us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.584293 (+183299us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:40.584345 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.584433 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.404555 (+820122us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:41.444478 (+ 39923us) raft_consensus.cc:1685] finished
0709 13:03:41.444500 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.444721 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90920,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":42556,"spinlock_wait_cycles":272055040,"thread_start_us":484760,"threads_started":2,"wal-append.queue_time_us":219406}
W20260709 13:03:41.410389  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134744us
I20260709 13:03:41.406117  1630 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:41.447585  2289 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.449201  8471 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.404755  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:41.451730  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47157.9us
I20260709 13:03:41.452337  6983 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:41.409936  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 95799.8us
I20260709 13:03:41.455986  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:41.456185  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.401609  8031 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.404372  8483 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.459458  8483 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:41.460170  8269 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.408636  1707 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.394958  8367 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.403467  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.402248  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.434501  8295 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:41.403537  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.447623  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.409035  2288 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:03:41.409340  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 144189us
I20260709 13:03:41.392819  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:41.411113  8501 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.218s	user 0.003s	sys 0.000s
I20260709 13:03:41.473162  1702 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:41.478035  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:41.457324  8470 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:03:41.403234  8509 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.489367  1554 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:41.491892  8440 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:41.491725  8405 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:41.492463  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.492951  7823 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:03:41.445335  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2539) took 1858 ms (client timeout 2156 ms). Trace:
I20260709 13:03:41.494352  1635 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:03:41.496815  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.505617  1555 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:41.487188  7654 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.410387  7959 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.516991  1705 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:41.526646  7842 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.531615  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 99881us
W20260709 13:03:41.516258  1706 rpcz_store.cc:269] 0709 13:03:39.586474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.591465 (+  4991us) service_pool.cc:224] Handling call
0709 13:03:41.444891 (+1853426us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54003af7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.445141 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599425792}
I20260709 13:03:41.618882  7654 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.534654  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:41.469947  8176 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:41.532732  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.526515  1626 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:41.532681  8264 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:41.533701  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86237us
I20260709 13:03:41.622318  8031 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:41.623731  8031 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.623979  2297 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:03:41.624362  8031 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.624567  2297 pending_rounds.cc:85] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 11
I20260709 13:03:41.625064  2297 pending_rounds.cc:107] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:03:41.625453  2297 raft_consensus.cc:2889] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:41.626457  8375 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:03:41.627239  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1519) took 1309 ms. Trace:
I20260709 13:03:41.627633  2297 rpcz_store.cc:276] 0709 13:03:40.317447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.318619 (+  1172us) service_pool.cc:224] Handling call
0709 13:03:40.319001 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.949846 (+630845us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540040ca20. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:03:41.626057 (+676211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.627173 (+  1116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389135616}
W20260709 13:03:41.632092  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 176296us
I20260709 13:03:41.634271  8031 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.635159  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:41.635725  1555 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:41.636778  8031 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.640640  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.643924  8450 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.644598  8450 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.644904  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:41.647053  2289 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:41.647399  8211 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:41.648502  8264 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:41.649207  8211 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:41.649552  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 150633us
I20260709 13:03:41.650251  7994 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.650374  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.650835  7994 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:41.651244  7994 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.650944  8225 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.652127  7654 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:41.652570  7654 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.652928  7654 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:41.653980  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 133836us
W20260709 13:03:41.655982  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132388us
I20260709 13:03:41.657692  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.658217  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:41.658613  2297 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:41.660293  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.661302  1554 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:41.663407  8378 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.664069  8261 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.664395  8383 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:41.664765  8261 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.665091  8383 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.665537  8377 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.616562  8231 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:41.666106  8377 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.666978  8393 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.667218  8261 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.667923  8261 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.668294  2297 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:41.668454  8261 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.668936  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.669222  8383 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.664134  8378 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.672350  8378 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.668602  8377 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.674350  8458 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.674068  7842 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.675031  8261 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.674959  8458 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.678406  8458 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.676056  8449 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.679232  8449 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.680243  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1514) took 2262 ms. Trace:
I20260709 13:03:41.680663  2288 rpcz_store.cc:276] 0709 13:03:39.418243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.421208 (+  2965us) service_pool.cc:224] Handling call
0709 13:03:39.421507 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.497741 (+2076234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.679943 (+182202us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:41.680132 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4569140608}
I20260709 13:03:41.681396  8449 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.681610  8285 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.682266  8231 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:41.682147  8285 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:41.632151  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 95321.1us
W20260709 13:03:41.684671  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.649667  8211 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:41.683486  8231 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.693041  7654 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.695147  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:41.695896  1707 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:41.684641  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:41.692166  1632 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:41.690261  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.646951  8450 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.653606  8225 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.681401  8176 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:41.709415  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86584.8us
I20260709 13:03:41.718050  7654 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:41.712566  8451 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:41.713311  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.714083  8398 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.714829  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.716081  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:41.716691  1634 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:03:41.667661  8393 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.717752  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.711974  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.717480  8478 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.726109  2291 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:41.706602  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.719564  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:41.720140  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.721728  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.728380  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.723168  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:41.711122  8457 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.076s	user 0.004s	sys 0.000s
I20260709 13:03:41.731053  2288 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:41.732270  2289 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:41.655993  8264 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:03:41.703238  8453 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:41.724125  8470 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.733623  8398 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:41.788784  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 104253us
I20260709 13:03:41.707895  8031 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:03:41.840484  1632 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:41.789983  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.841428  2297 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:41.726620  8483 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.792232  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 157205us
I20260709 13:03:41.793967  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.734115  8458 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:41.790983  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132897us
W20260709 13:03:41.794663  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58221.4us
W20260709 13:03:41.795744  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 72661.7us
I20260709 13:03:41.799038  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.800693  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.844434  1634 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
W20260709 13:03:41.804766  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1507) took 2775 ms (client timeout 3172 ms). Trace:
I20260709 13:03:41.804622  8211 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:41.805516  2290 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:41.810200  1706 raft_consensus.cc:2410] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:03:41.811480  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:41.817152  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1122) took 2081 ms (client timeout 2674 ms). Trace:
I20260709 13:03:41.821516  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2559) took 1303 ms. Trace:
I20260709 13:03:41.823192  2291 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:41.848555  2295 raft_consensus.cc:2410] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:03:41.819742  1701 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.006s
I20260709 13:03:41.849189  1701 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:41.848619  1705 rpcz_store.cc:276] 0709 13:03:40.517644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.432394 (+914750us) service_pool.cc:224] Handling call
0709 13:03:41.504317 (+ 71923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.541668 (+ 37351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.821311 (+279643us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b54003c1250. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.821422 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151160064}
W20260709 13:03:41.845346  2293 rpcz_store.cc:269] 0709 13:03:39.029158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.032014 (+  2856us) service_pool.cc:224] Handling call
0709 13:03:41.804461 (+2772447us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b540040b5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.804673 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5264597248}
W20260709 13:03:41.848014  1627 rpcz_store.cc:269] 0709 13:03:39.736091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.923519 (+187428us) service_pool.cc:224] Handling call
0709 13:03:41.816676 (+1893157us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54001432d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:41.816954 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435821312}
I20260709 13:03:41.847074  8458 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.857151  8458 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:41.793771  8440 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:41.854655  2294 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:41.862084  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.864108  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.847069  8225 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.879873  1634 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:41.880707  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:41.880949  1525 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:41.881831  1525 leader_election.cc:403] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.883000  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.882722  8031 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:41.879690  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:41.888499  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.891091  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:41.892025  2288 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:41.893515  1525 leader_election.cc:400] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:41.894006  1525 leader_election.cc:403] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.897181  1627 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.867012  8440 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:41.870471  8439 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.908623  1632 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:41.896880  8451 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.926699  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:41.893682  8225 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:41.869437  8487 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 13:03:41.870843  8483 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.940024  2294 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:03:41.943703  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56547.6us
I20260709 13:03:41.883610  8470 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.911080  8061 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:41.932883  8451 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:41.905085  8334 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:03:41.965740  1631 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:41.967933  2295 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:41.935539  8031 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.977829  8285 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.978242  8264 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:41.971503  8451 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.979858  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.980548  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.980815  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:41.981662  8521 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:41.983472  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:41.983949  8478 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:41.984375  8437 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:41.985201  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.985287  8393 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:41.985694  8439 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:41.988314  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:41.989351  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:42.045343  2289 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:41.990895  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:41.954147  8061 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.973749  8169 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:03:41.999665  8264 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.000658  8459 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:42.056403  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 179190us
W20260709 13:03:42.056764  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 84985.7us
W20260709 13:03:42.064667  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 89726us
I20260709 13:03:42.056322  1558 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:42.072568  2294 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
W20260709 13:03:42.067184  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 173638us
I20260709 13:03:42.080323  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:42.075201  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:42.081442  1553 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:42.063817  8061 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.101027  2296 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:42.103113  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:42.066437  8264 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.105769  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:42.105275  1552 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:42.107352  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.117018  2295 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:41.970484  8440 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.066661  8459 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:42.015785  7994 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.123569  2262 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:42.123997  2262 leader_election.cc:403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:42.119951  8456 raft_consensus.cc:2764] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:03:41.992313  8398 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.135032  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:42.108348  8393 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:42.129997  1601 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.139304  2294 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:42.056808  8225 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.147513  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.150568  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:42.146953  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:42.128304  8285 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.161329  1673 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:42.162036  8483 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:42.162698  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:42.178443  1709 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:42.141641  8393 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.188567  8443 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:03:42.130332  8478 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:03:42.163632  8450 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:42.203826  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:42.125720  8295 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:03:42.212601  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.216226  8479 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.217044  8469 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:42.191125  8393 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:42.247169  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 326967us
I20260709 13:03:42.206703  8440 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.234755  8450 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:42.260937  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 320028us
I20260709 13:03:42.211589  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:42.269629  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 126287us
I20260709 13:03:42.244127  8470 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:42.268381  2294 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:42.208141  8478 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:42.253247  8405 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.203126  8285 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:42.301810  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 128899us
I20260709 13:03:42.325783  2297 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:42.327536  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:42.341173  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:42.347114  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:42.353705  8285 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.354705  8450 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.311331  8451 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.353971  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3027) took 1067 ms. Trace:
I20260709 13:03:42.360747  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:42.364171  1701 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.365286  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:42.366887  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.373653  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:42.354727  8285 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:42.388563  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:42.363441  1629 rpcz_store.cc:276] 0709 13:03:41.286116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.431696 (+145580us) service_pool.cc:224] Handling call
0709 13:03:41.805974 (+374278us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400140428. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3988ebc 00007f70d39be08c 00007f70d39a893b 00007f70d39a83d2 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c
0709 13:03:41.806104 (+   130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.806724 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.806754 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.807247 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.992663 (+185416us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:41.992706 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.992776 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.353594 (+360818us) raft_consensus.cc:1685] finished
0709 13:03:42.353623 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.353883 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4083,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"spinlock_wait_cycles":29486336,"thread_start_us":185041,"threads_started":1,"wal-append.queue_time_us":476114}
I20260709 13:03:42.341187  8458 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.374698  7994 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:42.314596  8459 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.405685  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:42.392035  8525 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
W20260709 13:03:42.407073  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 62191.8us
W20260709 13:03:42.420915  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.425483  1704 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:03:42.433213  8458 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:42.440784  8514 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.729s	user 0.004s	sys 0.001s
I20260709 13:03:42.443274  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:42.444255  8211 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.445574  7994 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:42.448511  7842 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.450176  8261 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:42.450639  8261 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:42.456687  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:42.405983  8231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.020s	sys 0.067s
I20260709 13:03:42.390031  8439 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:42.475611  2288 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.001s
I20260709 13:03:42.477775  2288 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:42.456424  8334 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:42.462291  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:42.480443  1632 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:42.464061  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.481743  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.464587  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.482483  1708 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
W20260709 13:03:42.482757  8520 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.687s	user 0.001s	sys 0.001s
I20260709 13:03:42.483084  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.460984  1701 raft_consensus.cc:2468] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:42.478654  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:42.482854  1708 pending_rounds.cc:85] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 5
I20260709 13:03:42.485224  1708 pending_rounds.cc:107] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:42.485576  1708 raft_consensus.cc:2889] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:42.466114  8231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.020s	sys 0.067s
I20260709 13:03:42.484181  1632 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:42.492453  8533 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:42.504551  8533 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.490275  8367 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:42.517190  1628 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.818s	user 0.010s	sys 0.006s
I20260709 13:03:42.517104  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.517767  1628 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:42.515082  8061 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.521670  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.517722  8031 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:42.538609  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:42.539542  1629 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.507615  8533 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.545665  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.520160  8459 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.589645  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.595265  1632 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:42.605497  7787 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:42.606200  7787 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.606717  7787 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:42.607380  8518 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.823s	user 0.011s	sys 0.000s
I20260709 13:03:42.605590  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1108) took 1694 ms. Trace:
I20260709 13:03:42.613862  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1470) took 2574 ms. Trace:
I20260709 13:03:42.614418  2290 rpcz_store.cc:276] 0709 13:03:40.039152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.039289 (+   137us) service_pool.cc:224] Handling call
0709 13:03:41.804746 (+1765457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.847403 (+ 42657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.613571 (+766168us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b540040b5d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:42.613774 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4059120512}
I20260709 13:03:42.615945  8211 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:42.616472  8211 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.616892  8211 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:42.620762  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 3019) took 1455 ms. Trace:
I20260709 13:03:42.622823  1633 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:03:42.626822  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:42.621338  1626 rpcz_store.cc:276] 0709 13:03:41.165513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.194969 (+ 29456us) service_pool.cc:224] Handling call
0709 13:03:42.620427 (+1425458us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400296dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:42.620670 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721995520}
W20260709 13:03:42.641285  1627 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.731s	user 0.016s	sys 0.000s
I20260709 13:03:42.641927  1627 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:42.645664  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.646001  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:42.646066  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 490174us
W20260709 13:03:42.645294  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 542238us
I20260709 13:03:42.611304  1551 rpcz_store.cc:276] 0709 13:03:40.790015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.105973 (+315958us) service_pool.cc:224] Handling call
0709 13:03:41.410691 (+304718us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b5800002a98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:03:41.410994 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.411559 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.411594 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.441671 (+ 30077us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.497569 (+ 55898us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:41.497748 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.497890 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.482120 (+984230us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:42.484002 (+  1882us) raft_consensus.cc:1685] finished
0709 13:03:42.484025 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.484149 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5195,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"spinlock_wait_cycles":160802432,"thread_start_us":55480,"threads_started":1,"wal-append.queue_time_us":280315}
I20260709 13:03:42.620949  8169 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.650743  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.653985  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:42.633960  8451 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:42.645757  8349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.152s	user 0.016s	sys 0.030s
I20260709 13:03:42.643079  8031 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:03:42.671557  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 324534us
W20260709 13:03:42.657393  8349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.016s	sys 0.030s
I20260709 13:03:42.648084  8536 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.673297  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.655054  8169 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:42.670810  8031 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:42.681531  8169 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.670657  8231 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:42.672708  8487 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:42.690136  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 536721us
I20260709 13:03:42.693706  8535 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.694063  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:42.694579  8535 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.696043  8439 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:42.697297  8469 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:42.697875  8469 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.698292  8469 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:42.676213  8453 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.700160  8453 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.708346  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:42.681936  8443 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:42.686359  8031 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:42.688194  8528 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:03:42.683947  8437 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.703002  8453 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.680325  8451 consensus_queue.cc:1059] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:03:42.721922  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:42.720701  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.713831  8439 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:42.715190  8487 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:42.713658  8231 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:42.736224  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.737509  8405 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:42.724193  8443 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:42.726248  8439 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:42.722955  8437 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.712414  8536 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.755693  8406 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:42.759052  8406 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.759477  8406 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:42.753227  8537 raft_consensus.cc:3055] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:42.753890  8439 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:42.751739  8061 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.753048  8285 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:42.775053  8513 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.085s	user 0.004s	sys 0.000s
I20260709 13:03:42.764887  8458 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.782015  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 125769us
W20260709 13:03:42.783484  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 144092us
I20260709 13:03:42.767832  8537 raft_consensus.cc:740] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:42.755473  8393 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:42.791296  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 76922.8us
I20260709 13:03:42.763512  8450 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:03:42.825067  8443 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.826674  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:42.830605  8393 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:42.833474  8440 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.001s
I20260709 13:03:42.836694  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:42.837900  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:42.838940  8405 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.839190  8349 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:42.840109  8538 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:42.840426  8295 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:42.839524  8405 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:42.859575  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.858225  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:42.843195  8176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.008s	sys 0.004s
I20260709 13:03:42.790663  8458 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.871824  1632 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:42.867110  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:42.873469  1634 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:42.903334  2293 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:42.910363  1626 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:42.858603  8405 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.911919  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:42.869320  8176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.009s	sys 0.004s
I20260709 13:03:42.865173  8540 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:42.788652  8231 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.867889  8535 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.929904  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:42.929318  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:42.931741  8539 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:42.931934  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:42.932166  2290 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:42.932307  8539 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.932651  1554 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:42.933075  8458 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:42.932714  8539 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.933729  8169 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.937193  8536 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:42.913331  8537 consensus_queue.cc:260] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.938655  8537 raft_consensus.cc:3060] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.941772  8539 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:42.942798  8540 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:42.952327  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:42.953203  1559 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:42.911079  8459 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:42.931274  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 200913us
W20260709 13:03:42.929503  8501 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:42.823673  8487 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:42.977339  1629 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:42.989895  1598 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:42.992489  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:42.993191  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:42.945998  8295 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:42.954768  8437 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.017992  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:42.996381  8264 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.870s	user 0.004s	sys 0.022s
W20260709 13:03:43.032717  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.004604  8295 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:43.041414  1551 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:42.984169  8211 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:43.034860  8547 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:03:42.944608  8540 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.064917  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.016561  8459 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.037467  8539 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.067863  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.078569  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.079193  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.081559  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.077312  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.034379  8264 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.073560  8334 raft_consensus.cc:2804] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:43.073911  8509 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:43.022506  8561 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:03:43.101253  8560 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.102005  8560 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.081483  8393 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:43.087045  8551 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.100910  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.084709  8443 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.107707  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.109341  1701 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:43.109661  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:43.106628  8393 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.112082  1556 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:43.113978  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 1112) took 1593 ms. Trace:
I20260709 13:03:43.098397  8509 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.106108  8334 raft_consensus.cc:697] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:43.118076  1555 rpcz_store.cc:276] 0709 13:03:41.520492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.534318 (+ 13826us) service_pool.cc:224] Handling call
0709 13:03:43.109488 (+1575170us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54000e8b54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:43.109617 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36511616}
I20260709 13:03:43.107507  8375 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:43.110435  8561 consensus_queue.cc:1243] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 2579bae7bf1543bf9b10d68e93a6ebbb log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:03:43.129168  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 157840us
I20260709 13:03:43.139855  8553 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.141764  8553 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.115667  7842 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:43.140959  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:43.114398  7787 consensus_meta.cc:287] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.003s
I20260709 13:03:43.145920  8470 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:43.146477  8470 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:43.146059  7787 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.148347  8553 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.148556  7787 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.122979  8393 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:43.151911  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.152760  1551 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.159088  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.159142  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:43.160387  1629 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:43.160887  2295 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.165064  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:43.165674  2292 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.163834  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.115952  8483 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.168181  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:43.168989  1559 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:43.173388  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:43.124318  8509 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:43.104542  8295 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.172809  2291 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:43.195813  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.197867  1627 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:03:43.114209  8443 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.201496  8459 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:43.202068  8522 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.039s	user 0.003s	sys 0.000s
I20260709 13:03:43.203239  8211 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.203327  8264 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.156823  8544 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.204727  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.206302  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1178) took 1500 ms. Trace:
I20260709 13:03:43.207190  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:43.197299  8334 consensus_queue.cc:237] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.216039  1705 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:43.216023  1707 rpcz_store.cc:276] 0709 13:03:41.705515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.112300 (+1406785us) service_pool.cc:224] Handling call
0709 13:03:43.112659 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.125241 (+ 12582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.125291 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.127617 (+  2326us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.205963 (+ 78346us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:43.206008 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.206079 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.206135 (+    56us) raft_consensus.cc:1685] finished
0709 13:03:43.206146 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.206249 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7191,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3120,"thread_start_us":77854,"threads_started":1,"wal-append.queue_time_us":73563}
I20260709 13:03:43.217676  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:43.216264  1559 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:43.218122  1701 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:43.220103  1555 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:43.211615  7842 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:43.224591  7842 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.227406  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.123239  8551 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.147506  8470 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.254251  8574 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.255098  8574 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.218086  8295 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.173452  8483 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.262568  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.276813  1556 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:43.278780  1559 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:03:43.279803  1701 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:43.250588  8561 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.296452  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.296600  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.297138  1553 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:43.297462  1557 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.301834  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.302003  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.303278  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.304209  1556 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:43.304352  1707 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:43.305382  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.305666  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.305997  1707 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:43.306239  1556 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.306963  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.308259  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.312245  1556 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:43.313230  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.314116  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.315550  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3030) took 1696 ms. Trace:
I20260709 13:03:43.315915  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:43.316687  1559 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:43.318611  1556 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:43.324472  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.316035  1635 rpcz_store.cc:276] 0709 13:03:41.618993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.619165 (+   172us) service_pool.cc:224] Handling call
0709 13:03:41.632265 (+ 13100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.632636 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.632664 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.633253 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.871150 (+237897us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:41.871191 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.871261 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.201423 (+1330162us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:43.300904 (+ 99481us) raft_consensus.cc:1685] finished
0709 13:03:43.300938 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.315476 (+ 14538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34248960,"thread_start_us":237371,"threads_started":1,"wal-append.queue_time_us":465144}
I20260709 13:03:43.326402  1555 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:43.299577  8560 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.329164  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.229595  8544 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.336651  1705 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:43.351449  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2569) took 1217 ms. Trace:
W20260709 13:03:43.338708  8406 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.579s	user 0.003s	sys 0.007s
I20260709 13:03:43.338809  8459 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:43.352051  8578 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:43.314303  8470 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:43.351830  1705 rpcz_store.cc:276] 0709 13:03:42.134262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.204439 (+1070177us) service_pool.cc:224] Handling call
0709 13:03:43.351348 (+146909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.349574  8443 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.353591  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:43.354266  8558 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:03:43.354305  1705 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:43.355602  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.356184  1705 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.340065  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:43.341998  8031 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
W20260709 13:03:43.342689  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.034s	sys 0.042s
I20260709 13:03:43.353744  1555 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:43.361377  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.034s	sys 0.042s
I20260709 13:03:43.341127  1559 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:43.368793  1553 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:03:43.344914  8439 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.001s
I20260709 13:03:43.358855  1557 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:43.352468  8459 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:43.376850  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 971349us
I20260709 13:03:43.327824  8574 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.352577  8578 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.382788  8539 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:43.382944  8517 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.685s	user 0.004s	sys 0.007s
I20260709 13:03:43.383163  1556 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:43.376597  8459 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.389735  8261 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:43.391614  1674 connection.cc:256] RPC call timeout handler was delayed by -1.4914159100000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 13:03:43.358703  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 194980us
I20260709 13:03:43.394058  8533 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.433504  8533 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.434953  1635 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:03:43.435396  1674 connection.cc:256] RPC call timeout handler was delayed by -1.121337388000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:03:43.395800  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68199.4us
I20260709 13:03:43.435698  8285 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:43.436522  8285 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:43.397429  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3121) took 2058 ms. Trace:
I20260709 13:03:43.437119  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:43.437079  1633 rpcz_store.cc:276] 0709 13:03:41.339355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.437702 (+ 98347us) service_pool.cc:224] Handling call
0709 13:03:42.622482 (+1184780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.629923 (+  7441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.397116 (+767193us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400296dd0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:43.397343 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2723642240}
I20260709 13:03:43.438503  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:43.438723  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:43.439358  1627 raft_consensus.cc:2376] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23. Re-sending same reply.
I20260709 13:03:43.420382  8539 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:43.445060  8539 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.446187  8539 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:43.400177  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.420917  8578 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:43.401840  8551 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.405664  8483 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.352481  8406 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.409466  8544 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.411512  8393 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.414523  8405 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.420915  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.426242  6983 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.398941  8469 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
I20260709 13:03:43.371445  8031 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.298974  8561 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.381922  1705 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:43.500774  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1177) took 1800 ms (client timeout 2309 ms). Trace:
I20260709 13:03:43.375617  8439 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.459703  1551 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:03:43.501262  1706 rpcz_store.cc:269] 0709 13:03:41.700469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.703282 (+  2813us) service_pool.cc:224] Handling call
0709 13:03:43.469017 (+1765735us) spinlock_profiling.cc:243] Waited 87.1 ms on lock 0x7b5400042c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:43.500692 (+ 31675us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200400128}
I20260709 13:03:43.452917  1555 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:43.508750  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.461302  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:43.511680  6983 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.507839  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.513695  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:43.517334  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78896.7us
I20260709 13:03:43.518273  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:43.522194  1707 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:43.522337  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:43.524360  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.526952  1705 raft_consensus.cc:2376] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23. Re-sending same reply.
I20260709 13:03:43.462816  8211 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:43.509131  8540 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.529546  1560 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:43.530012  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.501487  8533 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.537385  2297 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:03:43.530224  2292 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:43.531491  1703 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:43.462718  8579 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.468390  8375 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:43.469290  7994 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:43.521476  8334 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:43.513862  8551 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:43.527061  8261 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.547844  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.553651  8393 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.556046  8334 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:43.558715  8551 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.560397  6983 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.560884  8575 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.562289  8405 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.536005  8285 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:43.581621  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:43.570765  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 109584us
I20260709 13:03:43.514639  8544 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:03:43.510160  8469 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.583979  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:43.585645  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67497.7us
I20260709 13:03:43.588336  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:43.589325  2290 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:43.591269  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:43.593050  1701 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:43.592473  8483 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:43.599529  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:43.582296  1706 raft_consensus.cc:2376] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 17. Re-sending same reply.
I20260709 13:03:43.568511  8535 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:43.547065  7994 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.604887  8285 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:43.611860  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.578120  8560 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.546548  8540 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.613181  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:43.615132  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:43.616291  2290 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:43.617565  8564 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:43.622517  8509 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.623904  2288 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:03:43.624011  8569 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.625530  8031 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.546636  8579 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:43.626448  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.616925  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:43.604758  8536 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:43.609136  7994 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.603981  8535 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:43.643918  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:43.641111  1601 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:43.613819  8231 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:43.648759  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.649610  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.628841  8483 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:43.649952  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:43.653264  8334 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.640811  8031 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:43.580314  8575 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.681664  2290 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:43.655939  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:43.684937  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:43.685715  2292 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:43.642889  8564 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:43.695398  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:43.689908  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 95714us
I20260709 13:03:43.648454  8535 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:43.649411  8565 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:03:43.666720  8537 consensus_meta.cc:287] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.728s	user 0.006s	sys 0.005s
I20260709 13:03:43.695098  8264 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:43.748576  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.660245  8061 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:43.668107  8398 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:43.723223  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:43.642437  8569 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:03:43.654825  8450 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 13:03:43.654402  8483 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:43.752398  1706 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:43.673831  8469 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.757201  1552 raft_consensus.cc:2410] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:43.642601  8560 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:43.675019  8509 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.723207  8564 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.639696  8285 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.677462  8031 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:43.865285  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 178980us
I20260709 13:03:43.867043  8031 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.748028  8406 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.653928  8375 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.766913  8561 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.768090  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.763612  8398 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.772634  8450 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.775113  8560 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.765069  8483 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:43.781201  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.895730  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:43.896327  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.897161  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.780793  8578 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.781791  8456 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:43.782207  8540 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.783819  8579 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:43.785669  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.905411  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.906154  1704 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:43.906942  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:43.788755  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.760476  8544 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:43.868940  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 103056us
I20260709 13:03:43.785679  7994 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:43.869992  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:43.916240  1553 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:43.771238  8459 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:43.876453  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 118106us
I20260709 13:03:43.756476  8565 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:43.877144  2292 raft_consensus.cc:1217] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Deduplicated request from leader. Original: 13.13->[16.14-17.15]   Dedup: 16.14->[17.15-17.15]
I20260709 13:03:43.919632  2292 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: [16.14-17.15]
I20260709 13:03:43.884239  1706 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
I20260709 13:03:43.890851  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:43.787245  8537 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.869376  8564 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.896483  1558 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:43.898507  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:43.901837  1551 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:43.907666  1704 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:03:43.869724  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1128) took 2147 ms (client timeout 2309 ms). Trace:
W20260709 13:03:43.926396  1631 rpcz_store.cc:269] 0709 13:03:41.722613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.862032 (+139419us) service_pool.cc:224] Handling call
0709 13:03:43.869558 (+2007526us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:43.869664 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193354112}
W20260709 13:03:43.927186  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 204097us
I20260709 13:03:43.902988  8456 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:43.929741  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:43.933142  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.931531  1599 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:43.927907  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:43.943799  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:43.919548  8551 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:43.948001  1524 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:43.922025  8061 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:43.950033  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:43.934376  8264 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:43.929709  8540 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:43.942811  8589 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:43.955034  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:43.965304  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:44.057590  2289 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:43.967885  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:43.974032  8596 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:43.975152  8565 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.065135  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.065675  1701 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:43.930706  8537 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:43.981597  8398 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:43.982280  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53874.9us
I20260709 13:03:43.933092  8470 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:03:43.953460  8450 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:43.989204  8459 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.990486  8536 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:43.940274  8405 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:43.992096  8538 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:43.993366  8456 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.088686  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.089561  1628 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:44.090063  1701 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:43.998849  7994 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:43.949390  8031 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:43.915226  8544 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.003350  8551 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:44.095247  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.092621  8031 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:44.096131  1705 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.096469  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.096593  8536 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:44.097079  1628 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:44.097726  1634 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:44.003865  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.004791  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:44.099547  2292 raft_consensus.cc:2368] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:44.012951  8439 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:44.031080  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.037972  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2551) took 3523 ms. Trace:
I20260709 13:03:44.043258  8169 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:44.102279  1708 rpcz_store.cc:276] 0709 13:03:40.514703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.514840 (+   137us) service_pool.cc:224] Handling call
0709 13:03:40.515112 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.492589 (+1977477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.037615 (+1545026us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54001b6a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:44.037876 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4509972352}
I20260709 13:03:44.027245  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.106125  1707 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:44.046159  8564 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.052125  8535 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.053479  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.108162  1703 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 16. Current term is 17.
I20260709 13:03:44.057930  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:44.030640  1553 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:44.061311  1627 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
W20260709 13:03:44.063212  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.374435   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:44.057148  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1267) took 1182 ms. Trace:
I20260709 13:03:44.070278  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:44.070936  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.072751  1632 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 13:03:44.072990  1633 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:44.072748  1631 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:44.074509  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 119613us
I20260709 13:03:44.078946  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1259) took 2190 ms. Trace:
I20260709 13:03:44.056409  8509 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.057469  8540 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.081293  1630 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:44.068320  8334 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.092686  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.091334  8538 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:44.095067  8551 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.097011  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.005666  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.009598  8578 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:44.007453  8575 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:44.111554  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.149020  1701 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:44.147990  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:44.126577  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 68723.4us
I20260709 13:03:44.149981  1707 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:44.150364  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2587) took 1033 ms. Trace:
I20260709 13:03:44.150760  1707 rpcz_store.cc:276] 0709 13:03:43.117057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.112257 (+995200us) service_pool.cc:224] Handling call
0709 13:03:44.150311 (+ 38054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.110502  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.096206  8031 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.127815  1558 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:44.125769  2293 rpcz_store.cc:276] 0709 13:03:41.888065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.888234 (+   169us) service_pool.cc:224] Handling call
0709 13:03:41.923111 (+ 34877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.904157 (+981046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.078595 (+1174438us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54004a6354. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:44.078852 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253560064}
W20260709 13:03:44.138520  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.109861  2294 rpcz_store.cc:276] 0709 13:03:42.875032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.875206 (+   174us) service_pool.cc:224] Handling call
0709 13:03:42.924728 (+ 49522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.983386 (+ 58658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.983432 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.984036 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.105868 (+121832us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:43.209796 (+103928us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.304943 (+ 95147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.305019 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.047677 (+742658us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b1800428440. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:44.054427 (+  6750us) raft_consensus.cc:1685] finished
0709 13:03:44.056941 (+  2514us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.057088 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45532,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":207880,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":52248,"spinlock_wait_cycles":5751168,"thread_start_us":254966,"threads_started":3,"wal-append.queue_time_us":195624}
I20260709 13:03:44.140373  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:44.228894  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 168993us
I20260709 13:03:44.110376  8453 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:44.147943  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.232273  1709 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:44.108863  8405 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:44.226387  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2588) took 1082 ms. Trace:
I20260709 13:03:44.235795  1705 rpcz_store.cc:276] 0709 13:03:43.144079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.145908 (+1001829us) service_pool.cc:224] Handling call
0709 13:03:44.148112 (+  2204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.225892 (+ 77780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.225945 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.226009 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:44.226045 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.226122 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.226301 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.229950  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1183) took 1035 ms. Trace:
I20260709 13:03:44.141601  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:44.226186  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 75925.8us
I20260709 13:03:44.238304  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:44.137373  8540 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.136266  7994 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:44.242990  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 172806us
I20260709 13:03:44.237164  1704 rpcz_store.cc:276] 0709 13:03:43.194633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.108780 (+914147us) service_pool.cc:224] Handling call
0709 13:03:44.229889 (+121109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.160046  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.246140  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.247149  1708 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:44.249889  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:44.253856  1702 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:44.254354  1702 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:44.251900  2260 connection.cc:256] RPC call timeout handler was delayed by -1.0940928369999483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:03:44.255060  2260 connection.cc:256] RPC call timeout handler was delayed by -1.0490319729999555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1675s for a response
I20260709 13:03:44.260934  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2589) took 1098 ms. Trace:
I20260709 13:03:44.267630  1706 rpcz_store.cc:276] 0709 13:03:43.162110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.147961 (+985851us) service_pool.cc:224] Handling call
0709 13:03:44.242233 (+ 94272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.260557 (+ 18324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.260595 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.260638 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:44.260670 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.260740 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.260872 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.130501  8483 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.261227  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:44.139981  8509 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.249779  7994 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.260888  8596 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:44.138172  8061 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.152209  8334 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:44.233284  8459 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:03:44.264700  8597 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:44.307322  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.272867  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2590) took 1107 ms. Trace:
I20260709 13:03:44.293684  1700 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:03:44.309422  1700 pending_rounds.cc:85] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 14
I20260709 13:03:44.309741  1700 pending_rounds.cc:107] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:03:44.310036  1700 raft_consensus.cc:2889] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.270664  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.270892  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:44.355666  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63916.9us
I20260709 13:03:44.356001  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.273197  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:44.356601  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51244.9us
I20260709 13:03:44.361832  1626 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:44.359719  1560 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [17.9-19.10]
I20260709 13:03:44.363306  1704 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:44.367056  1700 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:03:44.308384  1701 rpcz_store.cc:276] 0709 13:03:43.165344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.149723 (+984379us) service_pool.cc:224] Handling call
0709 13:03:44.248556 (+ 98833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.265858 (+ 17302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.265908 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.265969 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:44.272589 (+  6620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.272669 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.272794 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.292416  1702 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:44.379278  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 115416us
I20260709 13:03:44.379961  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:44.381682  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 108609us
I20260709 13:03:44.380298  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:44.394088  1702 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:44.231052  8264 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.393283  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:44.409725  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:44.413082  2293 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:44.383208  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:44.406400  1558 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:44.277904  8578 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.283980  7994 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:44.429666  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:44.270207  8458 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.140s
I20260709 13:03:44.419126  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.495630  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.379384  8487 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:44.553686  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:44.558995  1558 raft_consensus.cc:2393] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:44.559132  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.559966  1556 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:44.543076  1700 raft_consensus.cc:1240] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:03:44.564529  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3044) took 1110 ms. Trace:
I20260709 13:03:44.492409  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.412881  8375 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:44.588312  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.588331  1702 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:03:44.565085  1700 rpcz_store.cc:276] 0709 13:03:43.501733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.441286 (+939553us) service_pool.cc:224] Handling call
0709 13:03:44.542706 (+101420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:44.564110 (+ 21404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.564454 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.577953  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2511) took 1313 ms. Trace:
I20260709 13:03:44.407114  8604 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.466800  1708 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:44.514309  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.504174  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:44.589725  1702 pending_rounds.cc:85] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 11
W20260709 13:03:44.595522  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 202344us
I20260709 13:03:44.595969  1702 pending_rounds.cc:107] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:03:44.596415  1702 raft_consensus.cc:2889] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.416605  8597 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:44.591686  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.590066  1552 rpcz_store.cc:276] 0709 13:03:43.264835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.286437 (+ 21602us) service_pool.cc:224] Handling call
0709 13:03:44.569283 (+1282846us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540036c4d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:44.577858 (+  8575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014567808}
I20260709 13:03:44.593796  1633 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.497975  8537 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:44.554432  8596 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:44.598590  1558 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
W20260709 13:03:44.608901  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 228674us
I20260709 13:03:44.412143  8211 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.185s
I20260709 13:03:44.635310  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.605085  1554 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.374620  8538 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.587699  8535 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:44.439160  8176 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:44.622884  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.632339  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.632817  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:44.632452  1700 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:44.684839  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:44.442518  8578 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.600019  8564 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:44.483982  8285 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:44.693773  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.694406  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.694540  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:44.694986  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.695248  1552 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.647424  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2586) took 1536 ms. Trace:
I20260709 13:03:44.604200  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2596) took 1155 ms. Trace:
I20260709 13:03:44.649917  1631 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.588295  8375 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:44.603137  8595 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:44.593111  8487 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:44.620988  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 116718us
W20260709 13:03:44.711534  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 78844.5us
W20260709 13:03:44.650794  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 221213us
I20260709 13:03:44.686972  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2598) took 1233 ms. Trace:
I20260709 13:03:44.724553  1700 rpcz_store.cc:276] 0709 13:03:43.453904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.590115 (+1136211us) service_pool.cc:224] Handling call
0709 13:03:44.620431 (+ 30316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.686667 (+ 66236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.686905 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.686784  1556 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:44.696669  1708 rpcz_store.cc:276] 0709 13:03:43.448460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.397641 (+949181us) service_pool.cc:224] Handling call
0709 13:03:44.443930 (+ 46289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.594652 (+150722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.604123 (+  9471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.700379  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:44.703109  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.727483  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.730657  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.621233  8579 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:44.707450  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:44.731833  2260 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.507s (SENT)
I20260709 13:03:44.732067  8579 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.732424  1633 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.732584  8579 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.614295  8618 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:44.710005  1558 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: [15.12-16.13]
I20260709 13:03:44.714514  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.734390  1627 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:44.416791  8405 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.650398  8450 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:03:44.696444  1702 rpcz_store.cc:276] 0709 13:03:43.111304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.138495 (+1027191us) service_pool.cc:224] Handling call
0709 13:03:44.153110 (+ 14615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.647105 (+493995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.647348 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.688032  8544 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:44.714574  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.756253  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71496.2us
I20260709 13:03:44.756749  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.757102  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.758936  1628 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:44.772552  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:44.773715  8487 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:44.774123  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.710486  8538 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.762224  8509 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:44.764142  8483 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:44.693244  8285 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.765619  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:44.710579  8375 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.776925  2293 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:44.735615  8471 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:44.777160  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:44.818660  1629 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:44.774935  1628 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.777704  1558 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:44.725028  8601 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:03:44.710786  8470 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:44.772768  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:44.710927  8595 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:44.802615  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:44.832631  8603 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:03:44.832448  8595 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.834175  8595 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:44.834406  8605 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:03:44.802855  8176 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:44.720374  8609 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:44.736343  8551 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:44.852872  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 440436us
W20260709 13:03:44.817816  8600 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.532s	user 0.000s	sys 0.001s
I20260709 13:03:44.810957  1702 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:44.812417  1633 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:44.603650  8604 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:44.817865  1552 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:44.822664  1551 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:03:44.828200  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55542.3us
W20260709 13:03:44.828567  1445 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 444b494bb53842b59408a7230884398a (127.0.107.193:38031) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 444b494bb53842b59408a7230884398a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:44.828717  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:44.831125  8470 raft_consensus.cc:2764] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:44.840435  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:44.842403  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:44.785727  8544 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.803836  8334 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:44.851392  1709 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.001s
I20260709 13:03:44.808318  8285 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.898006  1704 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:44.882534  1709 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:44.890295  1559 raft_consensus.cc:2393] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:44.900552  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2593) took 1695 ms. Trace:
W20260709 13:03:44.891629  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 248899us
I20260709 13:03:44.901074  1709 rpcz_store.cc:276] 0709 13:03:43.204663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.112125 (+907462us) service_pool.cc:224] Handling call
0709 13:03:44.900463 (+788338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.902459  1709 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:03:44.733718  8618 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:44.906023  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.003s	sys 0.007s
W20260709 13:03:44.929540  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.003s	sys 0.007s
W20260709 13:03:44.932350  1523 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.660s (SENT)
W20260709 13:03:44.930051  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 668945us
W20260709 13:03:44.932461  2260 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.009s (SENT)
W20260709 13:03:44.908361  8602 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.466s	user 0.001s	sys 0.000s
I20260709 13:03:44.936388  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:44.939946  2291 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:03:44.937331  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1187) took 1637 ms. Trace:
W20260709 13:03:44.984476  8616 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:44.909650  8176 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.988551  8176 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:44.989277  8176 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:44.904871  8544 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:44.993037  8544 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:44.968919  1628 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:44.838701  8471 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.000578  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.002877  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:45.003413  8614 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:45.003947  1552 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:03:45.004906  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.005491  1552 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:45.000797  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1190) took 1644 ms. Trace:
I20260709 13:03:44.805531  8597 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:44.937844  8598 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.473s	user 0.006s	sys 0.000s
I20260709 13:03:45.009394  1560 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:45.012856  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.016506  1558 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.007635  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3043) took 1567 ms. Trace:
I20260709 13:03:44.906849  8594 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:03:45.017161  1701 rpcz_store.cc:276] 0709 13:03:43.439756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.368391 (+928635us) service_pool.cc:224] Handling call
0709 13:03:44.368832 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.369327 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.369387 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.369973 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.705397 (+335424us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:44.933494 (+228097us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.969986 (+ 36492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.972841 (+  2855us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.007435 (+ 34594us) raft_consensus.cc:1685] finished
0709 13:03:45.007454 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.007578 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36209,"prepare.queue_time_us":96877,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":30,"raft.queue_time_us":161,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":124,"thread_start_us":548031,"threads_started":2,"wal-append.queue_time_us":447070}
I20260709 13:03:45.016765  1552 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:03:45.021361  1552 pending_rounds.cc:85] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 8
I20260709 13:03:45.022456  1552 pending_rounds.cc:107] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:03:45.024545  1552 raft_consensus.cc:2889] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.025015  1552 pending_rounds.cc:107] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:03:45.025309  1552 raft_consensus.cc:2889] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:45.021610  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 113762us
I20260709 13:03:45.014212  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:45.008328  1706 rpcz_store.cc:276] 0709 13:03:43.355989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.269263 (+913274us) service_pool.cc:224] Handling call
0709 13:03:44.295678 (+ 26415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.360498 (+ 64820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.360534 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.372291 (+ 11757us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.652008 (+279717us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:44.652061 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.652148 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.000534 (+348386us) raft_consensus.cc:1685] finished
0709 13:03:45.000589 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.000747 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":599,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":20577,"thread_start_us":188178,"threads_started":1,"wal-append.queue_time_us":466994}
I20260709 13:03:45.012598  8564 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.065496  1558 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:45.066154  1523 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.720s (SENT)
I20260709 13:03:45.070225  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.099906  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.103225  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2530) took 1220 ms. Trace:
I20260709 13:03:45.103716  1555 rpcz_store.cc:276] 0709 13:03:43.882654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.896715 (+ 14061us) service_pool.cc:224] Handling call
0709 13:03:43.897047 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.914646 (+ 17599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.914692 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.923443 (+  8751us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.109324 (+185881us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:44.109382 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.109460 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.102964 (+993504us) raft_consensus.cc:1685] finished
0709 13:03:45.102988 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.103151 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5389,"raft.queue_time_us":14033,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6141,"thread_start_us":155068,"threads_started":1,"wal-append.queue_time_us":346431}
I20260709 13:03:45.120584  2293 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:03:45.122650  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 213467us
I20260709 13:03:45.123515  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.094802  8470 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:45.131724  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.132799  1551 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:45.123697  8031 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.870s	user 0.018s	sys 0.002s
I20260709 13:03:45.098208  8538 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.101778  8609 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:45.098076  8536 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:45.143713  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 113826us
I20260709 13:03:45.130616  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:45.121068  8578 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:45.092919  8564 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.153770  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.158393  1554 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:45.160014  1523 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.765s (SENT)
I20260709 13:03:45.160566  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:45.159369  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.135649  8470 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:45.162917  1555 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:45.161624  1554 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:45.166055  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:45.167737  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.168514  1702 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:45.150810  8609 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.173389  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.177974  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.174024  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2526) took 1306 ms. Trace:
I20260709 13:03:45.178558  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.181097  1551 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:45.182179  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.178217  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.178611  1557 rpcz_store.cc:276] 0709 13:03:43.867996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.868126 (+   130us) service_pool.cc:224] Handling call
0709 13:03:43.868514 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:43.879704 (+ 11190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.879734 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:43.894727 (+ 14993us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.105772 (+211045us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:44.105826 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.105897 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.173707 (+1067810us) raft_consensus.cc:1685] finished
0709 13:03:45.173748 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.173947 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7488,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4133,"thread_start_us":201160,"threads_started":1,"wal-append.queue_time_us":376049}
I20260709 13:03:45.183977  1551 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:45.175057  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.136036  8405 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:03:45.189515  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.194298  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.199519  8560 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:45.199674  8470 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:45.201028  8470 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.201256  8456 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.205459  8560 consensus_queue.cc:1243] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:45.206071  8560 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:45.206423  8560 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:45.207307  8560 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.157914  8643 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:45.208369  8629 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:03:45.179687  8609 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.209332  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.210634  8471 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.151507  8031 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.211880  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.212611  1706 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:45.212908  8031 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.213431  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.208652  8560 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:45.214288  8617 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.189354  8611 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.468s	user 0.001s	sys 0.000s
I20260709 13:03:45.217841  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2595) took 2006 ms. Trace:
I20260709 13:03:45.214576  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:45.217362  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.218613  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.194114  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.235625  1551 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
W20260709 13:03:45.241683  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 100318us
I20260709 13:03:45.245374  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:45.152983  8538 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:45.251436  1706 raft_consensus.cc:2368] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:45.224020  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.221159  1703 rpcz_store.cc:276] 0709 13:03:43.211552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.238282 (+1026730us) service_pool.cc:224] Handling call
0709 13:03:44.254236 (+ 15954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.264192 (+  9956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.264223 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.270620 (+  6397us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.409884 (+139264us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:44.409936 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.410092 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.217648 (+807556us) raft_consensus.cc:1685] finished
0709 13:03:45.217667 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.217785 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6426,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5126,"thread_start_us":131936,"threads_started":1,"wal-append.queue_time_us":324909}
I20260709 13:03:45.194715  1554 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:45.179317  8612 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.265946  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.268019  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.269634  1706 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:45.260656  1559 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:45.272892  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:45.268141  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.208387  8643 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.274535  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:45.272360  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.324831  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.325639  1559 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:45.326501  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.327240  1559 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:45.216637  8618 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:45.298843  1709 raft_consensus.cc:1275] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:03:45.329080  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:45.329723  1559 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:03:45.336361  1554 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:45.300011  1560 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:03:45.300647  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 149818us
W20260709 13:03:45.341344  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.301271  8564 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.304111  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:45.312109  7994 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.692s	user 0.002s	sys 0.009s
I20260709 13:03:45.316139  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3047) took 1751 ms. Trace:
I20260709 13:03:45.316383  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.317549  8648 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:45.317911  8375 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.279932  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:45.346277  1708 rpcz_store.cc:276] 0709 13:03:43.564690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.725887 (+1161197us) service_pool.cc:224] Handling call
0709 13:03:44.776672 (+ 50785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.792672 (+ 16000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.792701 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.820256 (+ 27555us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.080995 (+260739us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:45.081036 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.081113 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.315834 (+234721us) raft_consensus.cc:1685] finished
0709 13:03:45.315863 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.316057 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":148217,"threads_started":1,"wal-append.queue_time_us":428079}
W20260709 13:03:45.337791  8606 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.899s	user 0.002s	sys 0.000s
I20260709 13:03:45.190080  8640 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:03:45.350096  1551 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:45.352303  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 266770us
I20260709 13:03:45.214797  8617 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:45.352770  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 338678us
W20260709 13:03:45.354952  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.017s	sys 0.013s
W20260709 13:03:45.355473  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.017s	sys 0.013s
I20260709 13:03:45.364419  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2599) took 1850 ms. Trace:
I20260709 13:03:45.365007  1700 rpcz_store.cc:276] 0709 13:03:43.514234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.739348 (+1225114us) service_pool.cc:224] Handling call
0709 13:03:44.773382 (+ 34034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.836702 (+ 63320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.836751 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.836828 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:44.836874 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.364060 (+527186us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:45.364136 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.364333 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145053568}
I20260709 13:03:45.366966  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.369439  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.373203  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.296875  8643 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:45.351023  8565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.246s	user 0.016s	sys 0.012s
W20260709 13:03:45.355041  8540 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.083s	user 0.012s	sys 0.000s
W20260709 13:03:45.396960  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.400566  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.402112  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.375222  1700 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:45.345400  8604 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:45.368633  1523 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.873s (SENT)
W20260709 13:03:45.402254  1675 leader_election.cc:336] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.788s (SENT)
I20260709 13:03:45.404217  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:45.404363  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.377712  1554 raft_consensus.cc:2410] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 25 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:03:45.405509  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.406479  1551 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:45.409227  8539 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.413327  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.403179  1702 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
W20260709 13:03:45.397080  8565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.017s	sys 0.013s
I20260709 13:03:45.414285  1551 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:45.418604  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:44.941486  1707 rpcz_store.cc:276] 0709 13:03:43.300009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.256784 (+956775us) service_pool.cc:224] Handling call
0709 13:03:44.278469 (+ 21685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.287027 (+  8558us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:44.562222 (+275195us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.934778 (+372556us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.935004 (+   226us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:44.937061 (+  2057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.937134 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.937268 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":503443,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":34050,"thread_start_us":506933,"threads_started":2}
I20260709 13:03:45.417115  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:45.362448  2292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.023s	sys 0.024s
W20260709 13:03:45.420449  2292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.025s	sys 0.029s
I20260709 13:03:45.415796  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:45.419979  1554 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:03:45.419293  1708 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:45.349810  7994 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.427603  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.443800  8437 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:45.432569  1700 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:03:45.445052  1557 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:45.450194  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3132) took 1388 ms. Trace:
I20260709 13:03:45.450670  1626 rpcz_store.cc:276] 0709 13:03:44.062018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.062125 (+   107us) service_pool.cc:224] Handling call
0709 13:03:44.072904 (+ 10779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.375788 (+302884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.449969 (+1074181us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540019c154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:45.450134 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663898368}
I20260709 13:03:45.420023  8578 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.453853  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.414031  8540 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:45.460167  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 105367us
W20260709 13:03:45.461093  1523 leader_election.cc:336] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.874s (SENT)
I20260709 13:03:45.461546  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.462483  8453 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.463671  8453 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.464211  8453 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.467643  8453 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.469807  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:45.470851  1706 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:45.472235  1631 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:45.479846  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.498659  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.499372  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.500273  1633 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:03:45.501657  8225 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.502454  8225 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.503134  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.503475  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.505615  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:45.508174  1626 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:45.507995  8405 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:45.508206  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.425060  8604 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:45.513495  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:45.514333  8285 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.006s
W20260709 13:03:45.495221  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.521850  1627 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:03:45.521960  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.522045  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:45.527664  1706 raft_consensus.cc:2410] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:03:45.535694  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:45.592686  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 70811.7us
I20260709 13:03:45.593815  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.594760  8405 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.595458  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.597918  8453 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:45.598223  8651 log.cc:927] Time spent T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:03:45.598376  8453 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:45.598834  8453 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.598634  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.596642  8604 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.600363  8604 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:45.601042  8456 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.595510  8405 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:45.641000  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 105421us
I20260709 13:03:45.644122  1673 leader_election.cc:400] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:45.645077  1673 leader_election.cc:403] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:03:45.669099  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.675755  8437 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:45.678900  8405 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:45.679559  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81027.7us
I20260709 13:03:45.675997  8627 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:03:45.680428  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.680737  8617 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.680997  8627 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:45.681480  8627 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:45.682503  1523 leader_election.cc:400] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:45.682834  1523 leader_election.cc:403] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:03:45.676218  8471 raft_consensus.cc:3060] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.451861  8640 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.235s
W20260709 13:03:45.640493  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 119933us
I20260709 13:03:45.421809  8575 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 13:03:45.683707  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:45.713375  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3041) took 1464 ms. Trace:
I20260709 13:03:45.714464  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:45.806915  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 85699.9us
I20260709 13:03:45.808584  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:45.815611  1701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.008s	sys 0.046s
W20260709 13:03:45.816376  1701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.009s	sys 0.046s
I20260709 13:03:45.514973  8285 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.686954  8640 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:03:45.720746  8595 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.886s	user 0.014s	sys 0.000s
I20260709 13:03:45.703867  8604 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18.
I20260709 13:03:45.683949  8607 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.685498  8631 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.686209  8538 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:45.805789  8456 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.539944  8578 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:45.819967  8443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.022s	sys 0.033s
W20260709 13:03:45.872752  1673 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:44509 timed out after 1.656s (SENT) [suppressed 47 similar messages]
I20260709 13:03:45.823670  8560 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.881174  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.020s	sys 0.012s
W20260709 13:03:45.881580  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.020s	sys 0.012s
W20260709 13:03:45.879686  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.016s	sys 0.016s
I20260709 13:03:45.824858  8612 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:45.888093  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.017s	sys 0.016s
W20260709 13:03:45.888520  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 527299us
I20260709 13:03:45.891031  8617 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:45.873055  8443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.024s	sys 0.041s
I20260709 13:03:45.897233  1557 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
W20260709 13:03:45.825352  8634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.042s	sys 0.072s
I20260709 13:03:45.682523  8627 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:45.923651  1635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.045s
W20260709 13:03:45.926267  1635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.046s
I20260709 13:03:45.880673  8604 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:45.864899  8640 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:45.919459  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:45.937670  1557 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:03:45.898417  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.020s	sys 0.008s
W20260709 13:03:45.938412  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.020s	sys 0.008s
I20260709 13:03:45.896281  8612 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.940265  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.940526  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:45.938998  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 604645us
W20260709 13:03:45.932600  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.007s	sys 0.004s
I20260709 13:03:45.932389  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3050) took 1117 ms. Trace:
W20260709 13:03:45.928494  8634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.052s	sys 0.074s
I20260709 13:03:45.899722  8613 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:45.926060  8264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.032s
W20260709 13:03:45.952344  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.007s	sys 0.005s
W20260709 13:03:45.954775  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 542643us
I20260709 13:03:45.920375  8375 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:45.888787  8560 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:45.928750  8483 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:03:45.965512  2293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.009s	sys 0.008s
W20260709 13:03:45.967065  2293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.010s	sys 0.008s
I20260709 13:03:45.957119  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:45.956637  8264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.015s	sys 0.032s
I20260709 13:03:45.961764  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:45.951727  1635 rpcz_store.cc:276] 0709 13:03:44.815254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.836138 (+ 20884us) service_pool.cc:224] Handling call
0709 13:03:44.836411 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.836592 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:44.836892 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.928415 (+1091523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:45.928502 (+    87us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:45.928566 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.932074 (+  3508us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.932300 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6501,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3967,"raft.queue_time_us":122408,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":1091017,"threads_started":2,"wal-append.queue_time_us":672867}
I20260709 13:03:45.930925  8607 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.938280  8540 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:45.983471  2291 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.972736  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:45.960692  8627 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:45.945580  8612 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.943702  8604 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:45.996776  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:45.959908  8453 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:45.964533  8225 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.000203  1631 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:46.003463  2294 raft_consensus.cc:2410] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:45.995860  2263 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:46.011485  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.015231  2296 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:46.015407  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.016088  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.015142  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.016819  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.016876  2295 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:46.021831  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:46.023006  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:46.039352  2292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.026s	sys 0.046s
W20260709 13:03:46.039990  2292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.027s	sys 0.047s
I20260709 13:03:46.040879  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1553) took 1662 ms. Trace:
I20260709 13:03:46.041280  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2606) took 1281 ms. Trace:
I20260709 13:03:46.006696  8375 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:46.047541  1601 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:46.023689  1709 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:46.049312  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:45.989017  8483 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.041744  1704 rpcz_store.cc:276] 0709 13:03:44.760214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.899236 (+139022us) service_pool.cc:224] Handling call
0709 13:03:44.899617 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.998606 (+ 98989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.998638 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.998691 (+    53us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:44.998748 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.024767 (+1026019us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b5400042754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:46.037833 (+ 13066us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.041222 (+  3389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227164928}
I20260709 13:03:46.041492  2292 rpcz_store.cc:276] 0709 13:03:44.378580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.378726 (+   146us) service_pool.cc:224] Handling call
0709 13:03:44.417577 (+ 38851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.417772 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:44.691792 (+274020us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.040429 (+1348637us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.040507 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:46.040568 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.040653 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.040811 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27655,"prepare.queue_time_us":158408,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":464186,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2324,"thread_start_us":1524905,"threads_started":3,"wal-append.queue_time_us":983804}
I20260709 13:03:46.045351  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:46.051160  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.051916  1627 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:46.056252  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:46.056574  1558 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:46.018321  8604 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:46.057457  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:46.057998  1709 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:46.062593  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.026s	sys 0.008s
I20260709 13:03:46.069830  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.063570  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:46.070396  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.026s	sys 0.008s
I20260709 13:03:46.075521  1709 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:03:46.064330  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:46.076433  8225 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:46.076680  8578 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:46.056860  8544 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.027s
I20260709 13:03:46.077302  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.077972  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:46.073812  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3131) took 2012 ms. Trace:
I20260709 13:03:46.046800  8375 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.154491  1634 rpcz_store.cc:276] 0709 13:03:44.061712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.061842 (+   130us) service_pool.cc:224] Handling call
0709 13:03:44.062121 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.098071 (+ 35950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.068149 (+1970078us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b54000cc6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:46.073738 (+  5589us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71426304}
I20260709 13:03:45.979292  8640 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.082998  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1149) took 1369 ms. Trace:
I20260709 13:03:46.055939  8634 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:46.089087  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.019s	sys 0.016s
W20260709 13:03:46.176162  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.019s	sys 0.017s
I20260709 13:03:46.089843  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:46.177191  1555 raft_consensus.cc:2376] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19. Re-sending same reply.
W20260709 13:03:46.177333  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 158188us
I20260709 13:03:46.121081  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:46.094743  1709 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:03:46.074858  8630 raft_consensus.cc:2804] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:46.078017  8264 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.998637  8607 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.070714  8666 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:46.178104  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.159746  1632 rpcz_store.cc:276] 0709 13:03:44.713180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.713472 (+   292us) service_pool.cc:224] Handling call
0709 13:03:44.738117 (+ 24645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.012073 (+273956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.012102 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.022749 (+ 10647us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.144448 (+121699us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:45.144503 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.069540 (+925037us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b540014fd54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:46.069588 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.069631 (+    43us) raft_consensus.cc:1685] finished
0709 13:03:46.069643 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.082901 (+ 13258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44633,"raft.queue_time_us":98350,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":51574,"spinlock_wait_cycles":466569984,"thread_start_us":119239,"threads_started":1,"wal-append.queue_time_us":226960}
I20260709 13:03:46.061401  8595 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:03:46.180608  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:46.183225  1707 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:46.178062  1709 pending_rounds.cc:85] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
W20260709 13:03:46.191056  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 140533us
I20260709 13:03:46.275475  1709 pending_rounds.cc:107] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:03:46.275794  1709 raft_consensus.cc:2889] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:46.163024  8544 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.007s	sys 0.028s
I20260709 13:03:46.275887  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.280417  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1202) took 1068 ms. Trace:
I20260709 13:03:46.280865  1707 rpcz_store.cc:276] 0709 13:03:45.212034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.076227 (+864193us) service_pool.cc:224] Handling call
0709 13:03:46.280331 (+204104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:46.152531  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 88265.9us
W20260709 13:03:46.211930  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 126190us
I20260709 13:03:46.284672  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1200) took 1074 ms. Trace:
I20260709 13:03:46.285283  1709 rpcz_store.cc:276] 0709 13:03:45.210839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.076302 (+865463us) service_pool.cc:224] Handling call
0709 13:03:46.079624 (+  3322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.284346 (+204722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.284597 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.189745  8578 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:46.088006  8285 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:46.187340  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 771556us
I20260709 13:03:46.157497  8211 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
I20260709 13:03:46.158849  8375 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.162178  8539 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:46.285740  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 236049us
I20260709 13:03:46.206817  8675 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.300657  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.274183  8608 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.270372  1557 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:46.293567  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.300657  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:46.304391  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 126359us
I20260709 13:03:46.186587  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:46.305331  1707 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:03:46.270633  8666 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.313450  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.315233  1632 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:46.280671  8630 raft_consensus.cc:697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:46.283805  8443 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.317323  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:46.335105  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.327975  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1207) took 1096 ms. Trace:
I20260709 13:03:46.335593  1555 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:46.336784  1707 rpcz_store.cc:276] 0709 13:03:45.215374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.281857 (+1066483us) service_pool.cc:224] Handling call
0709 13:03:46.302727 (+ 20870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.311069 (+  8342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.311337 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.320269  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:46.301492  8657 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
W20260709 13:03:46.333106  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57331.8us
I20260709 13:03:46.209257  8264 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.322851  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:46.273468  8634 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:46.298873  8560 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.274226  8670 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:03:46.350996  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.364228  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:46.381281  1555 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.303200  8640 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:46.371963  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49151.8us
I20260709 13:03:46.297578  8539 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:46.306566  8676 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:03:46.304416  8675 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.404274  1634 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 16: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:46.398856  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47941.8us
I20260709 13:03:46.451128  1707 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:03:46.452001  1707 pending_rounds.cc:85] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:46.452291  1707 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:03:46.452608  1707 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:46.437528  1555 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:46.347313  8285 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:46.399005  8691 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:46.454463  8670 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:46.403311  8668 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:03:46.400933  8673 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:03:46.379864  8544 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.361254  8443 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.457175  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:46.457995  8544 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:46.458509  8544 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:46.459175  8443 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:46.459432  8544 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.320331  8608 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.462175  8608 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.394965  8634 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.450559  8539 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.363807  8578 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:46.468708  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:46.403057  8630 consensus_queue.cc:237] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.472589  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:46.400352  8669 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:03:46.474216  1555 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:46.474906  8607 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:46.475633  8660 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:03:46.477079  8681 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:46.477548  8681 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.477885  8681 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:46.444147  8678 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:46.480195  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.480484  8678 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:46.481305  8670 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.481473  8678 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:46.448475  8659 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 13:03:46.301072  8437 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:03:46.487288  8437 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:46.487712  8437 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:46.487412  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.488672  8437 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.489831  8437 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.506428  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:46.507722  1556 raft_consensus.cc:2410] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:46.321573  8666 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:46.321910  8538 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:46.418510  8667 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:03:46.417529  8609 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.208s	user 0.010s	sys 0.001s
W20260709 13:03:46.415253  8579 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.682s	user 0.010s	sys 0.005s
I20260709 13:03:46.334273  8453 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:46.351866  8658 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 13:03:46.532553  2295 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
W20260709 13:03:46.419260  8665 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
I20260709 13:03:46.348378  8540 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.547132  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.601241  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.604243  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.605329  2295 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:46.607959  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:46.611362  2291 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.001s
I20260709 13:03:46.396258  8264 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.617193  2291 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:46.618958  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.620720  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.529018  8538 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.630517  2292 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:46.633445  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.606845  8443 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.635604  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.635891  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.600512  8211 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.645025  1626 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:46.650096  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.652787  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:46.617336  8682 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.659044  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:46.662765  2263 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.925s (SENT)
I20260709 13:03:46.662981  1705 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:03:46.663249  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:46.664361  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.664409  8672 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:46.666811  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 331756us
I20260709 13:03:46.667692  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.549222  8579 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.547333  8609 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:46.629937  8538 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:46.624281  8176 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.670574  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:46.614216  8540 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.641930  8690 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.636160  8663 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:46.659668  8471 consensus_meta.cc:287] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.976s	user 0.012s	sys 0.000s
I20260709 13:03:46.685096  2291 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:46.699759  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1556) took 1455 ms. Trace:
I20260709 13:03:46.703059  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.700322  2293 rpcz_store.cc:276] 0709 13:03:45.244235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.244373 (+   138us) service_pool.cc:224] Handling call
0709 13:03:45.268206 (+ 23833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.400580 (+132374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.400619 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.412128 (+ 11509us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.967518 (+555390us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:45.967574 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.967662 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.699406 (+731744us) raft_consensus.cc:1685] finished
0709 13:03:46.699436 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.699661 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131874,"raft.queue_time_us":139859,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":554961,"threads_started":1,"wal-append.queue_time_us":675571}
I20260709 13:03:46.709661  1633 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:46.713608  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2609) took 1553 ms. Trace:
I20260709 13:03:46.717396  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.714164  1701 rpcz_store.cc:276] 0709 13:03:45.160167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.167816 (+  7649us) service_pool.cc:224] Handling call
0709 13:03:45.168188 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:45.180187 (+ 11999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.180235 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:45.195415 (+ 15180us) log.cc:844] Serialized 52 byte log entry
0709 13:03:45.817262 (+621847us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:45.817328 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.817434 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.713299 (+895865us) raft_consensus.cc:1685] finished
0709 13:03:46.713332 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.713517 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11542,"raft.queue_time_us":20950,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":23148,"thread_start_us":543098,"threads_started":1,"wal-append.queue_time_us":707094}
I20260709 13:03:46.665529  8443 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.658824  8211 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.701894  8471 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18.
I20260709 13:03:46.729094  2293 raft_consensus.cc:2368] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:46.733759  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.734404  2291 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:03:46.737401  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.716405  8672 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:46.746125  1706 raft_consensus.cc:2468] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:46.722926  8575 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:46.732395  8609 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:46.696359  8176 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.668792  8682 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.777979  8471 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:46.732146  8539 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.790977  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:46.774693  8612 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.766s	user 0.011s	sys 0.002s
I20260709 13:03:46.749688  8672 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.795334  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.752374  8560 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.710162  8663 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:46.791605  8689 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:03:46.804279  8540 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:46.779364  8264 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.813097  1704 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:46.704391  8690 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.814139  8539 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.820497  8693 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.824946  8703 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:46.771324  8634 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:03:46.844442  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.801937  8672 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.848824  1632 raft_consensus.cc:2410] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:46.799196  8675 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:46.849574  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173134us
W20260709 13:03:46.836995  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127840us
I20260709 13:03:46.868192  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:46.877122  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.841969  8676 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:46.800987  8612 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:46.873265  8686 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:03:46.829026  8560 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:46.838369  8682 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.888280  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:46.894552  2290 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:46.866669  8539 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:46.902132  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:46.902827  1707 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:03:46.903911  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 109100us
I20260709 13:03:46.904706  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:46.882573  8691 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:46.844875  8264 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:46.905035  8627 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.001s
I20260709 13:03:46.914359  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:46.904258  8643 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 1.491s	user 0.013s	sys 0.000s
I20260709 13:03:46.898689  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:46.920256  1702 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:46.864099  8693 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:46.878820  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 143610us
I20260709 13:03:46.869609  8634 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:46.942852  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71801.2us
I20260709 13:03:46.947431  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:46.913270  8540 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.947654  2291 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:46.953152  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:46.954833  8702 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.955407  8704 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:03:46.956126  8710 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:46.906903  8539 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.957631  8709 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:46.907617  8682 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:03:46.907755  8285 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:03:46.907897  8625 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 13:03:46.957079  8701 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:03:46.946013  8609 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:46.959713  8579 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:46.970656  8715 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:46.925609  8703 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:03:46.960789  8627 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.962471  8211 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:46.922595  8643 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:47.075589  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 167968us
W20260709 13:03:47.043140  8687 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:03:47.077087  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:47.086654  1703 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:46.966908  8540 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:46.967588  8702 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:47.089027  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135973us
I20260709 13:03:47.089730  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:47.082916  8375 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.762s	user 0.009s	sys 0.004s
I20260709 13:03:47.043704  8176 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:47.085448  8578 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 13:03:47.104961  1674 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.089136  8715 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:03:47.090153  8663 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.099932  8211 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:03:47.112087  8578 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:47.112948  8443 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:47.115370  8437 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.001s
I20260709 13:03:47.121868  8634 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.125874  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3141) took 1623 ms. Trace:
I20260709 13:03:47.126473  1633 rpcz_store.cc:276] 0709 13:03:45.502406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.507897 (+  5491us) service_pool.cc:224] Handling call
0709 13:03:47.125556 (+1617659us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540024f254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.125783 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2760301312}
I20260709 13:03:47.130317  8693 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.037494  8709 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:47.138567  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.145236  2297 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:47.061574  8627 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.123389  8264 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.090025  8609 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:45.713829  1630 rpcz_store.cc:276] 0709 13:03:44.248812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.257983 (+  9171us) service_pool.cc:224] Handling call
0709 13:03:44.278178 (+ 20195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.278424 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:44.302234 (+ 23810us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.213446 (+911212us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b18003a38a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:45.712519 (+499073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:45.712575 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:45.712624 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.712693 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.713304 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67005,"prepare.queue_time_us":23487,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":8019,"spinlock_wait_cycles":19773312,"thread_start_us":498771,"threads_started":1}
I20260709 13:03:47.158043  8211 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:47.136409  8710 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:47.168419  8672 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.106551  8375 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.170768  8560 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:47.174918  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 70018.3us
I20260709 13:03:47.190114  8722 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:47.191102  8710 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.136055  8640 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.200208  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.155419  8715 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:47.200345  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 252650us
I20260709 13:03:47.175863  8211 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.190948  8714 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:47.191824  8612 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:47.097693  8723 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:03:47.208113  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:47.211465  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.203167  8710 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:47.219679  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.220379  1705 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.221480  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.206236  8211 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:47.224027  8609 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.206300  8176 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:47.217445  8538 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.228834  8375 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.230007  8538 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:47.241417  1705 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:47.208729  8722 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:03:47.211992  8714 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.207494  8640 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.246555  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.263371  1708 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:47.255470  1628 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:47.260653  1701 raft_consensus.cc:2410] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:47.262174  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2552) took 1364 ms. Trace:
I20260709 13:03:47.282881  1557 rpcz_store.cc:276] 0709 13:03:45.898224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.940226 (+ 42002us) service_pool.cc:224] Handling call
0709 13:03:47.262002 (+1321776us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540036ecd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.262106 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756969216}
I20260709 13:03:47.284536  8730 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.285117  8730 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.287899  8730 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:47.286705  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48428.3us
I20260709 13:03:47.259914  8714 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.254740  7994 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.262105  8709 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:47.290931  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.257918  8672 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.292307  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:47.292670  8661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.000s	sys 0.013s
I20260709 13:03:47.292869  8613 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:47.293041  1708 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:47.258947  8612 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.294107  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.293911  8612 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.294891  1708 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:47.259194  8607 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.296255  8607 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.297350  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.258841  8675 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.298077  1701 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:47.298313  8675 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.298867  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.261531  8640 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.299657  1701 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:03:47.293344  8613 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.300489  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.304157  1708 raft_consensus.cc:2368] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 16. Current term is 17.
W20260709 13:03:47.293119  8661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.000s	sys 0.013s
W20260709 13:03:47.257504  8666 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.656s	user 0.002s	sys 0.009s
I20260709 13:03:47.247233  8726 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:47.304982  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.303890  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.317318  1701 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:47.302639  8569 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:47.288663  8730 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:47.303438  8648 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:03:47.314697  8681 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.836s	user 0.015s	sys 0.000s
I20260709 13:03:47.327931  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.305171  7994 consensus_queue.cc:1048] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:03:47.327396  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.322177  8607 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.329133  8675 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.329521  8540 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.341982  8540 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.342459  8540 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.344749  8540 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:47.345273  8655 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.458s	user 0.006s	sys 0.000s
I20260709 13:03:47.322187  8635 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:03:47.327183  8612 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.323673  8569 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:47.347018  8211 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.331380  8710 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.327412  8648 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.362511  8648 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:47.364341  8710 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.333391  8681 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.339229  8732 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.340824  8691 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:47.347118  8569 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.335457  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.372766  2294 raft_consensus.cc:2368] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:47.327243  8730 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.315999  8666 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.332024  8627 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:47.353560  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39480 (request call id 1135) took 2214 ms (client timeout 1925 ms). Trace:
I20260709 13:03:47.353809  1559 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:47.355055  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.392011  1700 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.369109  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:47.375726  1560 rpcz_store.cc:269] 0709 13:03:45.139230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.217084 (+ 77854us) service_pool.cc:224] Handling call
0709 13:03:47.353224 (+2136140us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b54002f9754. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.353426 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177811840}
I20260709 13:03:47.371367  1551 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:47.367960  8612 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.370289  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 65376.6us
I20260709 13:03:47.373730  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.339690  8483 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.373354  8714 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:47.396816  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:47.394809  8569 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:47.413144  1700 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:47.411444  1701 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:47.419467  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:47.398084  8678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.006s	sys 0.014s
I20260709 13:03:47.405201  8730 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:47.422313  8678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.006s	sys 0.015s
I20260709 13:03:47.422583  8730 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:47.396515  8612 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.422865  8681 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:47.424161  8613 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.425635  8648 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.424705  8612 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.420821  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.434470  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.435791  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.436573  1708 raft_consensus.cc:2376] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21. Re-sending same reply.
I20260709 13:03:47.408215  8607 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:47.440076  1707 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:47.410995  8635 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.445233  8648 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.454293  8211 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.405531  8661 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:47.456593  1708 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:47.457882  8666 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:47.458917  1708 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:03:47.459440  1708 pending_rounds.cc:85] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 9
I20260709 13:03:47.463141  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.459712  1708 pending_rounds.cc:107] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:03:47.464839  1708 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:47.465510  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.469061  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.441139  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.467017  8715 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.472635  8721 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.467154  8607 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.472644  8710 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:47.473495  8607 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.422977  8730 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.475214  8627 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:47.499111  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2542) took 2517 ms. Trace:
I20260709 13:03:47.499305  8627 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:47.499673  1552 rpcz_store.cc:276] 0709 13:03:44.981819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.109973 (+128154us) service_pool.cc:224] Handling call
0709 13:03:46.069944 (+959971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.070120 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.070142 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.070188 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:46.070239 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.070294 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.498851 (+1428557us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54000e8650. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.499008 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163514240}
I20260709 13:03:47.500726  8627 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.485991  8730 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:47.507025  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:47.497946  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:47.511250  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.523566  8470 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:47.524099  8470 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:47.524371  8607 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.524950  8470 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.526193  8470 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:47.526840  8675 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:47.527240  8675 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:47.532019  8738 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:47.532791  8738 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.533263  8607 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.534000  8607 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.540266  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.527626  8675 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:47.535516  8738 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:47.610495  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 236828us
W20260709 13:03:47.607537  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 69861.3us
I20260709 13:03:47.611270  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:47.612298  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.543262  8456 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:47.614302  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.614888  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.622362  8661 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:47.622988  8661 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.625305  8661 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.625862  8661 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.626309  8661 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.630218  8714 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:47.632717  8607 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.637118  8612 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:47.645474  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:47.646517  1558 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:47.665319  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.667847  1702 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:47.683508  1560 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:03:47.683957  1558 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.687902  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1205) took 2474 ms. Trace:
I20260709 13:03:47.688386  1704 rpcz_store.cc:276] 0709 13:03:45.214080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.155056 (+940976us) service_pool.cc:224] Handling call
0709 13:03:46.269182 (+114126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.840079 (+570897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.687608 (+847529us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54001b8854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:47.687830 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250317184}
W20260709 13:03:47.689321  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1213) took 2310 ms (client timeout 2035 ms). Trace:
W20260709 13:03:47.689721  1706 rpcz_store.cc:269] 0709 13:03:45.379407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.644110 (+264703us) service_pool.cc:224] Handling call
0709 13:03:47.689136 (+2045026us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001cb6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.689254 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2368335232}
I20260709 13:03:47.690982  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.693260  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.694247  1557 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:47.695336  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.696054  1706 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:47.696727  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.698118  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.679666  1702 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:47.703442  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:47.705621  1702 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:47.709831  1702 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:03:47.710760  1702 pending_rounds.cc:85] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:03:47.711442  1702 pending_rounds.cc:107] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:03:47.712124  1702 raft_consensus.cc:2889] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:47.684401  8456 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:47.684692  8612 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:47.682489  8710 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.729571  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.698935  8405 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:47.733304  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.717418  8456 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.724130  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:47.739006  1704 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.738926  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:47.740157  1626 raft_consensus.cc:2393] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:47.717865  8709 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.720876  8640 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.736831  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:47.736912  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.737468  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.740530  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:47.742693  2260 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.035s (SENT)
I20260709 13:03:47.747578  1558 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:47.753978  1557 raft_consensus.cc:2410] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 24 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:03:47.745369  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.749742  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.756764  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.746282  1634 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:47.757329  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.758236  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.758985  1558 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:47.761386  1704 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:47.763346  1560 raft_consensus.cc:2393] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 16: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:47.763608  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.764930  1630 raft_consensus.cc:2393] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:47.733695  8405 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.742362  8456 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:47.769829  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.772383  1708 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.774092  8539 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.765061  8661 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:47.783797  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2614) took 1833 ms. Trace:
I20260709 13:03:47.789232  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3150) took 1161 ms. Trace:
I20260709 13:03:47.789736  1628 rpcz_store.cc:276] 0709 13:03:46.627839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.628010 (+   171us) service_pool.cc:224] Handling call
0709 13:03:47.255259 (+627249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.281156 (+ 25897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.789022 (+507866us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54000dd9d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.789163 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386772224}
I20260709 13:03:47.789362  1709 rpcz_store.cc:276] 0709 13:03:45.950202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.294542 (+344340us) service_pool.cc:224] Handling call
0709 13:03:46.294885 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.222332 (+927447us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:47.242754 (+ 20422us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.611107 (+368353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:47.611156 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:47.611198 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.783538 (+172340us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400166a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:47.783584 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.783714 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1657,"prepare.queue_time_us":41024,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":132005,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":12111,"spinlock_wait_cycles":1506640256,"thread_start_us":602986,"threads_started":2,"wal-append.queue_time_us":507209}
I20260709 13:03:47.794426  8719 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:47.794737  8405 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:47.794957  8719 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:47.796269  8719 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.797416  8719 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:47.799789  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3138) took 2302 ms. Trace:
I20260709 13:03:47.800279  1631 rpcz_store.cc:276] 0709 13:03:45.497738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.518854 (+ 21116us) service_pool.cc:224] Handling call
0709 13:03:45.519259 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.004982 (+485723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.799460 (+1794478us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54001e8a54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:47.799662 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105445632}
I20260709 13:03:47.800828  8643 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:47.801525  2260 leader_election.cc:336] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.309s (SENT)
I20260709 13:03:47.801964  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.803908  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.805132  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1486) took 1262 ms. Trace:
I20260709 13:03:47.805533  2297 rpcz_store.cc:276] 0709 13:03:46.542954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.600919 (+ 57965us) service_pool.cc:224] Handling call
0709 13:03:47.804955 (+1204036us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.805051 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181194112}
I20260709 13:03:47.805891  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.744954  8739 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.807049  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.807265  8739 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:47.808256  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.809478  8739 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:47.805475  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.816053  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:47.818969  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:47.818087  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.756080  8722 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:47.827925  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 215221us
W20260709 13:03:47.829725  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.830295  1524 leader_election.cc:400] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:47.830585  1524 leader_election.cc:403] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:03:47.831462  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.832834  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:47.834827  2295 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:47.837875  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:47.838694  2292 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:47.839865  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.842446  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.843917  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.811811  8707 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.831946  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.826344  1706 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:47.876034  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.878118  2297 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.878556  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:47.878880  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:47.880244  1706 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:03:47.880373  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.882238  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.883328  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.884142  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:47.805516  8712 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.886534  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.887240  1555 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:47.894300  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.895256  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.897112  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.897997  1633 raft_consensus.cc:2410] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:47.875787  8722 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.902074  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.903689  1558 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:47.904592  8749 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:47.905123  8749 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:47.906032  8749 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.849272  8738 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:47.908243  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66221.9us
I20260709 13:03:47.914353  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:47.914708  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:47.915366  1553 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:47.912828  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.921298  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:47.922984  2292 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:47.923024  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:47.926786  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.928208  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:47.848470  8612 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:03:47.932801  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.933686  1633 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
W20260709 13:03:47.939378  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.941608  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:47.942965  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.943697  2289 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:47.943040  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.949203  2297 raft_consensus.cc:2393] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:47.956660  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:47.956899  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:47.957914  2297 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:47.958662  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.923568  8626 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:47.963595  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.962262  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:47.933094  8662 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
W20260709 13:03:47.948134  8740 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:03:47.970148  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 319957us
I20260709 13:03:47.879077  8707 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:47.966647  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.967053  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:47.978999  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:47.940128  8684 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:03:47.980768  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:47.970283  8661 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:47.984985  1601 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:47.965569  8483 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:03:47.978446  8610 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:47.986224  1601 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:47.964530  8061 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:03:47.984382  8650 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:47.998164  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3151) took 1365 ms. Trace:
I20260709 13:03:47.996268  8564 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.002108  8690 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:48.002600  8702 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:48.003124  8722 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.003688  8061 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:48.003989  8714 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:48.006011  1632 rpcz_store.cc:276] 0709 13:03:46.632367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.632967 (+   600us) service_pool.cc:224] Handling call
0709 13:03:47.997950 (+1364983us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540014fd54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:47.998097 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448375040}
I20260709 13:03:47.989451  8684 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:48.007438  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:48.065411  1557 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:47.991314  8661 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:48.022198  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:48.069415  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47355us
I20260709 13:03:47.995936  8610 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.016539  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:48.049942  1705 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:48.005129  8650 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.991019  8730 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:48.050160  8231 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:48.006582  8675 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.077402  8650 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.061465  8714 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:48.065109  8684 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:48.075986  8610 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:48.075515  8712 raft_consensus.cc:2764] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:03:48.059279  8670 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:03:48.093590  8456 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.137959  1557 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:48.139215  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:48.139936  1557 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:48.153065  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:48.158658  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200132us
I20260709 13:03:48.109676  8231 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.161595  1555 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:48.166370  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:48.184304  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 169669us
I20260709 13:03:48.188913  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.190635  1557 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:48.192039  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.192888  1557 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:03:48.167380  8594 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:48.194517  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.205981  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.208590  1560 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:48.213249  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:48.213935  1560 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.171269  8714 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:48.216766  8714 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.219645  8648 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:48.220191  8648 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:48.220439  8752 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:48.221144  8648 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.222522  8648 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:48.222790  8753 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:48.224804  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.226034  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.226639  1552 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
W20260709 13:03:48.228181  8751 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:03:48.229967  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.230597  1554 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:48.231211  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3008) took 3164 ms. Trace:
I20260709 13:03:48.232007  8755 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:48.231650  1559 rpcz_store.cc:276] 0709 13:03:45.067250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.333848 (+266598us) service_pool.cc:224] Handling call
0709 13:03:47.353412 (+2019564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.394244 (+ 40832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.217025 (+822781us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54002f9750. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:48.231152 (+ 14127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4643825024}
W20260709 13:03:48.233202  8756 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:03:48.233701  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.233960  8231 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.234372  1559 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:03:48.234886  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.235263  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.235697  1551 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:48.235915  1559 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:48.236256  8456 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.236548  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:48.236775  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.237046  8456 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.237774  8456 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.158475  8684 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.152302  8661 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.208408  8405 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.206805  8707 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:48.243531  2294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1488) took 1632 ms (client timeout 1638 ms). Trace:
I20260709 13:03:48.227491  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.231436  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.237726  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:48.259958  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1214) took 2869 ms (client timeout 2309 ms). Trace:
I20260709 13:03:48.253445  8650 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.232492  8755 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:48.259451  2294 rpcz_store.cc:269] 0709 13:03:46.611113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.611248 (+   135us) service_pool.cc:224] Handling call
0709 13:03:48.243286 (+1632038us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54005e1854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:48.243460 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727155584}
I20260709 13:03:48.252251  8730 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:48.237833  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:48.260720  1703 rpcz_store.cc:269] 0709 13:03:45.379724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.045075 (+665351us) service_pool.cc:224] Handling call
0709 13:03:48.249174 (+2204099us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400042c54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:48.249338 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379648384}
I20260709 13:03:48.263903  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:48.264736  1557 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:48.266489  1555 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:48.266875  1559 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:48.268340  1559 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:03:48.268447  1551 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:48.245899  8761 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:48.270422  8761 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.270810  8761 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.245632  8675 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:48.249743  8734 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.455s	user 0.005s	sys 0.000s
W20260709 13:03:48.290268  8746 log.cc:927] Time spent T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.519s	user 0.000s	sys 0.002s
W20260709 13:03:48.303743  8744 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.534s	user 0.001s	sys 0.001s
W20260709 13:03:48.307078  8750 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.313s	user 0.000s	sys 0.006s
I20260709 13:03:48.307319  8610 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.314507  8610 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.239738  8607 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:48.335440  1701 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:48.337064  1554 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:48.346634  1555 raft_consensus.cc:1275] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:48.349592  8456 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.266695  8755 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.353220  1555 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:03:48.357296  1557 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:48.339700  8684 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.361626  1560 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:48.346038  8626 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.383276  2291 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:48.403821  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 214956us
I20260709 13:03:48.401271  8456 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:48.414779  2295 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:48.419806  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:48.420660  2294 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:48.424446  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:48.421125  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:48.415310  8456 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:48.424746  8764 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 13:03:48.434782  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:48.431438  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:48.444600  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:48.445289  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:48.447710  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 253310us
I20260709 13:03:48.432837  8706 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:48.449050  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.436129  8650 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.457476  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:48.460695  8707 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:48.466465  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.439865  8625 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:03:48.467974  8607 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:48.468986  8625 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:48.469301  1631 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:48.469518  8625 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:48.438359  8578 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 13:03:48.436838  8670 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:03:48.475914  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:48.474292  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:48.476526  1626 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:48.471859  1630 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:03:48.477240  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:48.452379  8405 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.485802  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:48.486618  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:48.471549  8627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.028s	sys 0.012s
I20260709 13:03:48.488036  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:48.496162  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:48.497575  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:48.498342  1630 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:48.446295  8739 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:03:48.502280  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.470791  8706 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:48.484133  8625 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.474232  8675 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:48.517582  8670 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:48.518232  8670 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.523468  8675 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.523871  8738 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 13:03:48.524413  8738 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:48.524453  8675 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.525305  1629 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:03:48.532388  8706 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.533516  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:48.534142  2290 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:48.535753  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:48.494952  8627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.029s	sys 0.012s
I20260709 13:03:48.536983  8738 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:48.537470  8738 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.537914  8738 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.537942  8670 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:48.540663  1626 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:48.543941  1553 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:48.548715  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:48.550557  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:48.552119  1525 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:48.564664  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:48.497200  8676 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:48.584365  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:48.605311  1705 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:03:48.545270  8675 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:48.561975  8564 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.004s
I20260709 13:03:48.608595  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:48.616863  1707 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:48.618392  1598 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:48.623443  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:48.474568  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:48.625519  1635 raft_consensus.cc:606] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:48.626296  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:48.586386  8676 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:48.624297  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:48.635752  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:48.636714  1705 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
W20260709 13:03:48.637701  8771 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:48.638417  8714 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.642356  8714 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:48.610720  8610 raft_consensus.cc:2764] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:03:48.604177  8770 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:48.638741  1705 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:48.648746  8726 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:48.621977  8564 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.668418  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:48.658792  1705 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:48.663496  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:48.680109  2296 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:48.664510  2294 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:48.667639  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:48.680855  1627 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:48.682621  1631 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:03:48.611407  8755 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.644105  8684 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:48.691708  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 43997.9us
W20260709 13:03:48.698321  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 231210us
I20260709 13:03:48.671203  8770 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:48.703671  2263 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:48.709482  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:48.719129  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:48.712455  8670 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:48.733505  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 179244us
I20260709 13:03:48.736426  8770 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.711989  8664 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:03:48.730592  8648 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.508s	user 0.016s	sys 0.000s
I20260709 13:03:48.701999  8707 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:48.761754  8707 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.775879  8777 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:03:48.777062  8777 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:48.777462  8777 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:48.778393  8777 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.743355  8670 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:48.780375  8772 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:03:48.780915  8670 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.716205  8676 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.783668  8676 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.790776  2296 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:48.790812  8613 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:48.791306  8594 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:48.796041  8456 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.742844  8578 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
W20260709 13:03:48.788066  8761 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
I20260709 13:03:48.756073  8564 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:03:48.790686  8773 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:48.787556  8777 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.749680  8662 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
W20260709 13:03:48.852008  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.852758  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.818113  8594 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:48.875310  1632 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
W20260709 13:03:48.896138  1523 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.648s (SENT)
I20260709 13:03:48.938031  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.760221  8706 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.073s
I20260709 13:03:48.952369  8730 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.964694  8730 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:48.969249  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2630) took 1551 ms. Trace:
I20260709 13:03:48.970829  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:48.969887  1704 rpcz_store.cc:276] 0709 13:03:47.417776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.761941 (+344165us) service_pool.cc:224] Handling call
0709 13:03:47.762141 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.762363 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:47.824251 (+ 61888us) log.cc:844] Serialized 14 byte log entry
0709 13:03:48.968681 (+1144430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.968768 (+    87us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:48.968840 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.968985 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.969167 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":952716,"prepare.queue_time_us":62172,"prepare.run_cpu_time_us":8365,"prepare.run_wall_time_us":29573,"thread_start_us":192417,"threads_started":1,"wal-append.queue_time_us":372375}
W20260709 13:03:48.968482  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 1.01209e+06us
W20260709 13:03:48.973974  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.866570  8578 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:48.865283  8755 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:48.960306  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64244us
I20260709 13:03:48.833077  8761 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:48.947731  8456 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.980475  1674 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:48.984356  8702 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
W20260709 13:03:48.995363  1523 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.736s (SENT)
W20260709 13:03:48.983230  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.968415  8607 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:49.005155  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 295743us
I20260709 13:03:49.019119  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.027125  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:49.020465  1675 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:49.050770  1523 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.802s (SENT)
W20260709 13:03:49.053974  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.982573  8578 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:49.060990  1560 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:03:48.997411  8437 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:49.072754  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 56447.8us
W20260709 13:03:49.072746  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 92388.6us
I20260709 13:03:49.072974  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2625) took 1656 ms. Trace:
W20260709 13:03:49.072839  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 328512us
I20260709 13:03:48.967665  8730 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:49.075035  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.109802  1704 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:03:49.113174  1704 pending_rounds.cc:85] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 14
I20260709 13:03:49.113561  1704 pending_rounds.cc:107] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:03:49.113878  1704 raft_consensus.cc:2889] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:49.115394  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:49.119867  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69168.5us
I20260709 13:03:49.125375  2292 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:49.017419  8725 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.036278  8755 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.071458  8437 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:49.166306  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 142396us
W20260709 13:03:49.165868  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 92036.1us
W20260709 13:03:49.180495  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 65211.1us
I20260709 13:03:49.153743  8640 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:49.196030  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.197232  8776 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:03:49.198915  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3039) took 1198 ms. Trace:
I20260709 13:03:49.165395  8650 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.176267  8437 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:49.216853  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
W20260709 13:03:49.218340  8738 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
W20260709 13:03:49.219233  8783 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:03:49.219406  8643 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:49.206429  8471 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:49.221091  1628 raft_consensus.cc:606] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:49.220817  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2633) took 1799 ms. Trace:
I20260709 13:03:49.236940  8471 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:49.236960  1628 raft_consensus.cc:614] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.237995  8471 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.237181  1708 rpcz_store.cc:276] 0709 13:03:47.421629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.789497 (+367868us) service_pool.cc:224] Handling call
0709 13:03:47.789891 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.864832 (+1074941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.864883 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.865531 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.033382 (+167851us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:49.033456 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.033562 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.220597 (+187035us) raft_consensus.cc:1685] finished
0709 13:03:49.220620 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.220757 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1154604,"raft.queue_time_us":1076698,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":24949,"thread_start_us":167421,"threads_started":1,"wal-append.queue_time_us":308184}
I20260709 13:03:49.199412  1554 rpcz_store.cc:276] 0709 13:03:48.000965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.355259 (+354294us) service_pool.cc:224] Handling call
0709 13:03:48.360769 (+  5510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.365433 (+  4664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.365482 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.377096 (+ 11614us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.697985 (+320889us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:48.698036 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.698106 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.198615 (+500509us) raft_consensus.cc:1685] finished
0709 13:03:49.198647 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.198836 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1256,"raft.queue_time_us":4318,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":11328,"thread_start_us":287294,"threads_started":1,"wal-append.queue_time_us":388353}
W20260709 13:03:49.229138  8763 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.836s	user 0.003s	sys 0.011s
I20260709 13:03:49.200994  8627 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.235919  8643 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.230435  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1223) took 1629 ms. Trace:
I20260709 13:03:49.220266  8761 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:49.240995  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 165280us
I20260709 13:03:49.251585  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2634) took 1829 ms. Trace:
I20260709 13:03:49.259902  1701 rpcz_store.cc:276] 0709 13:03:47.601039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.803699 (+202660us) service_pool.cc:224] Handling call
0709 13:03:47.814375 (+ 10676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.337783 (+523408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.230111 (+892328us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b540010d1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:49.230368 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197796608}
I20260709 13:03:49.261904  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.155723  8725 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.268718  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.260193  1709 rpcz_store.cc:276] 0709 13:03:47.421931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.791392 (+369461us) service_pool.cc:224] Handling call
0709 13:03:47.791881 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.866314 (+1074433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.866379 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.961913 (+ 95534us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.179654 (+217741us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:49.179805 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.180249 (+   444us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.251371 (+ 71122us) raft_consensus.cc:1685] finished
0709 13:03:49.251395 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.251523 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1155285,"raft.queue_time_us":1074717,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":32193,"thread_start_us":165263,"threads_started":1,"wal-append.queue_time_us":270002}
I20260709 13:03:49.272418  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.276979  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:49.258594  8752 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:49.277776  1709 raft_consensus.cc:2410] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:49.284168  1635 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:49.284987  1701 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:49.258113  8727 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.295563  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:49.296418  1709 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:49.297623  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:49.300904  1708 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:03:49.303221  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 282857us
I20260709 13:03:49.304366  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.195434  8578 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.276839  8709 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.322305  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.225411  8738 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.286330  8792 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:49.333989  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:49.338243  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:49.337356  8470 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:49.343883  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.332286  8578 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:49.346526  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 119051us
I20260709 13:03:49.305212  8727 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.352039  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.354012  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.328568  8709 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.355608  1675 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.358373  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.345342  8725 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.357167  8470 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:49.361042  8470 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:49.361411  8470 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:49.360313  8437 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.359802  8640 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:49.365497  8727 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.366885  8437 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.369087  8709 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.359406  8738 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.369967  8807 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:49.371292  8807 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:49.371713  8807 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.372128  8807 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:49.359570  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:49.353134  8690 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:49.379489  1709 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:49.405646  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.406726  1709 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:03:49.408465  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.373241  8708 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:49.444272  8708 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:49.445147  8708 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.446453  8708 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:49.428257  8643 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:49.396338  8690 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:49.447273  8690 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:49.074649  1700 rpcz_store.cc:276] 0709 13:03:47.416260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.475314 (+ 59054us) service_pool.cc:224] Handling call
0709 13:03:47.480265 (+  4951us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:47.480615 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.480632 (+    17us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:47.481026 (+   394us) log.cc:844] Serialized 52 byte log entry
0709 13:03:47.731546 (+250520us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:47.757155 (+ 25609us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.782971 (+ 25816us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:03:48.842658 (+1059687us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b180037dce0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287 00007f70ce9ff208
0709 13:03:48.843105 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.072629 (+229524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.072710 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.072752 (+    42us) raft_consensus.cc:1685] finished
0709 13:03:49.072767 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.072914 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1428089,"prepare.queue_time_us":31245,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":70829,"raft.queue_time_us":1323842,"raft.run_cpu_time_us":2545,"raft.run_wall_time_us":313268,"spinlock_wait_cycles":9983104,"thread_start_us":146254,"threads_started":1,"wal-append.queue_time_us":368728}
I20260709 13:03:49.450150  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.450908  1700 raft_consensus.cc:2393] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 16: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:49.436028  8811 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:49.437110  8805 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:49.357080  8702 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:03:49.440587  8812 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:49.395002  8709 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:49.443862  8676 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.001s
I20260709 13:03:49.446506  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1506) took 1014 ms. Trace:
I20260709 13:03:49.419296  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:49.474184  1709 raft_consensus.cc:2376] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18. Re-sending same reply.
I20260709 13:03:49.475306  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:49.476823  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.359467  8471 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.478148  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:49.479347  2296 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:49.482205  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:49.483131  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.484673  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:49.486241  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:49.486868  2297 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:49.473984  2295 rpcz_store.cc:276] 0709 13:03:48.432344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.432533 (+   189us) service_pool.cc:224] Handling call
0709 13:03:49.446308 (+1013775us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54005e1854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:49.446447 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690414464}
I20260709 13:03:49.445040  8778 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.489324  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.491384  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:49.492244  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 136732us
I20260709 13:03:49.495141  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.495704  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.496946  1709 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:49.488022  8643 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:03:49.497781  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.498579  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:49.501339  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.502164  2296 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:49.518142  1628 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:49.518713  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.519603  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1296) took 1578 ms. Trace:
I20260709 13:03:49.521090  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.519992  2291 rpcz_store.cc:276] 0709 13:03:47.940791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.940915 (+   124us) service_pool.cc:224] Handling call
0709 13:03:47.941200 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.383997 (+442797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.519426 (+1135429us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400720954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:49.519553 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016027904}
I20260709 13:03:49.523794  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.524325  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:49.525300  1705 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:49.525942  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:49.516184  8471 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:49.526557  1700 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:49.506472  8725 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:49.515425  2296 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:49.497826  8719 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:49.539284  1700 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:49.549543  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:49.548578  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173726us
I20260709 13:03:49.492467  8709 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:49.530495  8819 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:49.535068  8676 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:49.549662  8643 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:49.507193  8778 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.489351  8812 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.555608  8471 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:49.575997  8471 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:49.578099  8471 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.581391  8709 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:49.583141  8708 consensus_queue.cc:1048] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:03:49.586645  8780 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.519s	user 0.004s	sys 0.000s
I20260709 13:03:49.562278  8643 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.593125  1560 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:49.604737  2290 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
W20260709 13:03:49.589037  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.608709  2289 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:49.586545  8690 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.550174  8719 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.605466  1556 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:49.610641  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:49.608565  8643 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.616310  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.617141  1554 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:49.617619  8613 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.618240  8723 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:49.617735  8729 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.618326  8613 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.619128  8729 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.620471  8613 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.621152  8729 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:49.619150  8812 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:49.626123  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:49.627121  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1167) took 1158 ms. Trace:
I20260709 13:03:49.627909  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:49.562724  8676 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.559068  8814 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:03:49.627555  1633 rpcz_store.cc:276] 0709 13:03:48.469012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.469138 (+   126us) service_pool.cc:224] Handling call
0709 13:03:48.469494 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:48.469840 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.469869 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:48.471803 (+  1934us) log.cc:844] Serialized 52 byte log entry
0709 13:03:48.741280 (+269477us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:49.004073 (+262793us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.004418 (+   345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.004534 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.585885 (+581351us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:49.626837 (+ 40952us) raft_consensus.cc:1685] finished
0709 13:03:49.626921 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.627059 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3685,"prepare.queue_time_us":411647,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":1546,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":21038,"spinlock_wait_cycles":174502144,"thread_start_us":502614,"threads_started":2,"wal-append.queue_time_us":385687}
I20260709 13:03:49.561022  8819 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:49.632525  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77610.7us
I20260709 13:03:49.634477  8721 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:49.634934  8721 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:49.635001  8819 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:49.635685  8730 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.636237  8730 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.634471  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.641644  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:49.556576  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3040) took 1555 ms. Trace:
I20260709 13:03:49.644659  1555 rpcz_store.cc:276] 0709 13:03:48.001286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.355397 (+354111us) service_pool.cc:224] Handling call
0709 13:03:48.355655 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.355872 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:48.375396 (+ 19524us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.437110 (+1061714us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b180044ce60. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:49.556146 (+119036us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.556203 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:49.556258 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.556367 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.556501 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19314,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":3032,"raft.queue_time_us":237,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3257,"spinlock_wait_cycles":8148864,"thread_start_us":118538,"threads_started":1,"wal-append.queue_time_us":1201894}
W20260709 13:03:49.654567  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.628232  8777 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
I20260709 13:03:49.668190  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.673574  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:49.637461  8661 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:49.674326  1555 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:49.661676  8808 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 13:03:49.667913  8799 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.371s	user 0.003s	sys 0.000s
I20260709 13:03:49.637027  8406 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:03:49.675344  8690 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.684670  8778 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:49.686950  8676 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:49.664083  8483 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:49.674692  8708 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:49.689338  8708 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:49.690351  8708 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.693835  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:49.694530  1633 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:49.696544  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1171) took 1225 ms. Trace:
I20260709 13:03:49.696916  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:49.696950  8822 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:49.697566  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.697923  1628 raft_consensus.cc:2393] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:49.698671  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1173) took 1203 ms. Trace:
I20260709 13:03:49.698798  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.699081  1632 rpcz_store.cc:276] 0709 13:03:48.495463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.495828 (+   365us) service_pool.cc:224] Handling call
0709 13:03:49.698510 (+1202682us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54001af9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:49.698618 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585798016}
W20260709 13:03:49.699988  8594 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.002s
I20260709 13:03:49.701562  8730 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.702587  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:49.702759  1627 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:49.703225  1700 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:49.700337  1705 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:49.696978  1631 rpcz_store.cc:276] 0709 13:03:48.470987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.471186 (+   199us) service_pool.cc:224] Handling call
0709 13:03:49.696389 (+1225203us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54002972d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:49.696488 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297285504}
W20260709 13:03:49.696574  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.627319  8818 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:03:49.675971  8661 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:49.671756  8811 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:49.724556  1708 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:49.679028  8816 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:49.697099  8676 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.723842  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.666898  8721 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.700426  8594 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.763698  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:49.727722  8818 consensus_queue.cc:1243] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:49.726583  8661 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:49.776304  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:49.726719  8806 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:49.777110  1631 raft_consensus.cc:2376] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18. Re-sending same reply.
I20260709 13:03:49.777782  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:49.765339  8406 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:03:49.765153  8721 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:49.791918  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.732203  8816 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.733554  8676 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.832598  8719 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:49.833349  8729 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:49.881011  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:49.881896  1632 raft_consensus.cc:2376] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22. Re-sending same reply.
I20260709 13:03:49.883625  2297 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 13:03:49.834956  1523 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.846s (SENT)
I20260709 13:03:49.840101  1451 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 15, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 15 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:49.771634  8664 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
W20260709 13:03:49.844719  8781 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.773s	user 0.006s	sys 0.000s
I20260709 13:03:49.849321  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.850157  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:49.920912  2288 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:03:49.850450  2292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1583) took 1188 ms. Trace:
I20260709 13:03:49.922456  2292 rpcz_store.cc:276] 0709 13:03:48.662296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.665877 (+  3581us) service_pool.cc:224] Handling call
0709 13:03:48.666169 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.126005 (+459836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.850198 (+724193us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:49.850386 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055348096}
I20260709 13:03:49.851996  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.925081  2293 raft_consensus.cc:2410] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:49.852321  2296 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:49.852938  8643 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:49.854012  8470 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.928831  8470 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:49.928901  8643 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:49.778847  8791 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.803510  8682 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:49.930437  8682 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.930994  8682 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.895291  1556 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:49.932366  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:49.932964  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:49.897105  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:49.935159  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:49.935835  1558 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:49.934672  1556 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:49.901571  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:49.918869  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.936916  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 101990us
I20260709 13:03:49.864075  8594 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:49.937546  1523 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.857s (SENT)
I20260709 13:03:49.938004  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:49.939783  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:49.940616  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:49.941468  1552 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 21 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:03:49.886340  8732 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:49.942276  8823 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:03:49.915982  8738 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:49.914876  8729 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:49.914634  8730 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:49.936269  8723 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:49.959546  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 180826us
I20260709 13:03:49.953873  8612 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:49.958544  8819 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:49.959143  8738 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.962000  8730 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.961613  8723 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.002269  8612 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:49.955780  8732 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:50.030905  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 129452us
W20260709 13:03:50.035058  8830 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:50.035456  8612 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:50.041357  8682 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:49.946928  8811 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.060431  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.064388  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.062785  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:50.072122  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 134592us
I20260709 13:03:50.008870  8729 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.076155  8729 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.076325  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:50.077111  1554 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:50.061486  8807 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.000s
W20260709 13:03:50.080570  8785 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.013s	user 0.004s	sys 0.005s
I20260709 13:03:50.080292  8807 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.082804  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2584) took 1603 ms. Trace:
I20260709 13:03:50.083176  1551 rpcz_store.cc:276] 0709 13:03:48.481697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.481885 (+   188us) service_pool.cc:224] Handling call
0709 13:03:48.482366 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.484444 (+  2078us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:48.484486 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.540092 (+ 55606us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.694690 (+154598us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:48.694734 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.694812 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.079973 (+1385161us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:50.082515 (+  2542us) raft_consensus.cc:1685] finished
0709 13:03:50.082537 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.082740 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1618,"raft.queue_time_us":1957,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9643,"spinlock_wait_cycles":166200064,"thread_start_us":154230,"threads_started":1,"wal-append.queue_time_us":277236}
I20260709 13:03:50.088433  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:50.090973  1551 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.076575  8682 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.059715  8676 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:50.075217  8730 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.026149  8661 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.102522  8723 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.107899  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:50.109459  8838 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:50.108239  8690 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.130108  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.131242  1558 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
W20260709 13:03:50.140784  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 108261us
W20260709 13:03:50.114746  8715 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.003s	sys 0.006s
I20260709 13:03:50.141670  1551 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:50.141692  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:50.115289  8807 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:50.150452  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68979.5us
W20260709 13:03:50.169876  1523 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.004s (SENT)
I20260709 13:03:50.170862  8811 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:50.171416  8836 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:50.171595  8811 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.172461  1558 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:03:50.171074  1523 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.062s (SENT)
I20260709 13:03:50.174763  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:50.150703  8715 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.003s	sys 0.007s
I20260709 13:03:50.139070  8690 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.187750  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:50.234025  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.186069  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1161) took 1477 ms. Trace:
I20260709 13:03:50.252359  1553 rpcz_store.cc:276] 0709 13:03:48.709051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.709250 (+   199us) service_pool.cc:224] Handling call
0709 13:03:48.709592 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.720113 (+ 10521us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:48.739484 (+ 19371us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.827461 (+1087977us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b1800377920. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce58423a 00007f70ce9eea2a
0709 13:03:50.014380 (+186919us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.032244 (+ 17864us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.032525 (+   281us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:50.032587 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.032736 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.185764 (+153028us) raft_consensus.cc:1685] finished
0709 13:03:50.185866 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.186005 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19113,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":27327,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":1662,"spinlock_wait_cycles":53448576,"thread_start_us":145539,"threads_started":1,"wal-append.queue_time_us":1392800}
I20260709 13:03:50.251992  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:50.227094  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.211931  8822 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.273782  8690 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.273694  8822 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.274566  8690 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.275131  8690 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.242455  8721 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:50.271047  8710 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:50.226272  8471 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.272686  8661 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.192710  8684 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:50.185616  8816 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.181700  8676 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.180172  8811 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:50.280901  8721 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.254456  8732 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.293023  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:50.294298  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 123252us
W20260709 13:03:50.288730  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.318492  2290 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:50.287523  8710 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:50.319978  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 178360us
I20260709 13:03:50.325259  1554 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:50.296779  8684 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:50.317098  8721 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:50.315179  8822 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:50.327081  8710 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.297420  8471 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.361905  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.363853  1631 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:50.314546  8676 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.374743  8643 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:03:50.376123  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.376377  8643 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:50.376868  8643 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:50.377898  8643 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.380895  8471 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.381263  1631 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:50.381917  2295 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:50.382783  8865 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:50.382972  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:50.383311  8865 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.383635  2294 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:50.383709  8865 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.387141  8863 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:50.387965  8863 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:50.388147  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:50.381285  2291 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:50.388517  8863 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.389671  8690 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:50.389840  8862 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:03:50.391125  8863 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.391405  8862 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:50.391858  8862 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:50.392474  8777 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:50.392824  8862 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.393903  8862 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:50.393517  8643 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.395902  8684 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.397455  8684 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:50.399214  8729 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.400229  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:50.402405  8856 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:03:50.404186  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:50.404971  1626 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:50.406482  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.408072  8710 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.410874  8721 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.411517  8405 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:50.413470  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:50.414938  8850 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:50.399699  8729 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.418116  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.437546  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:50.438462  8834 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:03:50.438917  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:50.439101  8405 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:50.439478  1628 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:03:50.421222  8853 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:03:50.404237  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:50.423594  8840 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:50.418131  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:50.392867  8777 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:50.427860  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:50.428645  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.429682  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:50.430156  1634 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:50.463615  1629 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:50.439227  1633 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:50.420082  8861 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:03:50.439680  8405 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:50.328863  8719 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:03:50.439812  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.491369  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.492511  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.492238  8405 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.493172  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1178) took 1214 ms. Trace:
I20260709 13:03:50.493676  8405 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.494992  8806 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.495712  8806 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:50.442066  8729 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:50.522709  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 94947.2us
I20260709 13:03:50.436172  8231 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.442600  8850 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.437563  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49515.1us
I20260709 13:03:50.545245  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:50.458719  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.546680  1631 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
W20260709 13:03:50.547289  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.366631  8822 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.368312  8709 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.463289  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.472292  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:50.475169  8710 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.477398  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:50.548763  1551 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.478765  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:50.553016  2294 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:50.479303  8721 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.487704  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:50.489549  8864 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.119s	user 0.003s	sys 0.000s
I20260709 13:03:50.461364  8791 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.493590  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.440448  8471 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:50.440613  1632 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:50.496681  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:50.497910  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.558102  1629 raft_consensus.cc:2410] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 21 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:03:50.499104  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.498878  1627 rpcz_store.cc:276] 0709 13:03:49.279050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.279620 (+   570us) service_pool.cc:224] Handling call
0709 13:03:49.702451 (+422831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.703243 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.492901 (+789658us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54001af9d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:50.493094 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971466112}
I20260709 13:03:50.558722  1626 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:03:50.559080  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.559602  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.560606  1626 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:50.500870  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.501929  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:50.522853  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.561765  1557 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:50.526139  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:50.459879  8837 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.562498  1551 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
W20260709 13:03:50.568697  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66848.9us
I20260709 13:03:50.579113  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:50.579810  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83256.8us
I20260709 13:03:50.574934  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.580888  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:50.601687  1628 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:50.544593  8714 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:50.620393  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:50.636242  1633 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:50.637461  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58473.2us
I20260709 13:03:50.544663  8578 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.625304  1627 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:03:50.556691  8729 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:50.628434  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 102396us
I20260709 13:03:50.532133  8732 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.656458  2263 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:50.651880  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:50.572013  8739 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.632059  8861 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:03:50.671634  8850 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.672397  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.673697  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.674273  1555 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:50.675094  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.556983  8822 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.677615  8822 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:50.678364  1551 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:50.629789  8723 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:50.680311  8723 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.680706  8723 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.655428  8791 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:50.684199  8791 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.684563  8791 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:50.684811  8676 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:50.689479  8865 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.654418  8867 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:03:50.665480  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.616633  8769 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:50.698431  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:50.552125  8564 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 13:03:50.653276  8729 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.602169  8863 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.697852  1555 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:50.700024  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:50.705431  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:50.712111  1557 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.549185  8231 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.641227  8406 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:50.553454  8709 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:50.719269  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.722059  1553 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:50.664050  8714 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:50.667850  8819 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.662936  8826 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.713613  8729 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.729398  1559 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:50.569716  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:50.695442  8843 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:50.733726  1629 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:50.705717  8719 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.664500  8578 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.748255  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108532us
I20260709 13:03:50.626191  8837 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.719937  8676 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:50.668262  8732 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.723273  8710 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:50.729331  8406 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:50.702996  8739 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.761140  1630 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:50.740991  8865 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:50.721861  8822 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 13:03:50.752380  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 182772us
I20260709 13:03:50.722280  8594 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 13:03:50.767795  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:50.603539  8862 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.722892  8863 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.750918  8871 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:50.771315  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:50.771122  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:50.727123  2294 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:03:50.756839  8676 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.777493  1552 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:50.727550  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.779778  1551 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:50.789374  1557 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:50.790832  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:50.780831  1558 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:50.798228  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:50.793326  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:50.744937  8826 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:50.750950  8719 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.768530  8710 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:50.804782  8690 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:50.767380  8843 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:03:50.904449  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 113710us
I20260709 13:03:50.808035  8626 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:50.769805  8721 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.767617  8816 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:03:50.795073  8676 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:50.776466  8872 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:50.918731  1552 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:50.802138  8859 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:50.923868  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:50.788661  8862 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:50.912015  8865 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.903972  8875 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:50.907088  8470 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.929190  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.042s	sys 0.042s
I20260709 13:03:50.940490  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:50.941484  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:50.945022  1631 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.945087  1557 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:50.926599  8626 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.795181  8714 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.947914  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 303456us
I20260709 13:03:50.952597  1601 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:50.946538  1556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.017s	sys 0.037s
W20260709 13:03:50.950328  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 368172us
W20260709 13:03:50.974678  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 320695us
I20260709 13:03:50.932521  8721 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:50.976217  1556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.018s	sys 0.038s
I20260709 13:03:50.977545  1557 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:50.983553  1631 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
W20260709 13:03:50.945534  8625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.010s	sys 0.017s
W20260709 13:03:50.944619  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.043s	sys 0.043s
I20260709 13:03:50.951450  8626 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.906708  8643 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:50.939203  8875 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:50.987313  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:51.021293  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:50.923672  8406 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.028492  1626 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:51.016017  1631 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:51.032850  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.033932  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:51.034530  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 80837us
I20260709 13:03:51.016846  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.017719  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:51.032799  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56675.9us
W20260709 13:03:50.990976  8625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.017s
I20260709 13:03:50.986307  8726 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:50.963524  1702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.036s
W20260709 13:03:51.118247  1702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.006s	sys 0.038s
W20260709 13:03:51.037043  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.025s
W20260709 13:03:51.119165  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.000s	sys 0.025s
I20260709 13:03:51.039600  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:50.948258  8470 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.112742  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.123694  1626 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:51.114534  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.040457  8875 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:50.943009  8859 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.107870  8578 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.110146  8471 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:51.126820  8826 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.019870  8564 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 13:03:51.111362  8231 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.142760  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:51.123266  8709 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.123191  8837 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.127885  8627 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:03:51.142400  8684 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.002s
I20260709 13:03:51.139168  8732 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.172926  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.173759  1634 raft_consensus.cc:2410] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:03:51.175326  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 139690us
I20260709 13:03:51.176548  8231 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:51.177583  8837 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:51.178021  8837 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:51.156226  8578 raft_consensus.cc:2764] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:03:51.178795  8837 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.159382  8802 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:03:51.181270  8791 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.159574  8826 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:03:51.182610  8837 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:51.183457  8791 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.184217  1631 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:51.181558  8802 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:51.185242  8802 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:51.186100  8802 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.187247  8802 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:51.191064  8676 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:51.195717  2262 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.819s (SENT)
I20260709 13:03:51.197956  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3048) took 1297 ms. Trace:
I20260709 13:03:51.007046  8710 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.198506  1560 rpcz_store.cc:276] 0709 13:03:49.900627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.935534 (+ 34907us) service_pool.cc:224] Handling call
0709 13:03:49.935827 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.023881 (+ 88054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.023927 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.023985 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:50.024030 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.197708 (+1173678us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:51.197766 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.197896 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201891712}
I20260709 13:03:51.202564  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:51.155076  8471 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:51.224460  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2639) took 2671 ms. Trace:
I20260709 13:03:51.222041  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:51.228770  2296 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.227919  1702 rpcz_store.cc:276] 0709 13:03:48.553251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.556385 (+  3134us) service_pool.cc:224] Handling call
0709 13:03:48.556752 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.013602 (+456850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.013634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.406846 (+1393212us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b1800452980. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8c52a6 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a
0709 13:03:50.407501 (+   655us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b2c000b1c58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:03:50.408072 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.118803 (+710731us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:51.118875 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.118954 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.224131 (+105177us) raft_consensus.cc:1685] finished
0709 13:03:51.224156 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.224388 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":587741,"raft.queue_time_us":459964,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10786,"spinlock_wait_cycles":24782848,"thread_start_us":710277,"threads_started":1,"wal-append.queue_time_us":747836}
W20260709 13:03:51.196779  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 397490us
W20260709 13:03:51.193643  8806 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.672s	user 0.005s	sys 0.010s
W20260709 13:03:51.281181  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 78674.5us
I20260709 13:03:51.281471  2294 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:03:51.150717  8816 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:03:51.278067  8625 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.344545  8625 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:51.346298  8785 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.419s	user 0.002s	sys 0.010s
I20260709 13:03:51.347311  8625 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.276068  8648 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.284446  8684 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:51.286357  8777 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.393936  8859 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.394188  8777 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.395533  8890 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:51.289811  2296 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:51.297586  8676 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.306339  8715 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:51.308743  8470 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.313185  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.313833  8739 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:51.319581  8870 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.523s	user 0.001s	sys 0.000s
W20260709 13:03:51.330272  2262 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.549s (SENT) [suppressed 10 similar messages]
I20260709 13:03:51.332942  8721 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.409555  8872 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:51.333662  8862 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:51.337312  8819 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
W20260709 13:03:51.221531  8883 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:03:51.410333  8738 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:51.338337  8723 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.657s	user 0.014s	sys 0.007s
I20260709 13:03:51.202289  8719 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.339352  8837 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.229638  8690 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:51.340785  8806 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.351169  8863 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.353698  8627 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:51.357666  8865 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.364415  8471 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.413668  8819 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:51.193288  8879 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:03:51.374631  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 109945us
I20260709 13:03:51.392266  8648 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.393106  8684 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:51.331573  8405 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.837s	user 0.005s	sys 0.005s
I20260709 13:03:51.410655  8872 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.339841  8709 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:51.406072  8715 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:51.459893  8729 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.000s
I20260709 13:03:51.459805  8709 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.460485  8729 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.408147  8578 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.462230  8709 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.462472  8729 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.459463  8715 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.461908  8578 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.457610  8690 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:51.370561  8874 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.420s	user 0.000s	sys 0.003s
I20260709 13:03:51.436093  8627 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:51.415107  8723 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.431320  8837 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.454449  8684 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:51.468187  8715 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.492130  8875 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.493362  8837 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.499126  8875 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.530982  1635 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:51.414455  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 377389us
I20260709 13:03:51.537165  1557 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:51.536974  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:51.540537  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 553133us
W20260709 13:03:51.542721  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.531117  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.019s	sys 0.034s
W20260709 13:03:51.543890  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.544288  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.020s	sys 0.035s
I20260709 13:03:51.546051  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:51.550868  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1173) took 1199 ms. Trace:
I20260709 13:03:51.552872  1555 rpcz_store.cc:276] 0709 13:03:50.346618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.725248 (+378630us) service_pool.cc:224] Handling call
0709 13:03:50.725516 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.725708 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:03:50.759986 (+ 34278us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.545566 (+785580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.545730 (+   164us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:03:51.545887 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.546050 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.546380 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25598,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1673,"thread_start_us":784669,"threads_started":1,"wal-append.queue_time_us":722576}
I20260709 13:03:51.556798  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:51.427676  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.568672  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:51.535315  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 611356us
I20260709 13:03:51.570850  1523 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:51.573401  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.573745  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.573936  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:51.567378  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1589) took 1188 ms. Trace:
I20260709 13:03:51.574616  2295 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:51.574944  2290 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:51.577031  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.577409  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:51.616762  2289 rpcz_store.cc:276] 0709 13:03:50.379174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.382794 (+  3620us) service_pool.cc:224] Handling call
0709 13:03:50.383153 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.383564 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.383600 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.384167 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.747151 (+362984us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:50.747202 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.747340 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.566726 (+819386us) raft_consensus.cc:1685] finished
0709 13:03:51.566770 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.567082 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3783,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"thread_start_us":362569,"threads_started":1,"wal-append.queue_time_us":563261}
I20260709 13:03:51.618726  2296 raft_consensus.cc:2393] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:51.580793  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.582535  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:51.583686  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.619920  1634 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:51.584784  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:03:51.620388  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46719.6us
I20260709 13:03:51.585422  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:51.619926  1630 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:51.621685  1632 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:51.622143  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:51.586476  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.624497  2291 raft_consensus.cc:2410] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:03:51.586479  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:51.590183  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:51.625901  1552 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:51.626995  8648 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.592178  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:51.627812  8648 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:51.592950  8406 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.628962  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:51.629104  8406 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.593032  8405 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:51.632066  8405 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.594002  8470 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:51.598819  8564 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:51.636525  8470 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.637270  8564 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:51.601356  8578 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.603125  8612 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:51.641438  8406 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.639611  8470 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.642179  8578 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:51.642699  8470 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.607255  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:51.643532  8470 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.644626  1556 raft_consensus.cc:2410] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:03:51.646201  1556 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:51.646234  8470 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.646929  8470 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.540540  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:51.609138  1554 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:51.611855  8626 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.009s
W20260709 13:03:51.615870  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.475385  8726 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:51.667747  1552 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:51.657892  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:51.687451  8709 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.653045  8578 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:51.825359  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 285179us
I20260709 13:03:51.825948  8897 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:51.822914  8709 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.826941  8897 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.681246  8682 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:51.736140  8806 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:51.702630  8722 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:51.635748  8405 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.737056  8732 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:51.715539  8730 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:51.666404  8661 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:51.846060  8709 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.702699  8719 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:51.848598  8895 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.820983  8626 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.851879  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3173) took 1420 ms. Trace:
I20260709 13:03:51.853464  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2610) took 1089 ms. Trace:
I20260709 13:03:51.673065  8676 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.647652  8470 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.716729  8723 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:51.703154  8721 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:51.898365  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 71233.5us
I20260709 13:03:51.642208  8648 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:51.841148  8682 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:51.728852  8684 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.909257  1558 rpcz_store.cc:276] 0709 13:03:50.763956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.797479 (+ 33523us) service_pool.cc:224] Handling call
0709 13:03:50.800947 (+  3468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.904131 (+103184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.904165 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.904211 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:50.904262 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.852852 (+948590us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:51.852932 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.853235 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18936064}
I20260709 13:03:51.908782  1635 rpcz_store.cc:276] 0709 13:03:50.431525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.487335 (+ 55810us) service_pool.cc:224] Handling call
0709 13:03:51.851565 (+1364230us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b5400280854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:51.851750 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238468992}
I20260709 13:03:51.897008  8661 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:51.948911  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 346895us
I20260709 13:03:51.913543  8405 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.934240  8578 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:51.972880  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 384576us
I20260709 13:03:51.948869  8844 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:51.956418  8612 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:51.911864  8709 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:03:51.952869  8869 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:51.981621  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:51.945699  8719 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:51.963265  8891 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.986099  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.004s	sys 0.016s
I20260709 13:03:51.977077  8897 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:51.974620  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:51.986747  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:51.982939  8714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.014s	sys 0.020s
W20260709 13:03:52.003499  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.004s	sys 0.016s
W20260709 13:03:51.987082  8610 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.024s
I20260709 13:03:51.964804  8895 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:52.008718  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:52.010195  8610 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.004s	sys 0.024s
W20260709 13:03:52.010391  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.012665  8626 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.013617  8626 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:52.014714  8626 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:52.014921  8877 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.015671  8231 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:52.016010  8877 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.016518  8231 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.018846  8877 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:52.020610  8887 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:52.020633  8684 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.021210  8769 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.022300  8859 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:51.995563  8710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.016s	sys 0.021s
I20260709 13:03:51.981719  8863 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:52.022034  8769 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.040091  8891 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.012594  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.041656  8769 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.041886  8889 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:52.042469  8889 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:52.042790  8891 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.007604  8405 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.043632  8889 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.043983  8405 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.045056  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:52.046562  8405 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.047508  8872 consensus_queue.cc:1059] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:52.049202  8872 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:52.049618  8872 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.049705  1554 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:03:52.050037  8872 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:52.050357  1451 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 21, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:52.007742  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 379930us
I20260709 13:03:52.056444  1451 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 17, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 17 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:52.027019  8730 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:52.047431  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.015127  8626 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:52.031391  8895 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.033458  8690 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:52.036408  8661 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.038307  8470 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:52.009569  8714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.015s	sys 0.021s
W20260709 13:03:52.039444  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 760739us
W20260709 13:03:52.071947  2262 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.021s
I20260709 13:03:52.051959  8844 raft_consensus.cc:3055] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:52.029163  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1226) took 4420 ms. Trace:
W20260709 13:03:52.083138  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 388463us
W20260709 13:03:52.083565  2262 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.010s	sys 0.021s
I20260709 13:03:52.089068  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.091926  8471 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:52.092423  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.092595  8471 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.094452  8722 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:52.094650  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.095427  8722 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.095642  1701 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:52.096190  8722 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.098896  8722 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:52.100036  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 132157us
I20260709 13:03:52.101046  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.102555  8626 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.104285  8626 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.998943  8676 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:52.060797  8710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.017s	sys 0.021s
W20260709 13:03:52.104169  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 100829us
I20260709 13:03:52.084846  8648 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.080540  1706 rpcz_store.cc:276] 0709 13:03:47.608435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.881328 (+272893us) service_pool.cc:224] Handling call
0709 13:03:47.881754 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.963111 (+1081357us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:50.193264 (+1230153us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b18004923a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:03:50.436648 (+243384us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.547204 (+1110556us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.548381 (+  1177us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.548626 (+   245us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:51.548665 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.548844 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.023555 (+474711us) raft_consensus.cc:1685] finished
0709 13:03:52.028669 (+  5114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.028869 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1164245,"prepare.queue_time_us":21388,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":1305977,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":2423,"spinlock_wait_cycles":9030400,"thread_start_us":1336413,"threads_started":3,"wal-append.queue_time_us":736697}
I20260709 13:03:52.164640  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.105556  8661 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:52.129643  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 799346us
I20260709 13:03:52.058934  1451 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 16, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.166829  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.071633  8690 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:52.184648  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.166327  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 508618us
I20260709 13:03:52.112957  1702 pending_rounds.cc:85] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
W20260709 13:03:52.166023  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 200367us
I20260709 13:03:52.183492  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.143841  8877 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.125458  8578 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.079996  8844 raft_consensus.cc:740] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:52.095922  8769 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:52.193722  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.109627  8714 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:52.186226  1702 pending_rounds.cc:107] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 18.13
W20260709 13:03:52.125286  8627 consensus_meta.cc:287] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.535s	user 0.014s	sys 0.002s
I20260709 13:03:52.117867  8863 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:52.113562  8564 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.223384  8564 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.223906  8564 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.229293  8710 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:52.203758  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.197650  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.108183  8682 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.206154  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.232228  8682 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:52.232342  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:52.232923  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.198722  1552 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.235360  8877 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.237633  8877 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.209959  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.108995  8730 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.209824  1451 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 15, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.242048  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.241948  1451 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 17, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.177098  8231 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.243904  8843 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.244133  8676 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:52.244428  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.244709  8843 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:52.245175  8843 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.121985  8726 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.245595  8843 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.071683  8612 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.246978  8406 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:52.204263  8471 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.247891  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:52.249418  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 291) took 62986 ms (client timeout 15000 ms). Trace:
I20260709 13:03:52.250082  8612 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.252910  8714 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.253367  8714 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.251573  8406 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:52.253978  8406 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:52.255088  8406 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.207842  1702 raft_consensus.cc:2889] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.256983  8406 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:52.212020  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.212301  8690 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.213426  1560 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:52.214613  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.216269  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.264245  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.264951  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:52.265974  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.216879  1556 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.217903  8844 consensus_queue.cc:260] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.220788  8769 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:52.221544  8627 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.222770  8863 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.123984  8719 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:52.272369  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.272888  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.273551  1701 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:52.249953  1458 rpcz_store.cc:269] 0709 13:02:49.172377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.172939 (+   562us) service_pool.cc:224] Handling call
0709 13:03:52.147694 (+62974755us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:41.074559 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:41.075669 (+  1110us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:41.075730 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:03:41.075801 (+    71us) tablet.cc:662] Decoding operations
0709 13:03:41.076902 (+  1101us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:41.077028 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:41.077079 (+    51us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:03:41.077886 (+   807us) tablet.cc:701] Row locks acquired
0709 13:03:41.077932 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:03:41.078185 (+   253us) write_op.cc:270] Start()
0709 13:03:41.078396 (+   211us) write_op.cc:276] Timestamp: P: 1783602221078103 usec, L: 0
0709 13:03:41.078439 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:03:41.088994 (+ 10555us) log.cc:844] Serialized 2152 byte log entry
0709 13:03:42.341782 (+1252788us) op_driver.cc:464] REPLICATION: finished
0709 13:03:42.759068 (+417286us) write_op.cc:301] APPLY: starting
0709 13:03:42.759184 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:42.830548 (+ 71364us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:42.830678 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:42.833659 (+  2981us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:42.833981 (+   322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:03:42.834045 (+    64us) write_op.cc:312] APPLY: finished
0709 13:03:42.834380 (+   335us) log.cc:844] Serialized 57 byte log entry
0709 13:03:42.834550 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:42.834660 (+   110us) write_op.cc:454] Released schema lock
0709 13:03:42.834733 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:42.924237 (+ 89504us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:42.927275 (+  3038us) write_op.cc:454] Released schema lock
0709 13:03:43.785156 (+857881us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e132a 00007f70ce966f14 00007f70ce966ec3 00007f70ce966e38 00007f70ce966df3 00007f70ce9660dc 00007f70ce98fe16 00007f70ce977cbb 00007f70ce96239d 00007f70ce965d75
Metrics: {"mutex_wait_us":3626205,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":401069,"apply.run_cpu_time_us":9461,"apply.run_wall_time_us":109344,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":242388,"prepare.run_cpu_time_us":35409,"prepare.run_wall_time_us":66223,"raft.queue_time_us":605867,"raft.run_cpu_time_us":23829,"raft.run_wall_time_us":1466179,"replication_time_us":1257604,"spinlock_wait_cycles":4240640,"thread_start_us":873380,"threads_started":4,"wal-append.queue_time_us":103193,"wal-append.run_cpu_time_us":110027,"wal-append.run_wall_time_us":2812951}]]}
I20260709 13:03:52.266564  1555 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:52.244756  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.271281  1552 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:03:52.275871  2262 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.689s (SENT)
I20260709 13:03:52.285169  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.285305  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.286774  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.286861  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.290693  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.272439  8844 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:52.292116  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.270454  8769 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:52.294184  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.294785  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.290535  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:52.280108  8863 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:52.302175  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.303685  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.299259  8710 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.309780  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.309685  8710 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.313218  8811 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:52.313808  8811 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:52.314744  8811 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.316131  8806 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.316794  8806 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.319399  8806 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.320118  1451 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.321967  8901 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.322067  1451 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:52.322597  8901 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.324021  8470 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.324681  8470 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.325271  8913 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.325865  8913 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.327072  8470 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.328322  8690 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:52.332321  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64991.2us
I20260709 13:03:52.334510  8892 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:52.334771  8803 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:52.335855  1631 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:03:52.335307  8803 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.340783  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.290817  8719 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.380335  1554 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:52.380380  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:52.380573  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 168700us
I20260709 13:03:52.384393  1556 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:52.343633  8710 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.388577  1701 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:52.341450  8803 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.385190  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.344702  8901 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:52.399469  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.349013  8714 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.413043  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.417680  1706 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:52.413815  1709 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:52.379112  8877 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:03:52.439459  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.369843  8843 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.445726  8803 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.446128  1451 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.451071  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.458662  8714 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:52.459013  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48662.5us
I20260709 13:03:52.436980  8901 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.443146  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.465560  1554 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:52.466591  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.467554  8627 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.461467  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:52.469866  2296 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.449612  1633 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:52.474411  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3182) took 1461 ms. Trace:
I20260709 13:03:52.474866  1633 rpcz_store.cc:276] 0709 13:03:51.012717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.018245 (+  5528us) service_pool.cc:224] Handling call
0709 13:03:51.107077 (+ 88832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.472322 (+1365245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.474049 (+  1727us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000de3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:52.474312 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3086635904}
I20260709 13:03:52.479259  1554 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:03:52.405160  8913 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:52.404258  8719 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.500880  8719 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.509655  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.517345  1556 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:52.508085  1554 pending_rounds.cc:85] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 16
I20260709 13:03:52.521178  1555 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:52.522022  1552 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:52.523275  1554 pending_rounds.cc:107] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:03:52.523762  1554 raft_consensus.cc:2889] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.499058  8564 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.527006  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.529839  2296 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:03:52.509132  8901 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.545751  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:52.543483  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 133045us
I20260709 13:03:52.529556  8803 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.552865  8803 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.557448  8564 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.558629  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:52.560775  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.561638  1552 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:52.546619  1553 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:52.542402  8901 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:52.580137  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.581265  1599 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.580610  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:52.583600  8714 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:52.583698  8914 raft_consensus.cc:2804] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:52.583911  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.584118  8714 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:52.584726  8714 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:52.585247  8912 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:52.585209  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.585945  8912 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.586372  8912 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:52.525622  8872 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.587749  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:52.584728  8914 raft_consensus.cc:697] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:52.591907  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43719.4us
I20260709 13:03:52.592137  8914 consensus_queue.cc:237] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.592959  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.594554  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.573521  8564 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:52.600132  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:52.602828  2296 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.603618  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.604427  1552 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:52.611456  1559 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:52.612977  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.615022  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.599081  8913 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:52.603106  8690 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:52.632246  8661 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.597355  8769 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.663944  1554 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:52.673171  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.674928  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.681087  1556 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:52.672055  8877 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.549045  1451 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:52.674989  8564 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.704535  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.710248  1554 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
W20260709 13:03:52.714075  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.712464  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:52.725405  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122402us
I20260709 13:03:52.667975  8863 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:52.724076  8626 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.008s
I20260709 13:03:52.735958  8777 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.736536  8777 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:52.737547  8682 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.004s
I20260709 13:03:52.738744  8777 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.739240  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:52.741792  8924 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:03:52.743629  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.744596  1706 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:52.746493  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.747073  1709 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:52.748059  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.748353  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.748824  2290 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:52.749080  1700 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.749650  1551 raft_consensus.cc:2393] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:52.748072  1705 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:52.750270  8837 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:52.750767  8837 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:52.751744  8837 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.754947  8877 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.755887  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.756049  8564 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.756701  1708 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:52.758265  8564 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.759305  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.760443  1703 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:52.760672  1451 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.767020  1552 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:03:52.768674  2296 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:52.708412  8690 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.772706  1709 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:52.777061  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.783133  1560 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:03:52.791482  8690 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.792438  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.793045  2295 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:52.793877  8690 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:52.794009  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.794605  1709 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:52.794946  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:52.777349  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.796031  8889 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:52.796592  1703 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.796669  8889 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.798889  8889 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.800115  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.800851  1635 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:52.805716  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:52.811383  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.768163  8843 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.816078  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.817472  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.821422  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.822079  1707 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:52.828722  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.829437  1635 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:03:52.829762  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.830120  8690 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:52.831277  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.835819  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:52.836889  1709 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:52.842530  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.843283  1556 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:52.850940  1554 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:52.852736  1700 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:52.853229  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:52.855021  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.856137  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:52.856843  1553 raft_consensus.cc:3060] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.857635  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:52.773205  8719 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 13:03:52.839416  8406 consensus_meta.cc:287] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.005s
I20260709 13:03:52.771387  8661 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.861842  1706 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:03:52.873181  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.861627  1558 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:52.876245  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.877079  2290 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:52.870217  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.873816  2297 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:52.851819  8684 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:03:52.879380  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.879575  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.879966  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:52.872397  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.880904  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:52.882381  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.875864  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.890316  1631 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:52.892160  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:52.892426  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.893116  1630 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:03:52.893182  2290 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:03:52.905822  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.901012  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.871593  8872 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:52.913268  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:52.839972  8564 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:03:52.846014  8723 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:03:52.928565  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:52.926553  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.932655  1634 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:52.935002  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:52.936937  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:52.941854  1632 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:52.944365  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:52.944972  1706 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:52.923902  8710 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:52.918802  8690 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:52.949206  1700 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:52.942076  8627 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:52.962586  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:52.963182  1706 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.944144  8709 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:52.969897  8709 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:52.970273  8932 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:52.970377  8709 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:52.971228  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 111573us
I20260709 13:03:52.971716  1630 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.972172  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.973591  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:52.974810  1451 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 16, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.976799  1451 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 14, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 14 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:52.978067  1706 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:52.978447  1451 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 2 to 19, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:52.980332  8709 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:52.981534  1700 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:52.981523  8877 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:52.961264  8930 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:03:53.015887  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.016872  2288 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:03:52.950991  8690 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.018060  8690 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:52.980353  8928 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:53.019964  1451 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 20, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 20 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.022094  1451 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 16, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.950635  8406 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:53.025686  8406 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:53.032545  8936 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:03:53.041316  8564 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:53.042132  8564 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:52.942528  8811 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:03:53.044308  8811 consensus_queue.cc:1243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:53.008939  8661 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.026886  8406 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.010769  8863 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.009953  8719 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:53.059350  1553 raft_consensus.cc:2468] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
W20260709 13:03:53.066808  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 332759us
I20260709 13:03:52.998116  8803 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.068s
W20260709 13:03:53.108868  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69421.3us
I20260709 13:03:53.112510  8811 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.113096  8811 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.073942  8406 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.118283  1553 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:03:53.121141  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:53.077147  8892 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:53.072096  8714 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.149991  1630 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:53.143157  8564 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.142887  8709 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:53.174640  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.177884  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:53.184490  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 210979us
W20260709 13:03:53.187207  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.185488  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 232408us
W20260709 13:03:53.189540  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 192508us
I20260709 13:03:53.192736  8709 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:53.193377  8943 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:53.194304  8578 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.197611  8769 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:53.203212  8901 consensus_meta.cc:287] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
W20260709 13:03:53.206933  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86714.5us
W20260709 13:03:53.209106  8931 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.573s	user 0.002s	sys 0.000s
I20260709 13:03:53.209858  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:53.211422  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 361953us
W20260709 13:03:53.211619  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.192186  1675 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:53.212119  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:53.214604  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55029.9us
I20260709 13:03:53.180747  8811 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.215623  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:53.166471  8710 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:53.115130  1451 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 10 to 15, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 15 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:53.226634  1599 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:53.211144  8942 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:03:53.233687  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:53.228405  8863 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.240497  8946 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:53.203660  8901 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.241633  1674 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:53.212045  8892 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:03:53.233062  8661 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:53.250813  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 75583.6us
I20260709 13:03:53.250852  8956 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:53.233582  8811 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:53.274412  8811 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.275041  8811 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:53.275931  8944 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:03:53.277740  8953 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:53.276868  1451 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.278832  8958 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:53.279382  8958 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.238796  8564 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:53.242367  8863 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.282236  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:53.282886  8940 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.239s	user 0.002s	sys 0.000s
I20260709 13:03:53.283885  8958 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:53.283982  8889 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:53.228600  8769 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:53.285249  8769 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.286962  8714 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.235740  8949 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 13:03:53.238430  8723 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:53.288264  8723 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:53.289322  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 47818.3us
I20260709 13:03:53.289407  8723 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.290851  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:53.288286  8949 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:53.292199  8913 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:53.269945  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:53.294469  1631 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.266841  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.295089  2296 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
W20260709 13:03:53.276975  8950 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:53.235457  8933 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:03:53.307512  8769 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.308992  8913 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:53.303365  8863 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.297184  8777 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 13:03:53.296756  8956 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:53.306617  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:53.314213  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.318282  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:53.262035  8661 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.321805  8901 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.326577  1632 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:53.278967  1451 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:53.333252  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.292236  8949 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:53.334606  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.339500  1555 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:53.339913  2295 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:53.336485  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:53.336839  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:53.347477  1551 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:53.352743  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.356572  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:53.360257  1633 raft_consensus.cc:2410] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
W20260709 13:03:53.323704  8912 consensus_meta.cc:287] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.001s
I20260709 13:03:53.332568  8661 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:53.368110  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:53.373065  8872 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.374285  2295 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:53.374861  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.316606  8877 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:03:53.319597  8951 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:03:53.356753  8937 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.327896  8913 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:53.375751  8872 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.384471  8956 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:53.394060  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76507.4us
I20260709 13:03:53.342300  8690 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.360877  8610 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
W20260709 13:03:53.412781  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 122131us
I20260709 13:03:53.365294  8912 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:53.419876  8949 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:53.414710  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 77966.8us
I20260709 13:03:53.421250  8949 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:53.421675  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:53.392277  8710 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.430265  8610 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:53.430721  8610 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:53.408505  8564 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:53.432096  8564 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.432437  8564 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:53.418267  8811 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.396476  8937 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.436122  8682 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:53.436718  8682 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:53.437572  8682 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.440652  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:53.442319  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:53.453404  8676 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:53.453902  8676 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.454286  8676 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:53.455873  8872 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.459353  8967 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:53.459849  8967 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:53.461550  8967 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.462806  8967 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:53.462567  8676 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.465178  8690 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.469416  8627 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:53.469935  8627 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:53.470866  8627 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.471361  8676 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.454296  8777 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:53.473361  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.473960  8627 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.474094  1552 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.474498  8578 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:53.475814  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:53.477592  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 119153us
I20260709 13:03:53.479749  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:53.477771  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.498437  1555 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.500139  2297 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:53.505641  1552 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:53.507606  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:03:53.519002  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 172295us
W20260709 13:03:53.520448  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 130078us
I20260709 13:03:53.521507  1631 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:53.523490  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.524119  1556 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:03:53.524550  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 103006us
I20260709 13:03:53.522606  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:53.522958  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:53.529703  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 52446.5us
I20260709 13:03:53.532037  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:53.520949  8952 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:53.527357  8837 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:53.528553  8913 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:53.544593  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.528975  8811 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.531471  8872 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:53.554579  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:53.529297  8690 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:53.546169  8952 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:53.557514  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 81839us
I20260709 13:03:53.553458  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:53.554250  8406 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:53.545337  8914 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:53.547863  8648 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:03:53.554250  8913 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:53.577848  8914 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:53.593492  8914 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:53.580252  8956 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:53.596179  8956 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.583940  8937 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.585672  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:53.581049  8726 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:03:53.587656  8406 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:53.589813  8958 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.563158  8952 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:53.581477  1674 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:53.582481  8964 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:53.592614  8913 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.562141  8872 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:53.563444  8661 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.601418  1555 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:53.607359  1632 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:53.611562  2297 raft_consensus.cc:1240] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 19. Ops: []
I20260709 13:03:53.633735  1556 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:53.578778  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.637424  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.641906  1552 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:53.643117  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.625620  8962 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:53.652064  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.640677  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:53.650014  1633 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:53.654160  1559 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:53.656281  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:53.656678  1552 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:53.658565  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.639885  8723 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:53.660812  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:53.657810  1601 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:53.648617  8690 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:53.668030  8912 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:53.669682  8837 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:53.670198  8837 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:53.671917  8937 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:53.672469  8937 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:53.668048  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 90898.4us
I20260709 13:03:53.664002  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:53.648242  8964 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:53.696930  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 142432us
I20260709 13:03:53.673406  8937 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.662487  8710 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:53.704947  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.664366  8863 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:53.711602  2297 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:53.709527  2293 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:53.692198  8714 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:53.722011  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.727174  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:53.722029  8610 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.728555  8610 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:53.733534  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:53.743966  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:53.725761  8710 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:53.725272  8863 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:53.746372  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.739513  8951 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:53.761888  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.762599  1553 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:53.737816  8714 raft_consensus.cc:697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:53.766237  1556 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:53.763945  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:53.766353  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.747906  8682 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:53.779681  1701 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:53.779125  8898 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:53.785382  8710 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.758051  8951 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:53.813002  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.818225  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.725174  8872 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:53.828183  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:53.818638  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59611.2us
I20260709 13:03:53.826825  8978 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.774226  8682 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:53.730178  8964 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.845072  1700 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:03:53.777347  8956 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.820652  8661 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.871366  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.833491  8958 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:53.846197  8682 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:53.908743  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:53.909392  1553 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:53.916402  2296 raft_consensus.cc:2410] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:53.911613  2297 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:53.827716  8406 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:03:53.917557  1701 raft_consensus.cc:2410] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:03:53.932762  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:53.842510  8978 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:53.940443  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67274.7us
I20260709 13:03:53.924953  8965 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:53.957759  1701 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:53.960213  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:53.961038  1701 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:53.965109  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:53.968415  1704 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:53.913166  8863 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:53.947062  8965 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:53.969777  1705 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:03:53.973575  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 148588us
I20260709 13:03:53.974460  1701 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:53.976600  1706 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:53.926573  8877 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:03:53.981230  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:53.983793  1706 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.986258  1633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:53.987118  1633 raft_consensus.cc:606] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:03:53.987504  1633 raft_consensus.cc:614] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:53.945436  8714 consensus_queue.cc:237] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
W20260709 13:03:53.995604  2260 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.885s (SENT)
I20260709 13:03:53.996201  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:53.972677  8610 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:03:54.001072  1708 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:53.934260  8951 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.037480  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.065506  1706 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:54.066639  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81594.2us
I20260709 13:03:54.066529  8843 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:54.085577  8933 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:54.100099  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.120208  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.121016  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
W20260709 13:03:54.120705  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 379638us
I20260709 13:03:54.121552  1702 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:54.123247  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 113094us
I20260709 13:03:54.093209  8877 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:54.106498  8843 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:54.134147  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:54.135052  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 139471us
I20260709 13:03:54.117009  8406 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.095979  8984 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:03:54.174463  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.134806  8964 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.173359  2260 leader_election.cc:336] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.658s (SENT)
I20260709 13:03:54.179389  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:54.181526  8981 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:03:54.133351  8877 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.169880  8661 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.188107  1700 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:54.169786  8684 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.232758  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.175617  8769 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:54.188860  8964 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.176641  8843 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.178571  8969 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.008s
I20260709 13:03:54.180151  8978 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.249655  1708 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:03:54.257517  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 84187.9us
I20260709 13:03:54.235857  8987 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:54.183904  8726 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:54.256564  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:54.237747  8406 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.240746  8937 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:54.300872  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.246068  8994 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:54.122354  8965 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.301817  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 127391us
I20260709 13:03:54.297847  8612 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:03:54.241217  8661 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.307972  8967 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.004s
I20260709 13:03:54.308410  9000 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:54.327982  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:54.240124  8877 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:54.308977  8837 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.638s	user 0.014s	sys 0.005s
I20260709 13:03:54.332691  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:54.248163  8769 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:54.235728  8564 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.803s	user 0.007s	sys 0.007s
W20260709 13:03:54.299216  8993 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:03:54.307476  1523 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.947s (SENT)
I20260709 13:03:54.313985  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:54.337991  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:54.331126  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 74647.9us
W20260709 13:03:54.328429  8952 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.002s
I20260709 13:03:54.340874  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.347407  1626 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:54.309852  8987 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:54.333238  2262 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.549s (SENT)
I20260709 13:03:54.351259  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.352962  1631 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:54.333354  2295 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.333983  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:54.316640  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.339387  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:54.340245  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:54.355779  2293 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:54.340879  1627 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:03:54.343878  1630 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 13:03:54.333686  8913 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:54.327227  8914 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.005s
I20260709 13:03:54.348773  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:54.337167  8877 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.338193  8769 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.245373  8684 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.360764  2296 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.357365  1630 pending_rounds.cc:85] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 16
W20260709 13:03:54.362949  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56424.9us
I20260709 13:03:54.363674  8627 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.373113  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.363159  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:54.380188  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:54.383286  1630 pending_rounds.cc:107] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:03:54.383680  1630 raft_consensus.cc:2889] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.384047  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.384805  1707 raft_consensus.cc:2368] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20.
I20260709 13:03:54.384635  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:54.386830  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 61617.2us
I20260709 13:03:54.387564  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.371671  8937 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:54.337689  8837 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.344460  8564 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.350322  8952 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.392923  1634 pending_rounds.cc:85] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 12
I20260709 13:03:54.399215  1634 pending_rounds.cc:107] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:03:54.399519  1634 raft_consensus.cc:2889] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.393702  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.392278  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:54.400303  1705 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:03:54.397794  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64590us
I20260709 13:03:54.346907  8726 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.405689  1709 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:54.405618  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.405985  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.389523  8965 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.386161  8690 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:54.389247  8710 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:54.388631  9000 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:54.419879  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.375823  8877 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.372551  8914 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.439344  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.714951  2293 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:54.715132  1705 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:54.432826  8710 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.468012  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:54.453681  8987 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.417156  8578 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:54.432089  1633 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:54.700562  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 315974us
W20260709 13:03:54.722364  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 302336us
I20260709 13:03:54.702751  2296 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:54.404247  8913 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:54.404125  9006 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:54.433174  1626 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:54.716408  2295 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:03:54.724800  1628 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:03:54.725106  8949 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:54.725436  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:54.726300  1704 raft_consensus.cc:2410] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:03:54.726441  1629 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:54.720245  8710 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:54.728128  9016 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:54.728681  9016 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.718247  8682 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.786s	user 0.013s	sys 0.009s
I20260709 13:03:54.736757  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:54.755100  1523 leader_election.cc:336] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.068s (SENT)
I20260709 13:03:54.758634  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.759032  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:54.737010  8999 log.cc:927] Time spent T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.348s	user 0.004s	sys 0.004s
I20260709 13:03:54.759846  1704 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:54.736212  9000 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.746344  8956 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.749219  8964 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.753350  8937 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.782632  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.789619  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:54.790208  2296 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:54.790393  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.783581  9006 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.795668  9006 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.798092  9006 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.772111  8406 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.801821  8877 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.803941  8769 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.805230  2290 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:03:54.810602  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.822974  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67889.4us
I20260709 13:03:54.832470  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.754930  8682 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.838764  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 115191us
I20260709 13:03:54.837894  1708 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:54.782580  9018 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.816284  8661 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.842005  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.848807  2296 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:54.827540  8877 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:54.862596  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.863442  1705 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:03:54.864481  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:54.864902  9008 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:54.864753  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:54.865521  1707 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:54.865732  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.866947  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.864801  9007 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:54.873106  9016 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.874851  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:54.875890  8777 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:54.876276  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:54.877161  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.881934  1559 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:03:54.876677  8777 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:54.879840  1553 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:54.880671  1635 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:54.875938  8933 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:03:54.837572  8952 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.879685  8937 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:54.777398  9000 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.849854  8837 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.878334  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.912400  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.913131  2296 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:03:54.913431  2293 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:54.913704  2296 pending_rounds.cc:85] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 13
I20260709 13:03:54.914081  2296 pending_rounds.cc:107] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:03:54.914417  2296 raft_consensus.cc:2889] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.889601  2295 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:54.881371  8769 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.890420  8949 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:54.891038  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:54.902488  8863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.014s	sys 0.014s
I20260709 13:03:54.893877  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:03:54.892557  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122271us
I20260709 13:03:54.897554  8709 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.900725  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.903982  9018 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:54.902964  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.878964  8682 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.908841  1707 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:03:54.933223  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:54.852487  8684 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.909973  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.910408  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.905355  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.805126  8956 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:54.935956  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.935923  2289 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:54.936280  1701 raft_consensus.cc:2393] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:54.922966  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 160572us
I20260709 13:03:54.924638  8949 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.941380  8690 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:54.942260  8949 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.942934  8913 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:54.943400  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:54.927392  8901 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:54.944514  8901 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:54.923300  8863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.015s	sys 0.015s
I20260709 13:03:54.945449  8901 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:54.945907  9000 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.948158  8956 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:54.949602  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:54.950263  1555 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:03:54.951094  8769 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:54.951894  9023 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:54.951627  8769 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.952582  9023 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.954119  8769 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.955004  9023 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:54.955853  9003 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:54.955024  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.958084  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:54.958909  9012 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:03:54.945953  1705 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:03:54.959035  1703 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:03:54.961120  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:54.962530  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:54.965498  1703 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:54.916103  8914 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:54.975344  1705 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:54.956499  9003 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:54.983381  1703 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:03:54.989372  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:54.990204  2293 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.897502  8877 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:54.993367  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 286898us
I20260709 13:03:54.993541  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:54.994463  2260 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 1.622s (SENT)
I20260709 13:03:54.998695  1559 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:54.991421  8682 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.992467  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.005496  1703 raft_consensus.cc:2376] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 25. Re-sending same reply.
W20260709 13:03:55.012984  2260 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.141s (SENT)
I20260709 13:03:55.013466  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.015906  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.017607  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.018479  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.019170  1708 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.019315  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:55.024963  2293 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:55.028231  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.029117  1703 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:55.030329  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.031136  1703 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:55.018445  8777 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:55.032698  1560 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:03:55.033129  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.033883  1703 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:55.033983  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.034687  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.011219  8682 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.038389  8682 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.042161  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.042980  1703 raft_consensus.cc:2368] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:55.043848  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.044907  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.045256  8877 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.046504  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:55.026909  1524 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:55.051257  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.031123  9003 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.063519  1559 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:55.045939  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.053387  8877 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:55.070556  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.089336  1708 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
W20260709 13:03:55.089774  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 197492us
I20260709 13:03:55.101788  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.104362  1703 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:55.104874  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1245) took 1201 ms. Trace:
I20260709 13:03:55.105338  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.105955  1703 raft_consensus.cc:2376] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 20. Re-sending same reply.
I20260709 13:03:55.106791  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.107499  1703 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:55.108268  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.109031  9001 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.109378  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.109573  9001 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.110491  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.111135  1703 raft_consensus.cc:2368] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 18. Current term is 19.
I20260709 13:03:55.088796  9020 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.105315  1700 rpcz_store.cc:276] 0709 13:03:53.903313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.903485 (+   172us) service_pool.cc:224] Handling call
0709 13:03:53.903848 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.240525 (+336677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.104609 (+864084us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54003dca54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:55.104798 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640282368}
I20260709 13:03:55.112598  9020 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.112958  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.113201  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.113512  9001 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.113976  1700 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:55.114235  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.114912  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.088114  8709 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.115489  8709 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.116246  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.117552  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.117698  8709 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.118453  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.118774  8564 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.118708  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.120362  1700 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:55.121140  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.122303  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.122566  9020 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.122977  1700 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:55.123858  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.125329  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.119633  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.113920  1705 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 13:03:55.126955  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.127177  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.127951  1705 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:03:55.128481  1560 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:55.128978  8578 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:55.129354  8578 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:55.129459  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.129705  1705 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:03:55.130120  1560 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:55.130125  1705 pending_rounds.cc:85] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 10
I20260709 13:03:55.130208  8578 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.130775  1705 pending_rounds.cc:107] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:03:55.131147  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:55.131289  1705 raft_consensus.cc:2889] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.132227  1555 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
W20260709 13:03:55.087932  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.130330  1703 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:55.132902  1601 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.119544  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.137547  1700 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:55.137991  1708 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:55.071888  8877 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:55.138185  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1527) took 1405 ms. Trace:
I20260709 13:03:55.067751  8710 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.141218  8992 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:55.100080  8956 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.141636  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.142477  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.143283  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.143657  1560 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:55.144716  1551 raft_consensus.cc:1275] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:55.145903  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:55.148463  1559 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:55.146898  2297 rpcz_store.cc:276] 0709 13:03:53.726717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.726864 (+   147us) service_pool.cc:224] Handling call
0709 13:03:55.132063 (+1405199us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:55.132172 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88111360}
I20260709 13:03:55.152292  1630 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:55.157163  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.153136  1555 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:55.156301  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:55.160315  1627 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.158057  1629 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:55.162418  9003 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:55.147435  8777 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:55.165154  9003 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.161005  8578 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:55.165925  8777 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.167284  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.149039  8949 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:03:55.168479  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.169914  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.170874  1704 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:55.171183  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.155664  8837 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:03:55.173341  8682 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:55.162503  9020 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.174161  9020 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.151433  8992 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.160540  8956 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.167945  1703 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:55.177269  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.188525  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.164374  8709 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:55.199231  8709 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.178267  8992 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:55.180384  8956 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.208853  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.210685  8872 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:55.211292  8872 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.197477  9005 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:55.215301  1627 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:55.218613  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:55.221203  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.199713  8709 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:55.231340  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.231431  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.169929  8949 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:03:55.235811  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.219142  8843 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:55.238583  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.238426  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.248150  1626 raft_consensus.cc:2368] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:03:55.213477  8872 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.212579  8777 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:55.228066  9005 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.250372  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.251381  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:55.231463  9020 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.252393  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.204278  9009 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.090s
I20260709 13:03:55.260077  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.254845  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:55.262945  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.263573  1627 raft_consensus.cc:2393] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:55.250975  9005 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:55.264411  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.266721  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.269773  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.270927  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.228824  8877 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.271342  9000 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.276885  8710 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.278059  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.278295  8951 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.279332  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:55.280206  9015 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:55.280746  9015 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:55.280875  9020 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:55.281330  9020 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:55.281715  9020 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:55.281616  9015 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.282554  8709 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.284212  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:55.285183  8863 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:55.285566  8863 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:55.285946  8863 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.290143  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.291102  1700 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:55.291990  8837 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:55.292294  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.294263  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.294600  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.294874  2293 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 12.
I20260709 13:03:55.295931  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:55.288370  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.300120  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:55.309391  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.316129  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.318589  9020 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.332862  8811 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:55.335078  9020 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.259620  8872 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 13:03:55.344159  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44158.7us
I20260709 13:03:55.340234  9000 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.346745  8710 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:55.347339  1701 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:03:55.348387  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.350716  8954 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.351239  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1208) took 1034 ms. Trace:
I20260709 13:03:55.351373  8954 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.354169  8954 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:55.347265  8710 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.356137  8710 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:55.357213  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68916us
I20260709 13:03:55.358013  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:55.360237  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.359292  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:55.417490  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52101us
I20260709 13:03:55.378369  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.386129  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.424754  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.427574  1629 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:03:55.429170  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.335136  8811 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:03:55.380724  9018 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.379269  8612 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.308504  8984 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 13:03:55.351645  1634 rpcz_store.cc:276] 0709 13:03:54.316778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.316959 (+   181us) service_pool.cc:224] Handling call
0709 13:03:54.329829 (+ 12870us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:54.699140 (+369311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.699173 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:54.699872 (+   699us) log.cc:844] Serialized 78 byte log entry
0709 13:03:55.009625 (+309753us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:55.148441 (+138816us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.349151 (+200710us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b18002bafe0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70cb8f9a40 00007f70cb8f44e4 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf 00007f70ce9fe038
0709 13:03:55.349418 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.349762 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.349918 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.350921 (+  1003us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54000cd0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:55.350966 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.351013 (+    47us) raft_consensus.cc:1685] finished
0709 13:03:55.351024 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.351172 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191155,"prepare.queue_time_us":357683,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":56983,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":41875,"spinlock_wait_cycles":153163392,"thread_start_us":581137,"threads_started":3,"wal-append.queue_time_us":370848}
I20260709 13:03:55.424755  9020 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:55.438764  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 79525.9us
I20260709 13:03:55.340770  8648 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:03:55.441533  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1213) took 1077 ms. Trace:
I20260709 13:03:55.424865  9007 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:55.444968  1631 rpcz_store.cc:276] 0709 13:03:54.364658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.374849 (+ 10191us) service_pool.cc:224] Handling call
0709 13:03:54.375190 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.699450 (+324260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.699480 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.699515 (+    35us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:54.699551 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.441272 (+741721us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b540014fd54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:55.441328 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.441467 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136685440}
I20260709 13:03:55.444069  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:55.449990  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 124695us
I20260709 13:03:55.534260  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.590413  1701 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:55.482520  1704 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:55.569836  9041 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:55.569458  9020 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.536253  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:55.550566  9002 log.cc:927] Time spent T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.830s	user 0.003s	sys 0.000s
I20260709 13:03:55.546876  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.547901  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3110) took 1363 ms. Trace:
I20260709 13:03:55.605363  1633 rpcz_store.cc:276] 0709 13:03:54.184619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.184768 (+   149us) service_pool.cc:224] Handling call
0709 13:03:54.185073 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.724824 (+539751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.547574 (+822750us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54001c34d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:55.547820 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542944000}
I20260709 13:03:55.606366  9020 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.607031  8843 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.607560  8843 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.608747  9020 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.609541  8843 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.606439  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.610494  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:03:55.611320  1706 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:55.613216  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:55.614241  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.615078  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.615931  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.488893  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.458113  1634 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:55.383399  8769 consensus_queue.cc:1059] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:03:55.538208  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.616416  1633 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:55.431070  9019 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:03:55.462070  9005 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.453138  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.463310  8564 consensus_queue.cc:1048] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:03:55.503139  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.619313  1700 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:03:55.519730  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.620537  1708 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:55.454756  1631 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:03:55.365748  8863 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.621071  8889 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:55.621528  8889 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.621853  8889 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:55.622959  8863 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.399547  8992 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.623858  9048 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.624374  9048 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.625288  8992 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.625636  1627 raft_consensus.cc:2368] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26.
I20260709 13:03:55.446710  9007 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.626581  9007 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:55.457136  8978 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:03:55.539366  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.636811  1632 raft_consensus.cc:2368] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22.
I20260709 13:03:55.637565  9005 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:55.382791  9023 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 13:03:55.639652  8843 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:55.639946  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 186926us
I20260709 13:03:55.640892  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.588660  1629 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:03:55.642040  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.642408  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:55.642856  8843 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:55.643664  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.644145  1631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:55.644750  1631 raft_consensus.cc:606] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:55.645112  1631 raft_consensus.cc:614] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:55.499064  8913 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:55.572048  9039 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:03:55.646469  8913 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.598201  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:55.647874  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 49750.2us
I20260709 13:03:55.648653  8913 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:55.649181  8956 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:55.532151  1557 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:55.588088  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 144948us
I20260709 13:03:55.631866  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:55.618695  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 129940us
I20260709 13:03:55.631215  1628 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:03:55.653645  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.642055  9023 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:55.648597  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:55.590842  8933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.006s	sys 0.018s
I20260709 13:03:55.650417  8710 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.645110  1629 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:03:55.655896  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:55.602567  1701 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:55.631866  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.666350  1633 raft_consensus.cc:2410] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:55.656258  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.657644  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:55.658478  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.662456  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.667572  1704 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:55.664002  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.667994  1628 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:55.668121  9044 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:55.668574  9044 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.668929  9044 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:55.669234  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:55.670476  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.672008  9043 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:55.675981  8726 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.681205  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.681856  1701 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:55.685534  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:55.686182  2292 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:55.693753  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.695564  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.695410  8892 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:55.696579  8892 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.696935  1674 leader_election.cc:400] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:03:55.697278  1674 leader_election.cc:403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:55.698206  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.698375  8892 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:03:55.699187  8892 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.699241  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.701296  1674 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.701522  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.700058  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:55.659428  8933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.006s	sys 0.019s
I20260709 13:03:55.707679  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:55.709344  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.628494  8988 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.713410  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.708807  1704 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:55.702342  1634 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:03:55.659619  8956 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.737143  2292 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:55.714074  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:55.728441  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:55.738545  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:55.738752  2297 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 12.
I20260709 13:03:55.739692  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:55.732002  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.728317  1701 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
W20260709 13:03:55.735473  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52364us
W20260709 13:03:55.744495  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 63600.1us
I20260709 13:03:55.748096  1704 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:55.754520  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.756254  1554 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:55.758334  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:55.759070  1555 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:55.707993  9023 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:55.712546  8892 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:55.713680  8933 consensus_queue.cc:1243] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:55.729436  8901 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:55.765852  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:55.739452  9030 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.765496  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.772517  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.773546  1702 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:55.758194  9001 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.773931  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.774556  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:55.763319  8612 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:55.777554  1704 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:55.769383  2293 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:55.708473  8877 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
W20260709 13:03:55.781293  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46997.2us
W20260709 13:03:55.783581  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127790us
I20260709 13:03:55.770301  9038 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.773156  9000 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:55.791633  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 82358.7us
I20260709 13:03:55.737349  8988 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:55.823153  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.819666  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:55.820212  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 75021.3us
I20260709 13:03:55.825312  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.827184  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.829795  8892 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:55.814971  8843 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:55.856923  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.825628  8769 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:55.783066  9001 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:55.776449  9005 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:03:55.776391  8777 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:03:55.860023  1701 raft_consensus.cc:1275] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
W20260709 13:03:55.860553  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101973us
I20260709 13:03:55.876109  9058 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:55.881130  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61534.6us
I20260709 13:03:55.866185  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:55.884203  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:55.882550  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57980.9us
I20260709 13:03:55.887707  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:55.852105  8951 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:55.900652  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.903292  1701 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:55.860034  8769 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:55.923174  1701 raft_consensus.cc:2468] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:03:55.882848  9054 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.934363  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:55.935335  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:55.901836  8949 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:55.935949  1635 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:03:55.894029  9057 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:55.942478  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:03:55.946723  1635 raft_consensus.cc:606] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
W20260709 13:03:55.947535  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173661us
I20260709 13:03:55.948397  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:55.937146  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 170726us
I20260709 13:03:55.932235  8769 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.929517  8933 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:55.953303  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:55.854116  9023 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:55.945634  9040 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:03:55.937588  8710 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:55.929589  8951 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:56.229871  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47153.8us
W20260709 13:03:56.215382  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 350562us
I20260709 13:03:56.232729  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:56.234994  8682 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:56.237989  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3195) took 1003 ms. Trace:
I20260709 13:03:56.238565  1632 rpcz_store.cc:276] 0709 13:03:55.234377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.235799 (+  1422us) service_pool.cc:224] Handling call
0709 13:03:56.237679 (+1001880us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b5400296dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:56.237906 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884606720}
I20260709 13:03:56.248131  1633 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:56.252157  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 368015us
I20260709 13:03:56.253237  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.108831  8949 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.190614  8933 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:56.069871  9057 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:56.285604  1632 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:03:56.265043  8982 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 2.219s	user 0.000s	sys 0.005s
I20260709 13:03:56.278648  8933 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:56.266412  9066 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:56.285246  9015 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:56.326591  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 94058.5us
I20260709 13:03:56.330090  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:56.335063  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 386694us
W20260709 13:03:56.325091  9007 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 13:03:56.343003  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:03:56.349776  9060 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:03:56.351877  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:03:56.353392  9044 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.001s
W20260709 13:03:56.354175  9065 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:03:56.354008  9044 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.355690  2296 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21.
W20260709 13:03:56.359194  8889 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.000s
I20260709 13:03:56.359984  9001 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:56.361419  9050 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.470s	user 0.001s	sys 0.000s
W20260709 13:03:56.372720  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 140122us
I20260709 13:03:56.333030  8988 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.330886  9062 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:56.400827  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 147694us
I20260709 13:03:56.401944  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.396723  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.414130  1599 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.359654  8889 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.336138  9015 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:56.430616  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 101894us
I20260709 13:03:56.347429  9007 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.449829  9086 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:56.450536  9086 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.451016  9086 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:56.452281  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 109390us
I20260709 13:03:56.453423  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:56.465118  8889 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.407097  9057 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.467386  9071 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:03:56.433625  9062 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:56.469596  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.432241  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:03:56.492132  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 78083.6us
I20260709 13:03:56.492985  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:56.493076  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60925.7us
I20260709 13:03:56.494055  1598 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:56.504601  1560 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 22: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:03:56.507654  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54353.1us
I20260709 13:03:56.508723  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:56.507545  8564 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.003s	sys 0.006s
W20260709 13:03:56.511063  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.003s	sys 0.006s
I20260709 13:03:56.494010  8889 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:56.514339  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.005s	sys 0.014s
W20260709 13:03:56.521763  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.005s	sys 0.014s
W20260709 13:03:56.516251  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.003s	sys 0.016s
I20260709 13:03:56.526479  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:56.526960  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.004s	sys 0.016s
W20260709 13:03:56.518395  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.004s	sys 0.007s
I20260709 13:03:56.530290  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:56.534991  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:56.504662  8951 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.538548  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:56.518807  8564 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.003s	sys 0.007s
I20260709 13:03:56.544836  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:56.545554  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:56.548702  2293 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:03:56.560171  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66191.7us
I20260709 13:03:56.502573  8811 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:03:56.567045  2295 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:56.561203  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:56.562222  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 69321.1us
I20260709 13:03:56.568734  2292 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:56.563879  9079 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:56.512588  9044 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:56.564571  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 150924us
I20260709 13:03:56.568925  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:56.569042  8564 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:56.622960  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:56.536926  9066 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:03:56.629911  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68820.7us
I20260709 13:03:56.605329  8987 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:56.613444  9079 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.642109  9066 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:56.648546  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 79683.7us
I20260709 13:03:56.651297  1598 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:56.607016  9074 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:56.651580  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:56.637545  8682 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:56.653291  9063 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:03:56.670845  2293 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:56.677466  9082 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:56.678282  9095 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:56.679028  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:56.678794  9095 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.680249  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:56.681277  9079 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.681797  9079 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.682070  9091 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.683097  9078 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:56.684171  9055 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.685187  9076 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:56.659544  9067 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.685710  9076 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:56.686024  9067 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.685657  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1349) took 1393 ms. Trace:
I20260709 13:03:56.687837  9084 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.689040  9081 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:56.689666  9081 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.690130  9081 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:56.692086  9094 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:56.692727  9094 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.693272  9094 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:56.688153  2294 rpcz_store.cc:276] 0709 13:03:55.292232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.303898 (+ 11666us) service_pool.cc:224] Handling call
0709 13:03:55.304306 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.306454 (+  2148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.306490 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.332127 (+ 25637us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.499395 (+1167268us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b18003f46c0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:56.538889 (+ 39494us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.538923 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.538991 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.685379 (+146388us) raft_consensus.cc:1685] finished
0709 13:03:56.685401 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.685597 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3535,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":14552832,"thread_start_us":39307,"threads_started":1,"wal-append.queue_time_us":1337196}
I20260709 13:03:56.694955  9067 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.696019  9064 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:56.696411  9079 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.696923  9080 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:56.695938  8612 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.916s	user 0.010s	sys 0.002s
I20260709 13:03:56.697815  8612 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:56.698865  9083 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:03:56.704659  9085 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:56.706076  9099 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:03:56.732031  9090 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.732933  9090 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.734262  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
W20260709 13:03:56.741276  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 142309us
I20260709 13:03:56.659276  9073 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.686918  9076 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.667930  9059 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.741415  9007 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.738300  9015 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.770637  9092 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.759811  9073 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.775931  9092 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.778072  9073 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.778209  9092 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.763118  9059 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.781008  9059 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:56.792243  8769 consensus_meta.cc:287] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
I20260709 13:03:56.792863  8769 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.768620  9076 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:56.804900  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:56.806629  1555 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:56.799913  9059 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:56.802292  9092 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:56.811621  9092 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.814013  9092 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.817343  9018 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:56.817741  9018 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:56.818552  9018 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.828540  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3202) took 1487 ms. Trace:
I20260709 13:03:56.829020  1627 rpcz_store.cc:276] 0709 13:03:55.341454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.344011 (+  2557us) service_pool.cc:224] Handling call
0709 13:03:56.828321 (+1484310us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002756d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:56.828476 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635737472}
I20260709 13:03:56.803131  9015 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:56.831167  1553 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:03:56.836470  9015 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.838861  9015 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.843006  9081 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.830922  9059 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.848302  9090 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:56.868232  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 949974us
I20260709 13:03:56.871376  9059 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:56.872340  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.797866  8933 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.850425  8769 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.851468  9048 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.851310  9086 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.819572  9018 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:56.861330  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 995130us
I20260709 13:03:56.869773  8892 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:56.868800  1626 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
W20260709 13:03:56.877329  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.881634  2290 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:56.885526  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:56.890719  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2720) took 1107 ms. Trace:
I20260709 13:03:56.891532  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:56.888285  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 236797us
W20260709 13:03:56.892911  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.891103  1704 rpcz_store.cc:276] 0709 13:03:55.783476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.783607 (+   131us) service_pool.cc:224] Handling call
0709 13:03:55.817224 (+ 33617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.817701 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.817746 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.824613 (+  6867us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.528539 (+703926us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:56.808920 (+280381us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.890439 (+ 81519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.890511 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.890547 (+    36us) raft_consensus.cc:1685] finished
0709 13:03:56.890559 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.890668 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81208,"prepare.queue_time_us":341196,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":3492,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":27636,"thread_start_us":923116,"threads_started":2,"wal-append.queue_time_us":744339}
I20260709 13:03:56.893811  1599 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:56.887948  9054 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.889464  9003 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.892529  8933 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.893025  8958 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.898936  8901 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:56.899888  8892 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:56.930058  2297 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:56.930514  2297 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:56.931948  8892 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:56.932746  9081 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.904492  8710 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:56.931360  2297 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.925971  9054 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.952363  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3127) took 1285 ms. Trace:
I20260709 13:03:56.933512  9003 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.956406  2297 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.955559  9094 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:56.982103  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1643) took 1275 ms (client timeout 1551 ms). Trace:
W20260709 13:03:56.982518  2296 rpcz_store.cc:269] 0709 13:03:55.707011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.707354 (+   343us) service_pool.cc:224] Handling call
0709 13:03:56.981903 (+1274549us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54005e06d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:56.982040 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484847488}
I20260709 13:03:56.983050  9094 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:56.952558  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:39515 timed out after 1.538s (SENT) [suppressed 32 similar messages]
I20260709 13:03:56.967957  8901 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:03:56.986352  8901 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:56.987183  2297 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:03:56.987632  2297 pending_rounds.cc:85] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 14
I20260709 13:03:56.987929  2297 pending_rounds.cc:107] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:03:56.988224  2297 raft_consensus.cc:2889] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:56.988296  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2704) took 1035 ms. Trace:
I20260709 13:03:56.988837  1553 rpcz_store.cc:276] 0709 13:03:55.953021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.987474 (+1034453us) service_pool.cc:224] Handling call
0709 13:03:56.987814 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.987986 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.988011 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.988052 (+    41us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:56.988084 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.988146 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.988237 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:56.937783  8710 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.989897  8710 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:56.952833  1635 rpcz_store.cc:276] 0709 13:03:55.666514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.264753 (+598239us) service_pool.cc:224] Handling call
0709 13:03:56.294910 (+ 30157us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:56.524766 (+229856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.524822 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:56.538362 (+ 13540us) log.cc:844] Serialized 52 byte log entry
0709 13:03:56.800503 (+262141us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:56.800556 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.800649 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.952120 (+151471us) raft_consensus.cc:1685] finished
0709 13:03:56.952142 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.952302 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47259,"raft.queue_time_us":125648,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":7317,"thread_start_us":393698,"threads_started":2,"wal-append.queue_time_us":307702}
I20260709 13:03:56.928192  8958 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.995364  8958 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.960577  8892 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:56.973206  9086 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.976835  9095 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:56.996465  9071 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:56.997134  9071 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.998214  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2721) took 1173 ms. Trace:
I20260709 13:03:57.000366  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2670) took 2062 ms. Trace:
I20260709 13:03:57.000743  9071 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:56.980139  9003 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:56.989310  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1646) took 1244 ms. Trace:
I20260709 13:03:56.959846  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1217) took 1429 ms. Trace:
I20260709 13:03:57.006083  2297 rpcz_store.cc:276] 0709 13:03:55.745202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.749622 (+  4420us) service_pool.cc:224] Handling call
0709 13:03:55.749962 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.929950 (+1179988us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b54005de420. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce95bdc0 00007f70ce9e61c0 00007f70ce9e7231 00007f70ce9e3d8f 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c
0709 13:03:56.989047 (+ 59097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.989258 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49190144}
I20260709 13:03:57.005010  1629 rpcz_store.cc:276] 0709 13:03:55.529959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.764416 (+234457us) service_pool.cc:224] Handling call
0709 13:03:55.774100 (+  9684us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:55.778887 (+  4787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.778918 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.790904 (+ 11986us) log.cc:844] Serialized 52 byte log entry
0709 13:03:56.872435 (+1081531us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b18000ddce0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8d9e07 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1 00007f70ce99fa0f
0709 13:03:56.900349 (+ 27914us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.900397 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.900480 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.959506 (+ 59026us) raft_consensus.cc:1685] finished
0709 13:03:56.959527 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.959782 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26081,"raft.queue_time_us":17760,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4253,"spinlock_wait_cycles":4560640,"thread_start_us":27240,"threads_started":1,"wal-append.queue_time_us":1137113}
I20260709 13:03:57.000762  1556 rpcz_store.cc:276] 0709 13:03:54.937844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.937957 (+   113us) service_pool.cc:224] Handling call
0709 13:03:54.938218 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.946426 (+  8208us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:03:54.946649 (+   223us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.164424 (+217775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:55.182824 (+ 18400us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:55.182873 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.000143 (+1817270us) spinlock_profiling.cc:243] Waited 7.99 ms on lock 0x7b54002efed4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:57.000187 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.000313 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24999,"prepare.queue_time_us":25791,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":26548,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":18387200,"thread_start_us":192327,"threads_started":1,"wal-append.queue_time_us":196121,"wal-append.run_cpu_time_us":1132,"wal-append.run_wall_time_us":1067440}
I20260709 13:03:56.998580  1702 rpcz_store.cc:276] 0709 13:03:55.825240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.825356 (+   116us) service_pool.cc:224] Handling call
0709 13:03:55.825671 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.847022 (+ 21351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.847067 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.851833 (+  4766us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.855038 (+1003205us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b180023aa00. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb871c0a 00007f70d4fb1219 00007f70d28b858f 00007f70cb8f7571 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd
0709 13:03:56.997310 (+142272us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b180023aa00. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f6c02 00007f70ce57e20b 00007f70ce583d23 00007f70ce5840d1
0709 13:03:56.997455 (+   145us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.997747 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.997936 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.998002 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.998039 (+    37us) raft_consensus.cc:1685] finished
0709 13:03:56.998049 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.998161 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25895,"raft.queue_time_us":127,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":20954,"spinlock_wait_cycles":30704640,"thread_start_us":30164,"threads_started":1,"wal-append.queue_time_us":24454}
I20260709 13:03:57.014287  9003 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:57.018143  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 80134.9us
W20260709 13:03:57.019549  8937 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:03:57.027139  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.008566  9095 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.034075  9095 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.036461  9095 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.037853  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 3090) took 2111 ms. Trace:
I20260709 13:03:57.038288  1707 rpcz_store.cc:276] 0709 13:03:54.926795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.932902 (+  6107us) service_pool.cc:224] Handling call
0709 13:03:57.037625 (+2104723us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54001333d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:57.037769 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695899008}
W20260709 13:03:57.027889  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 75382.6us
W20260709 13:03:57.040032  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 148558us
I20260709 13:03:57.041472  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:57.041625  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.042342  8612 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:57.042768  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 3105) took 1313 ms. Trace:
I20260709 13:03:57.043031  8710 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.043617  8872 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:57.045121  8710 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.045728  8710 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:57.047057  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2699) took 1095 ms. Trace:
I20260709 13:03:57.047531  1560 rpcz_store.cc:276] 0709 13:03:55.951748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.953094 (+  1346us) service_pool.cc:224] Handling call
0709 13:03:57.046779 (+1093685us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b540036c9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:57.046956 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846619904}
I20260709 13:03:57.050392  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3125) took 1388 ms. Trace:
I20260709 13:03:57.053377  1556 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:57.041563  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.050884  1633 rpcz_store.cc:276] 0709 13:03:55.661602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.251093 (+589491us) service_pool.cc:224] Handling call
0709 13:03:56.251458 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.276645 (+ 25187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.276674 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.276736 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:56.276766 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.050068 (+773302us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b54001e9454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:57.050155 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.050323 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57354112}
I20260709 13:03:57.057827  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:57.043295  1557 rpcz_store.cc:276] 0709 13:03:55.728998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.041354 (+1312356us) service_pool.cc:224] Handling call
0709 13:03:57.042689 (+  1335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.059193  8958 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:57.057322  8892 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:57.042801  8612 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.062963  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2697) took 1111 ms. Trace:
I20260709 13:03:57.063258  8612 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:57.070452  9093 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.611s	user 0.002s	sys 0.000s
I20260709 13:03:57.072489  1557 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.074759  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.075234  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.063403  1555 rpcz_store.cc:276] 0709 13:03:55.951258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.352979 (+401721us) service_pool.cc:224] Handling call
0709 13:03:56.353260 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.807991 (+454731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.062730 (+254739us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54003faa54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:57.062902 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042237184}
I20260709 13:03:57.059367  8710 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:03:57.082257  8863 raft_consensus.cc:3055] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:57.082683  8863 raft_consensus.cc:740] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:57.083956  8709 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.097407  8949 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:57.099093  8949 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.078999  9055 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.100126  9055 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.101321  8949 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.102156  9055 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.103330  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1219) took 1185 ms. Trace:
W20260709 13:03:57.105635  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46067.2us
I20260709 13:03:57.103807  1632 rpcz_store.cc:276] 0709 13:03:55.917745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.300801 (+383056us) service_pool.cc:224] Handling call
0709 13:03:56.318363 (+ 17562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.447387 (+129024us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:56.447874 (+   487us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.510025 (+ 62151us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.611715 (+101690us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.649607 (+ 37892us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.650419 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.650552 (+   133us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:56.650801 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.650933 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.651006 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.823204 (+172198us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b18004e44e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:57.033300 (+210096us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b18004e44e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:57.069867 (+ 36567us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:03:57.103089 (+ 33222us) raft_consensus.cc:1685] finished
0709 13:03:57.103113 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.103264 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173709,"prepare.queue_time_us":177284,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":132,"raft.queue_time_us":418211,"raft.run_cpu_time_us":650,"raft.run_wall_time_us":63335,"spinlock_wait_cycles":84515456,"thread_start_us":240790,"threads_started":3,"wal-append.queue_time_us":10565}
I20260709 13:03:57.087085  8937 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.108515  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.113314  9092 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.087932  9095 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.118364  9119 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:57.119892  9092 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.113330  8913 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 13:03:57.096596  9030 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.098446  9120 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:57.130465  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.130605  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.130961  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2705) took 1177 ms. Trace:
I20260709 13:03:57.131318  1559 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:03:57.129117  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.116359  9055 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:03:57.131361  1553 rpcz_store.cc:276] 0709 13:03:55.953244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.002129 (+1048885us) service_pool.cc:224] Handling call
0709 13:03:57.002455 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.002780 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:57.009308 (+  6528us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.130646 (+121338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.130694 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:57.130739 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.130801 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.130910 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":852,"prepare.queue_time_us":8699,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":18634,"thread_start_us":103441,"threads_started":1}
I20260709 13:03:57.134320  8958 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.135414  1674 leader_election.cc:400] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22.
I20260709 13:03:57.137614  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.139053  1674 leader_election.cc:400] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26.
I20260709 13:03:57.139550  1674 leader_election.cc:403] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.144342  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.127326  9095 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.154692  9095 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.125792  9119 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.155375  9119 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:57.156002  8892 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26.
I20260709 13:03:57.121910  8937 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.159706  8937 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.136693  9030 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.129379  8710 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:03:57.168668  9030 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.170249  9074 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:57.170517  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.170840  9074 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.172560  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.173120  9074 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.174940  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.175563  2289 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.177969  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.161172  9092 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.180240  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:57.181507  9003 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:57.174955  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:57.159861  8958 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:03:57.183163  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.184403  1552 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.184489  8958 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:57.184881  8958 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:57.185690  8958 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.186935  9064 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.187186  8958 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.187422  9064 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:57.227106  8933 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:57.227521  8933 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:03:57.188454  9069 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.826s	user 0.001s	sys 0.001s
I20260709 13:03:57.224000  9113 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:03:57.191241  1626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3204) took 1805 ms (client timeout 1587 ms). Trace:
I20260709 13:03:57.229173  8998 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:57.229561  8998 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:57.197621  9054 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.230321  8998 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.232918  9003 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:57.233034  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.233459  9003 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.200296  8863 consensus_queue.cc:260] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.234423  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.234767  8863 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:57.235138  1634 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:57.236281  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:57.229010  1626 rpcz_store.cc:269] 0709 13:03:55.385549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.385739 (+   190us) service_pool.cc:224] Handling call
0709 13:03:57.191058 (+1805319us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001414d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:57.191161 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3329849728}
I20260709 13:03:57.239945  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1215) took 1712 ms. Trace:
I20260709 13:03:57.240996  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.241739  1630 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:57.243327  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.243986  2293 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:57.244139  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.244784  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.245353  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.246447  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.247408  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.205755  9089 consensus_queue.cc:1048] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:57.250243  1635 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:57.251255  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.207304  9043 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:03:57.207978  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 542404us
I20260709 13:03:57.253640  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.254899  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.254830  8933 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.208889  9096 raft_consensus.cc:2804] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:57.256117  9096 raft_consensus.cc:697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:57.256219  8933 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.209653  8709 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.209506  8682 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:57.256902  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.256873  8709 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.258051   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:57.258484  1629 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:57.259230  8709 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.261005  9043 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.261492  9043 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.262387  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.262976  1704 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:03:57.264180  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.264282  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.264828  1556 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:57.264946  1707 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:03:57.231572  8998 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.265892  9043 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.269665  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.270486  1708 raft_consensus.cc:2393] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:57.271311  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.272202  1709 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 14: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:57.258087  1632 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:57.181315  9120 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:57.213826  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3209) took 1700 ms. Trace:
I20260709 13:03:57.214383  9082 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:57.217525  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.278987  2294 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:03:57.218142  2297 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:57.218400  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.218622  9078 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:57.219301  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:57.220685  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 477139us
I20260709 13:03:57.223042  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.277089  1631 rpcz_store.cc:276] 0709 13:03:55.513714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.645852 (+132138us) service_pool.cc:224] Handling call
0709 13:03:55.655100 (+  9248us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:55.662463 (+  7363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.662495 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:55.708998 (+ 46503us) log.cc:844] Serialized 78 byte log entry
0709 13:03:55.884561 (+175563us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:56.281520 (+396959us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.287676 (+  6156us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.398198 (+110522us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.398374 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.398446 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.853504 (+455058us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b180019f7e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:56.880052 (+ 26548us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b180019f7e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:57.083294 (+203242us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b180019f7e0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:57.115731 (+ 32437us) raft_consensus.cc:1685] finished
0709 13:03:57.115752 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.213605 (+ 97853us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b54000cc6d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:57.213752 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":802798,"prepare.queue_time_us":47604,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":158,"raft.queue_time_us":409933,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":10117,"spinlock_wait_cycles":76254080,"thread_start_us":586088,"threads_started":3,"wal-append.queue_time_us":200339}
I20260709 13:03:57.285938  2295 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:03:57.181993  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.223129  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:57.224359  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 330634us
I20260709 13:03:57.286798  1559 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:03:57.287055  2290 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:03:57.223954  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.288110  2288 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:57.225477  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:57.288718  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.288561  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.240340  1628 rpcz_store.cc:276] 0709 13:03:55.527654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.671236 (+143582us) service_pool.cc:224] Handling call
0709 13:03:55.682774 (+ 11538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.736075 (+ 53301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.736123 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.736748 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.527396 (+790648us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.529390 (+  1994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.529496 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.699946 (+170450us) raft_consensus.cc:1685] finished
0709 13:03:56.699970 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.239654 (+539684us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b54002972d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:57.239883 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38589,"raft.queue_time_us":48375,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8467,"spinlock_wait_cycles":25793792,"thread_start_us":790226,"threads_started":1,"wal-append.queue_time_us":772135}
I20260709 13:03:57.290308  9078 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.291733  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.292050  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.293486  9052 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:57.289023  1554 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:03:57.293956  9052 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.294441  9052 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:57.293517  1628 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:57.296511  9120 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:57.297061  9120 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:57.297662  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:57.298484  9059 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:57.297964  9120 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.298880  9059 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:57.299327  9059 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:57.285626  2291 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:03:57.300468  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:57.257010  9096 consensus_queue.cc:237] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.301234  9068 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:57.301651  9068 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.302000  9068 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.303939  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.304598  2292 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:03:57.308073  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.310108  1599 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21.
I20260709 13:03:57.310477  1599 leader_election.cc:403] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.296651  1632 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:57.320156  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.299453  9120 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.165566  8612 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.331279  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.332273  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.334293  9018 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:57.336213  1558 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:03:57.320451  8956 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.327348  9055 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:57.322077  9119 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.331094  9062 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 13:03:57.336776  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.339133  1703 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:57.342161  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.337373  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:57.340792  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:57.350092  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.359925  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:57.344583  2292 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:57.349416  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.349275  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.370664  1626 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 14: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:57.376163  2289 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 14.
I20260709 13:03:57.380566  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.379496  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.385587  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:57.384577  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47300.4us
I20260709 13:03:57.377955  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:57.389559  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.390873  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.392690  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.394760  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.400535  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.402647  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.406137  1630 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 25. Current term is 26. Ops: []
I20260709 13:03:57.409050  1524 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.337805  9055 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.332197  8769 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:57.337713  8612 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.415594  1524 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.368803  8726 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:57.424396  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.372411  9062 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.369125  8710 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:57.473423  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 75364.4us
I20260709 13:03:57.337374  8956 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.407971  9018 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:57.390317  9090 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:57.421277  8877 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:57.389283  8987 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.456851  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:57.478228  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.426332  8843 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:57.408102  9020 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:57.480537  1554 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.464478  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:57.421754  9055 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.473067  9092 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:57.468495  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.408188  9100 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.414690  9051 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:57.368839  9068 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:57.497331  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.408558  8901 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.427433  9089 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:57.415035  8951 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:03:57.507436  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50714.4us
I20260709 13:03:57.424700  9064 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.409955  8682 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 13:03:57.415904  8954 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 13:03:57.479358  9119 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.394090  9003 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:57.519155  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54831us
I20260709 13:03:57.525369  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:57.519639  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 101954us
I20260709 13:03:57.515868  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.552165  1630 raft_consensus.cc:2393] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:57.492846  9018 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:57.495266  9090 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:57.538252  1633 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:57.408198  8872 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:57.530957  9051 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.498739  8877 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.543710  1634 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:57.508944  8843 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.518044  8612 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:57.491473  9062 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.555923  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.495048  9049 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:03:57.523717  9092 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:57.558645  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.559397  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.562001  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.681044  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.563546  1706 raft_consensus.cc:1240] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 25. Current term is 26. Ops: []
I20260709 13:03:57.535737  9089 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.682394  1702 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:57.682785  9089 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.683032  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.683245  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.531423  9104 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.508152  8987 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.684196  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.684557  9092 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:57.685393  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.536360  9054 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:57.686409  9131 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:03:57.656153  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.560825  9018 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:57.535849  8709 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:03:57.684247  1707 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.745575  1706 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.664125  1554 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:57.664536  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.577556  8612 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:03:57.665895  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.536504  9100 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.565871  9090 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:57.668570  9136 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:03:57.669374  9130 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:03:57.672015  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.731027  9052 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.672015  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.673206  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.658643  8954 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:03:57.667388  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.683748  1702 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.491739  8710 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.686111  9054 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.687566  8987 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.656190  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.737246  1450 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:57.697052  9068 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.698273  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.701586  9018 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.706661  9059 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.711841   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:57.571839  9051 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.819399  9068 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:57.819948  9059 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:57.821327  9059 consensus_queue.cc:1059] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:57.821970  9059 consensus_queue.cc:1243] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:03:57.822885  8863 consensus_meta.cc:287] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.000s
I20260709 13:03:57.717401  9020 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:57.655675  8892 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:57.538355  9001 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:03:57.838729  1705 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:57.723951  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:57.725790  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.726157  9062 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.575353  8877 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.727567  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:57.860926  8710 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:57.862666  8726 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21.
I20260709 13:03:57.728617  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:57.730641  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.732787  9019 raft_consensus.cc:993] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:03:57.733287  9064 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.734623  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.566104  8872 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.752002  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.655146  1700 raft_consensus.cc:1240] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: [22.11-22.11]
W20260709 13:03:57.750571  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83080.5us
I20260709 13:03:57.538851  8901 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:57.577556  8843 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:57.758538  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.764674  1707 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:57.654932  9141 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.673182  9132 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:03:57.905357  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.906323  1707 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
W20260709 13:03:57.786651  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 57481us
I20260709 13:03:57.731027  9092 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.912232  9092 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:57.800765  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47583.9us
I20260709 13:03:57.750705  8612 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:57.696043  9089 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:57.793329  9103 log.cc:927] Time spent T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:03:57.815048  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.817051  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.902885  9117 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:57.916561  9117 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.917074  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.917845  1630 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:03:57.918712  9117 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:57.918833  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:57.829921  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:57.920073  1633 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.920773  1553 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:57.834610  2290 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:57.835103  1703 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:57.924975  9082 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.835672  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:57.843863  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 120005us
I20260709 13:03:57.845477  9055 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:57.850835  9120 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.002s
I20260709 13:03:57.852058  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.855592  9051 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:57.870045  1705 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:57.792639  8949 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:03:57.839515  8892 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:57.833354  1706 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:57.895102  1560 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:57.919545  8958 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.732s	user 0.001s	sys 0.006s
I20260709 13:03:57.897032  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.970971  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.971287  1559 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:57.907943  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.910059  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:57.915696  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.916034  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:57.896242  1554 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:57.823737  8863 raft_consensus.cc:3055] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:03:57.968950  1709 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 18: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:03:57.964000  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.973598  1704 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:57.975824  1701 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:03:57.886327  8682 raft_consensus.cc:1081] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:03:57.981398  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:03:57.976154  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68854.9us
I20260709 13:03:57.970062  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:57.863907  8777 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:57.889828  8872 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:58.008913  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 298253us
I20260709 13:03:58.008702  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:58.009341  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:58.014278  1703 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:03:57.932729  9130 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:03:58.024171  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:03:57.981438  9149 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:58.013351  1552 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:58.029830  1634 raft_consensus.cc:1217] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Deduplicated request from leader. Original: 14.13->[15.14-17.15]   Dedup: 15.14->[17.15-17.15]
I20260709 13:03:58.030417  1634 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: [15.14-17.15]
W20260709 13:03:58.009104  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 288235us
I20260709 13:03:57.902837  9062 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.019654  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.026702  1628 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:58.027428  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:58.008767  1706 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:58.029079  2291 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:58.037238  1706 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:03:58.013674  8998 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.003s
I20260709 13:03:58.035321  1700 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:03:57.974499  9057 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.009812  8956 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.049049  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.044785  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.057745  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.007932  9147 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:57.929112  8710 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:03:58.054418  1632 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:58.034121  8951 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:58.061307  1700 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:03:58.100137  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:57.930115  9141 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:58.089867  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 62514.4us
I20260709 13:03:58.106752  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.022639  9120 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:03:58.055871  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.100967  2292 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.018915  8958 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:03:58.007015  8863 raft_consensus.cc:740] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.091663  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.048578  9020 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.109872  1634 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:58.117549  1708 raft_consensus.cc:2410] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
W20260709 13:03:58.119444  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.017s	sys 0.032s
I20260709 13:03:58.058491  8956 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:58.126745  1633 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:58.142392  1629 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:58.096377  9119 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:58.099771  8951 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:58.149353  8951 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:58.149013  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.032s
W20260709 13:03:58.151008  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 597100us
I20260709 13:03:58.151916  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.044759  9144 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.101s
I20260709 13:03:58.134523  1702 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:58.155375  1707 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.104066  9052 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:58.156400  1709 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:03:58.104236  9003 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.049233  9062 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:58.060079  9057 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.168912  1707 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:03:58.169001  9057 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:58.158735  8913 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:58.170682  8913 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:58.173018  9143 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:03:58.158572  8710 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:58.174571  8710 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:58.094310  8892 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.175608  8710 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.176666  8892 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:58.177330  8710 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.160363  8956 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:58.184275  8901 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:58.188393  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39480 (request call id 1244) took 1060 ms. Trace:
I20260709 13:03:58.166220  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:03:58.190582  1627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.019s	sys 0.046s
W20260709 13:03:58.192029  1627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.020s	sys 0.046s
I20260709 13:03:58.195225  1558 raft_consensus.cc:2393] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 19: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:58.195933  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:58.190243  1555 rpcz_store.cc:276] 0709 13:03:57.128290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.128485 (+   195us) service_pool.cc:224] Handling call
0709 13:03:57.128786 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.129042 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:57.504226 (+375184us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.185119 (+680893us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.188037 (+  2918us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.188093 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:58.188144 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.188216 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.188339 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8450,"prepare.queue_time_us":18341,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":55741,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"thread_start_us":1041368,"threads_started":3,"wal-append.queue_time_us":343591}
I20260709 13:03:58.197378  2292 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:58.105419  9090 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.100939  9147 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.198557  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:58.190497  8648 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:58.191789  9144 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:58.192123  9003 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:58.224933  1551 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:58.154143  9020 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.227602  9141 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.226235  9144 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:58.228511  9148 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:58.196553  8913 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.222723  8648 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:58.234076  8877 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.243944  9100 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.244760  9055 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:58.248572  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50097.3us
W20260709 13:03:58.247712  8933 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.988s	user 0.011s	sys 0.000s
I20260709 13:03:58.214701  9018 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.277158  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.249697  9154 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.258175  9054 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:58.263592  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.020s	sys 0.024s
W20260709 13:03:58.266049  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.185539  9142 consensus_queue.cc:1175] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b2c3934407d422495728f84374c6b53" responder_term: 22 status { last_received { term: 22 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 22 index: 15 } }
I20260709 13:03:58.248095  9003 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:58.203011  8863 consensus_queue.cc:260] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.271253  8769 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:58.289525  9090 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.290232  8877 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.284277  9003 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:58.290967  8648 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.291549  8612 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.294343  9062 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:58.296350  9144 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.277127  9051 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:58.298441  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:58.299894  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:58.325702  1558 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:58.301981  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:58.304955  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.306174  9095 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:58.327364  2290 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 25: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:58.305750  8843 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.307490  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:58.309468  2293 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:58.310130  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.329924  8951 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.332670  1633 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:58.332726  9020 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:58.312987  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.021s	sys 0.024s
I20260709 13:03:58.333356  8951 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:58.333923  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 582016us
I20260709 13:03:58.334450  9147 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.334666  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:58.327596  9095 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:58.335695  9147 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:58.336297  9147 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.336030  9095 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.319708  8710 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:03:58.337600  8710 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:58.337227  9147 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.328881  8648 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.338063  8710 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:58.340395  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:03:58.340691  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.343269  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.348251  2296 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:03:58.313516  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 221931us
I20260709 13:03:58.351835  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:58.353031  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:58.354107  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:58.326632  8769 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.382704  1554 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:58.341784  9144 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.337427  9095 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.392225  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.365069  9141 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.396130  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.389662  9051 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:58.397863  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.391826  9055 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:58.391564  9074 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:58.402858  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.404668  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3223) took 1046 ms. Trace:
I20260709 13:03:58.384380  9001 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 13:03:58.382167  9100 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
W20260709 13:03:58.407389  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.405081  1635 rpcz_store.cc:276] 0709 13:03:57.358523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.388985 (+ 30462us) service_pool.cc:224] Handling call
0709 13:03:57.463047 (+ 74062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.474585 (+ 11538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.474615 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.490647 (+ 16032us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.975183 (+484536us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:58.402608 (+427425us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.403651 (+  1043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.403735 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.404468 (+   733us) raft_consensus.cc:1685] finished
0709 13:03:58.404483 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.404608 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11045,"raft.queue_time_us":17542,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":14694,"thread_start_us":795027,"threads_started":2,"wal-append.queue_time_us":738851}
W20260709 13:03:58.401885  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 377777us
I20260709 13:03:58.390165  8992 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:03:58.428614  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:58.402716  9055 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.404021  9074 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.398881  9141 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.419122  9054 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.394129  8710 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.402351  8872 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.446620  1598 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:58.451102  9141 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.451969  1706 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:58.447917  9054 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.455251  8612 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.456950  9054 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.459893  9078 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.460311  8843 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.461726  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.461942  8872 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.462564  2296 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25.
I20260709 13:03:58.439283  9055 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.464156  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.439574  8769 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.441100  9075 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:58.468137  8951 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:58.440639  9074 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.472033  9064 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:58.475288  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.459887  9003 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.487941  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3151) took 1095 ms. Trace:
I20260709 13:03:58.488590  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.489446  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.493197  1524 leader_election.cc:400] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25.
I20260709 13:03:58.493830  1524 leader_election.cc:403] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:03:58.488492  1630 rpcz_store.cc:276] 0709 13:03:57.392943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.023712 (+630769us) service_pool.cc:224] Handling call
0709 13:03:58.024089 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.033763 (+  9674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.033800 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.034259 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.437391 (+403132us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.437429 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.437500 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.487631 (+ 50131us) raft_consensus.cc:1685] finished
0709 13:03:58.487658 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.487863 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13545,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11067,"thread_start_us":320911,"threads_started":1,"wal-append.queue_time_us":367921}
I20260709 13:03:58.494149  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.494567  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.495431  1700 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:58.495957  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:58.497185  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 343809us
I20260709 13:03:58.498152  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:58.499416  1626 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:58.497114  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:58.501737  1630 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:03:58.502285  1559 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:58.504014  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:58.505544  8684 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:58.489732  8709 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:03:58.514498  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 334992us
I20260709 13:03:58.517096  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.508718  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.485900  9064 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:58.504112  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:58.497563  1701 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:58.529778  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.502509  9147 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.493367  9051 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.490543  9057 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:58.507530  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.521890  1704 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
W20260709 13:03:58.524117  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.533383  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.547570  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.549175  1558 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:58.514576  8709 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:58.513824  9059 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.551213  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:58.552354  1552 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:03:58.554171  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46756.1us
I20260709 13:03:58.510167  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:58.561450  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51377.1us
I20260709 13:03:58.534473  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:58.506414  8949 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:03:58.541101  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54578.4us
I20260709 13:03:58.567662  1700 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:58.508879  8684 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:58.570277  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.564102  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:58.532079  8872 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.528534  9064 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:58.575585  1700 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:58.579349  1523 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.598232  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.583033  8877 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:58.604274  8877 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:58.604799  8877 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.570015  9052 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:58.612547  9003 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.613457  8709 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.573933  8612 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:58.599846  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.540165  9074 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.622812  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.623972  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:58.628743  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.630256  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:03:58.589624  9102 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.634640  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:58.633592  1700 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:03:58.632434  9020 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:03:58.632202  9003 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:58.640190  9003 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:58.640625  9003 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.642093  9078 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:58.616204  1552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:03:58.645396  9074 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.633821  8843 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:03:58.634537  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:58.646555  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:58.647704  1558 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:58.648943  1558 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:58.631619  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:58.548228  9147 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.566550  9059 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.660154  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.614005  8877 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.626924  8709 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:58.620885  8949 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:58.668826  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.619990  9160 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.592880  8872 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.632584  9092 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25.
I20260709 13:03:58.672839  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:58.689884  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.647728  8564 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:58.633931  8612 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.663635  9062 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:58.665357  9082 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:03:58.671600  9074 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:58.667819  8843 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.637876  9051 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:03:58.700264  8877 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.701859  8872 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.674149  9020 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:58.708757  1703 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:58.709316  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:58.719048  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:03:58.712738  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44012.8us
I20260709 13:03:58.658854  8684 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.691747  8949 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:58.695123  8564 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.721315  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.711375  9068 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:58.717370  9074 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:58.716656  8843 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.730604  8684 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.731207  8564 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.732435  9162 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:58.734488  9147 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.725754  9090 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:58.723984  8769 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:58.726248  9057 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.011s
W20260709 13:03:58.745450  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.714826  9062 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.746483  9074 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.792655  9117 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:58.793074  9117 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:03:58.793879  9117 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.795047  9117 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.786120  8612 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:58.796192  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 3149) took 1228 ms (client timeout 1565 ms). Trace:
I20260709 13:03:58.796304  1704 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:58.796614  1705 rpcz_store.cc:269] 0709 13:03:57.568232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.671734 (+103502us) service_pool.cc:224] Handling call
0709 13:03:58.795973 (+1124239us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b540016c454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:58.796102 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103790976}
I20260709 13:03:58.752487  8872 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.797905  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.797835  8872 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.798492  1705 raft_consensus.cc:2376] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 21. Re-sending same reply.
I20260709 13:03:58.799314  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.800217  9090 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:58.801293  9090 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.798506  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1222) took 1415 ms. Trace:
I20260709 13:03:58.804617  1629 rpcz_store.cc:276] 0709 13:03:57.383594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.020759 (+637165us) service_pool.cc:224] Handling call
0709 13:03:58.142137 (+121378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.150713 (+  8576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.798318 (+647605us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000cd0d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:58.798451 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261446400}
I20260709 13:03:58.806025  9062 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.807992  8872 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.726861  8726 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:58.809435  8726 consensus_queue.cc:1243] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer 444b494bb53842b59408a7230884398a log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:58.809916  8726 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:58.810235  8726 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:58.811000  8726 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.829391  8726 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.813351  9059 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.715122  9055 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.831643  8578 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:58.832309  8578 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:58.832949  9055 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.837566  9059 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:03:58.838197  9059 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.840140  1705 raft_consensus.cc:2376] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 19. Re-sending same reply.
I20260709 13:03:58.841254  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.841912  1705 raft_consensus.cc:2376] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 20. Re-sending same reply.
I20260709 13:03:58.842912  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.844720  1705 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:58.846160  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:58.816431  8843 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.782804  9185 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:58.813230  9003 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.856897  1556 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:58.781170  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:58.869699  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1237) took 1170 ms. Trace:
I20260709 13:03:58.827809  8564 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:58.877660  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.876650  1634 rpcz_store.cc:276] 0709 13:03:57.699565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.140657 (+441092us) service_pool.cc:224] Handling call
0709 13:03:58.160294 (+ 19637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.160499 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:58.407225 (+246726us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.662140 (+254915us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.869310 (+207170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.869374 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:58.869423 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.869494 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.869633 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12235,"prepare.queue_time_us":294143,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2288,"raft.queue_time_us":40066,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":20596,"thread_start_us":648231,"threads_started":4,"wal-append.queue_time_us":127805}
I20260709 13:03:58.870823  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:03:58.881247  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 100135us
I20260709 13:03:58.879374  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:58.883678  1705 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:58.886898  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:58.827664  9160 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:58.914793  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.893100  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.867947  9059 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.872642  9057 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.913324  1552 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:58.916589  1552 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.915784  2292 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.918111  9162 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:58.918185  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.918728  9162 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.919983  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:58.920884  9162 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:58.891420  8578 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:58.926879  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:58.923530  8578 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.928717  1551 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:58.929113  1556 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:58.930400  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:58.890738  9055 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:58.891745  8684 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:03:58.931100  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:58.938546  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:58.939899  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.941370  2292 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:03:58.941815  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:58.944193  2296 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:58.945075  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:58.947484  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:58.947983  1552 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:58.954391  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:58.993875  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2715) took 1511 ms. Trace:
I20260709 13:03:58.994313  1553 rpcz_store.cc:276] 0709 13:03:57.482791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.482967 (+   176us) service_pool.cc:224] Handling call
0709 13:03:57.501445 (+ 18478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.921173 (+419728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.993596 (+1072423us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54002607d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:58.993805 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":963923200}
W20260709 13:03:58.999559  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 45268.5us
I20260709 13:03:59.004460  2291 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:59.004833  2294 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:03:58.942664  9089 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:58.949239  9059 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:59.006982  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.007880  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.008311  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.015344  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.017359  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.018723  1556 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 21. Ops: []
I20260709 13:03:58.929661  9057 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.019836  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.018134  1626 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:03:59.020972  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.021580  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.038160  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.040961  1631 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25.
I20260709 13:03:59.048975  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.050928  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.051537  1634 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:59.050961  2294 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:03:59.053475  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.054497  1634 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:59.018862  9102 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.004801  8612 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:59.057500  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.022118  9104 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.042300  9078 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.042390  9089 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:03:59.064277  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48544.8us
I20260709 13:03:59.066349  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.069612  1634 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:59.071866  2290 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
W20260709 13:03:59.074438  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.050527  9059 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:59.077541  1634 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:59.078600  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:59.067097  9183 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:03:59.080687  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:59.081573  9196 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:59.086807  9019 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:59.087267  9019 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.088164  9019 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.089442  9019 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:59.090334  8710 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:59.091169  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:59.091575  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.073035  9104 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.095420  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3142) took 1532 ms. Trace:
I20260709 13:03:59.095999  9104 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.096297  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.096971  1700 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:03:59.097821  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:59.098336  1700 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:59.101962  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.102942  9168 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.103538  9168 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.105005  8709 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.105363  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.105873  9168 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.105576  8709 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.106431  9168 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.107053  9168 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.108398  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.109313  9057 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.112684  9059 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.114100  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3117) took 1941 ms. Trace:
I20260709 13:03:59.072489  9078 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.075412  9089 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.117102  8892 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:59.110982  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 612883us
I20260709 13:03:59.115947  9054 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.065876  9102 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.120054  9003 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.122023  9102 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:59.122207  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 486512us
I20260709 13:03:59.077651  8612 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:59.123620  8612 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.119503  9089 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:59.091600  2293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57942 (request call id 1553) took 1866 ms (client timeout 1788 ms). Trace:
W20260709 13:03:59.130873  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50273.4us
I20260709 13:03:59.130020  1635 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.135212  1705 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:03:59.118805  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:03:59.095819  1709 rpcz_store.cc:276] 0709 13:03:57.563229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.030842 (+467613us) service_pool.cc:224] Handling call
0709 13:03:58.064040 (+ 33198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.158651 (+ 94611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.095161 (+936510us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54001cbbd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:59.095361 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211633920}
I20260709 13:03:59.131309  1523 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.132511  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:59.144572  1551 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:59.146706  1555 raft_consensus.cc:1240] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:59.147698  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.152120  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:59.152870  1705 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.155007  1628 raft_consensus.cc:2410] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:03:59.132931  9089 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:59.165198  2294 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:03:59.168107  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 49421.4us
W20260709 13:03:59.129347  2293 rpcz_store.cc:269] 0709 13:03:57.224788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.244559 (+ 19771us) service_pool.cc:224] Handling call
0709 13:03:59.091423 (+1846864us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400729054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:59.091534 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383232384}
I20260709 13:03:59.162688  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.170797  1523 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:59.169623  1598 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:03:59.114511  1557 rpcz_store.cc:276] 0709 13:03:57.172757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.174451 (+  1694us) service_pool.cc:224] Handling call
0709 13:03:57.174756 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:57.179785 (+  5029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.179815 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:57.180363 (+   548us) log.cc:844] Serialized 52 byte log entry
0709 13:03:57.499001 (+318638us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:57.524364 (+ 25363us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.930399 (+406035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.930465 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.235455 (+304990us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b180010ebe0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:59.078489 (+843034us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b180010ebe0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:03:59.113874 (+ 35385us) raft_consensus.cc:1685] finished
0709 13:03:59.113899 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.114037 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30911,"prepare.queue_time_us":56866,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":154945,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":180,"spinlock_wait_cycles":277267072,"thread_start_us":723956,"threads_started":2,"wal-append.queue_time_us":540053}
I20260709 13:03:59.177938  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.178609  1709 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:59.179654  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.080717  8564 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.182488  1703 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:59.182711  1635 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:59.186457  8949 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.189220  8726 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.189911  9104 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:59.190557  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:59.195086  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.204710  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.194371  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:59.195819  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.196448  2289 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:03:59.206408  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:03:59.196450  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.127408  8937 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.190975  9186 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:03:59.200016  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.199853  8949 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.201557  9168 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:59.205343  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.210548  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:59.216284  8914 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:59.221355  8914 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.221738  8914 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:59.222558  8949 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.216472  9062 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:59.216214  9092 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:59.225373  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.225657  9113 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:59.227313  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.228657  8564 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:59.229169  9104 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:59.229360  9186 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:59.229992  9104 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:59.230398  9104 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.234314  1557 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:59.178474  9160 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:59.236655  9160 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:59.237632  9160 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.243013  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:59.245643  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.246702  1554 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:59.181994  9043 consensus_queue.cc:1059] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:03:59.247804  9043 raft_consensus.cc:3055] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:59.248191  9043 raft_consensus.cc:740] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:59.249125  9043 consensus_queue.cc:260] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.250348  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1560) took 1219 ms. Trace:
I20260709 13:03:59.178102  9095 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:03:59.204993  9117 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:59.183333  9159 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:59.239426  9078 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.250546  9043 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.175529  9003 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:59.226156  1704 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:03:59.216133  1705 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:03:59.258311  9159 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.258895  1557 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:59.280920  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:59.261133  9167 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.261762  9144 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.250749  2288 rpcz_store.cc:276] 0709 13:03:58.031349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.031520 (+   171us) service_pool.cc:224] Handling call
0709 13:03:58.031867 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.190983 (+159116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.191029 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.191456 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.449231 (+257775us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:58.449279 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.249933 (+800654us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:59.249990 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.250033 (+    43us) raft_consensus.cc:1685] finished
0709 13:03:59.250047 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.250282 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4421,"raft.queue_time_us":4576,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6823,"spinlock_wait_cycles":354501888,"thread_start_us":223568,"threads_started":1,"wal-append.queue_time_us":289099}
I20260709 13:03:59.213579  1700 raft_consensus.cc:2410] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:59.218781  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.179919  9057 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.250888  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.251441  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.120112  9054 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:03:59.252158  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:59.253713  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.254076  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.350912  1702 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:03:59.353803  1702 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:59.355324  1702 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:59.254711  1554 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:59.275055  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:03:59.356513  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 81606us
I20260709 13:03:59.276777  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.357607  2295 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:03:59.282351  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.209892  8843 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:59.360105  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:59.281162  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.368789  1557 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:03:59.285713  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.304270  1709 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:03:59.220280  8709 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.231272  9104 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:03:59.305583  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54783.4us
I20260709 13:03:59.180811  9068 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.098s
I20260709 13:03:59.252904  9092 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.370047  1704 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:03:59.377226  1708 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:59.371348  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.283735  9003 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:03:59.388113  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.253587  9020 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 13:03:59.268102  8937 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.306556  9117 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.371083  8949 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:59.384200  9120 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:59.256067  9195 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.136s
I20260709 13:03:59.373402  8843 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.428869  9052 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:59.434546  1552 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.399689  9100 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:59.441254  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.449375  1631 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:03:59.286733  9159 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:59.445336  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58846 (request call id 1284) took 1767 ms (client timeout 1958 ms). Trace:
W20260709 13:03:59.450791  1707 rpcz_store.cc:269] 0709 13:03:57.677578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.909811 (+232233us) service_pool.cc:224] Handling call
0709 13:03:59.445153 (+1535342us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540010d1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:03:59.445272 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417908608}
I20260709 13:03:59.369324  9057 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.389514  9092 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.442443  9120 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.453285  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.304581  9167 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.346092  9144 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.373159  9168 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:59.374042  8684 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 13:03:59.401556  9104 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.376634  8914 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.445130  8949 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:59.456290  1631 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:03:59.445372  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.458768  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:03:59.441454  8578 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.509s	user 0.003s	sys 0.007s
I20260709 13:03:59.446158  8843 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:59.446419  9052 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:03:59.463418  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.464275  2291 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:59.465713  9159 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:59.465766  9054 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.466156  9144 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.467363  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:03:59.486133  2292 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:03:59.487589  8914 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.488054  9075 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:03:59.488941  9075 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.477535  9185 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:03:58.889752   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:59.457834  9196 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.470031  9003 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.440732  9117 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.460353  9092 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.472350  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.456271  9113 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:59.451887  9100 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.517272  9100 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:59.478399  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.462165  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.457508  9120 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:59.452191  9119 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.523885  9113 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.524046  9119 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.524928  9113 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.480422  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.482679  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.483749  8949 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.488241  8914 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:59.532188  8914 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.533281  9167 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.533147  8914 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.469169  8937 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.504119  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:59.460948  9074 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.512313  1701 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
W20260709 13:03:59.516495  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44251.3us
I20260709 13:03:59.526104  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.528085  9119 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:03:59.530671  1451 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 213) took 24803 ms (client timeout 15000 ms). Trace:
I20260709 13:03:59.542227  1551 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:59.545934  1632 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 14.
I20260709 13:03:59.545986  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:03:59.549600  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.534452  9059 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:59.528107  9113 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:59.555166  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.555925  9068 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:59.536197  8914 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:59.572778  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:59.552825  1451 rpcz_store.cc:269] 0709 13:03:34.727659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.727946 (+   287us) service_pool.cc:224] Handling call
0709 13:03:59.530521 (+24802575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:54.111988 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:54.182487 (+ 70499us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:54.182547 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:03:54.182563 (+    16us) tablet.cc:662] Decoding operations
0709 13:03:54.339879 (+157316us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:54.339988 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:54.340033 (+    45us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:03:54.341783 (+  1750us) tablet.cc:701] Row locks acquired
0709 13:03:54.341840 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:03:54.342018 (+   178us) write_op.cc:270] Start()
0709 13:03:54.342174 (+   156us) write_op.cc:276] Timestamp: P: 1783602234341951 usec, L: 0
0709 13:03:54.342227 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 13:03:54.343077 (+   850us) log.cc:844] Serialized 7494 byte log entry
0709 13:03:55.028457 (+685380us) op_driver.cc:464] REPLICATION: finished
0709 13:03:55.237621 (+209164us) write_op.cc:301] APPLY: starting
0709 13:03:55.252239 (+ 14618us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:55.652211 (+399972us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:55.652352 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:56.846430 (+1194078us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:56.846874 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:03:56.846935 (+    61us) write_op.cc:312] APPLY: finished
0709 13:03:56.847416 (+   481us) log.cc:844] Serialized 172 byte log entry
0709 13:03:56.905973 (+ 58557us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:56.935142 (+ 29169us) write_op.cc:454] Released schema lock
0709 13:03:56.951417 (+ 16275us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:56.960264 (+  8847us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:56.960298 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2693177,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":168756,"apply.run_cpu_time_us":56092,"apply.run_wall_time_us":1723272,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83623,"num_ops":19,"prepare.queue_time_us":250807,"prepare.run_cpu_time_us":16240,"prepare.run_wall_time_us":630305,"raft.queue_time_us":221676,"raft.run_cpu_time_us":15127,"raft.run_wall_time_us":122580,"replication_time_us":686041,"thread_start_us":623458,"threads_started":5,"wal-append.queue_time_us":507278,"wal-append.run_cpu_time_us":36859,"wal-append.run_wall_time_us":2406141}]]}
I20260709 13:03:59.513908  9196 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.582253  9196 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.579883  8863 raft_consensus.cc:3060] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:59.585472  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.588658  2292 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:59.588389  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.590128  1525 leader_election.cc:400] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25.
I20260709 13:03:59.591933  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.592967  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.598214  1551 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:03:59.596803  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.595332  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.600245  2290 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.595618  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.600908  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.601087  1559 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:03:59.610651  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:59.589200  9210 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:59.613087  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.547891  9074 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.590528  9068 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:59.600356  9207 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:59.620590  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.619750  9201 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.588867  1450 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:59.567793  9052 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.642465  2290 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:59.636940  9201 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:03:59.645666  9201 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.645836  1450 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 19, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 19 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:59.619329  9092 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.637902  9207 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.615290  9020 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:03:59.661597  1634 raft_consensus.cc:2368] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:59.648135  9052 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:59.651422  9210 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:59.643050  9209 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:59.635512  9120 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:03:59.671969  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58961.6us
I20260709 13:03:59.673378  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.676730  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:03:59.662875  9207 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:59.669739  9210 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.687316  9120 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:59.693045  9092 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.693849  9092 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:59.694360  9092 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:03:59.696797  9074 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.679935  9160 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:59.683784  9210 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:59.713192  8998 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:59.647746  1450 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:59.719920  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3153) took 1923 ms. Trace:
I20260709 13:03:59.724195  1633 rpcz_store.cc:276] 0709 13:03:57.796142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.141529 (+345387us) service_pool.cc:224] Handling call
0709 13:03:58.141910 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.333105 (+191195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.719551 (+1386446us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001e9454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:03:59.719839 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438262528}
I20260709 13:03:59.731245  1525 leader_election.cc:400] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:59.734395  1553 raft_consensus.cc:1240] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:59.715896  8914 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.738758  9201 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.741084  9057 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.717528  8998 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.744320  9117 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.716102  8843 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.695297  9092 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.715322  9100 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.756942  1450 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:59.766247  1558 raft_consensus.cc:1275] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 13:03:59.683279  9213 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:03:59.773144  9075 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.779181  1555 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:59.780454  1525 leader_election.cc:403] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:03:59.800951  1558 pending_rounds.cc:85] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 12
I20260709 13:03:59.805721  1558 pending_rounds.cc:107] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 13:03:59.805969  1558 raft_consensus.cc:2889] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:59.808156  9201 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.810197  1450 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 22, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 22 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:03:59.812256  8998 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.813475  9100 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:03:59.813657  8998 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:59.817744  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86736.7us
I20260709 13:03:59.822080  1630 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:59.822633  9213 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:59.790622  8914 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.824556  9159 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:03:59.824373  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 320324us
I20260709 13:03:59.829574  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.821630  9075 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:03:59.830919  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 245392us
I20260709 13:03:59.825060  9159 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.834697  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:03:59.835561  1634 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:59.819468  8863 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:03:59.836256  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:03:59.836921  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:59.837772  1559 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:59.855593  1628 raft_consensus.cc:1275] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:59.839471  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:59.834235  1630 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:59.857527  1555 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:59.838011  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:59.865612  2292 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:59.858119  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:03:59.853026  9119 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.855958  1556 raft_consensus.cc:2393] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:03:59.869086  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:03:59.832934  9100 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:03:59.876070  1672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.837080  8872 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.853926  9159 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:59.884091  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.882112  1707 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:59.876679  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.882614  2292 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
I20260709 13:03:59.886572  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:59.883419  8709 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.887859  8709 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.889508  9168 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:59.889907  8709 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.890842  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:03:59.877080  9119 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.891865  1708 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:03:59.893011  9119 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:03:59.876897  9144 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:59.893836  9144 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.894197  9144 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:59.900352  1705 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:03:59.902374  8843 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:03:59.903472  8914 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:03:59.903755  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.890067  9168 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:03:59.928571  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 51192.2us
I20260709 13:03:59.935110  9159 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.936524  1703 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: [22.14-24.15]
I20260709 13:03:59.938580  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:03:59.939213  2293 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:59.909911  9100 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:03:59.941960  1598 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:59.947547  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.931003  8914 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:59.925156  9201 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:03:59.957315  2293 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:03:59.959002  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:03:59.961133  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:03:59.970561  9168 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:03:59.919643  8872 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:03:59.977312  1598 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:03:59.944238  8710 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.963126  9142 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:59.960889  9159 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.013291  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.014622  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.015215  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.015623  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.016021  1705 raft_consensus.cc:2393] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:00.016624  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.016839  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:03:59.959954  8914 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:00.017668  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:03:59.964636  8951 raft_consensus.cc:3055] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:59.960127  8998 consensus_queue.cc:1059] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:04:00.005388  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:00.000792  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 3159) took 1484 ms (client timeout 1640 ms). Trace:
I20260709 13:03:59.991444  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.031345  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.006018  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.011956  1633 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:59.990867  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:00.031975  1634 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:00.012635  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.030881  1626 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:59.962966  8726 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:59.966662  9201 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.004426  8863 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:03:59.967257  9080 raft_consensus.cc:2764] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:03:59.984987  8710 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.045236  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.045399  9201 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.045827  1552 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:00.006819  9207 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.002177  2292 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:00.046542  9062 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:00.046908  8710 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.022609  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:03:59.989893  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.047879  9207 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.043675  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.048516  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.049191  1554 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.019565  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.017854  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.051170  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.024505  1704 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:00.051707  1705 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.052553  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.052706  9080 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.053120  1704 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:00.053247  9080 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.035221  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.035946  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.035926  9100 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.004395  9209 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:04:00.037207  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.038143  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.067456  1628 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:00.067385  1626 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:00.068732  1626 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:00.069859  1626 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:00.024811  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:03:59.997395  9204 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.531s	user 0.001s	sys 0.001s
W20260709 13:04:00.008985  9223 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:04:00.038941  9227 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:59.990576  9213 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
W20260709 13:04:00.030851  1701 rpcz_store.cc:269] 0709 13:03:58.516051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.529472 (+ 13421us) service_pool.cc:224] Handling call
0709 13:04:00.000626 (+1471154us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54001b7454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:00.000735 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2191881600}
I20260709 13:04:00.040469  1560 raft_consensus.cc:1240] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:00.041123  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.041769  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:03:59.981683  9049 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:04:00.042596  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.078842  1632 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:00.004004  9220 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:00.043920  8726 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:00.080168  8992 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:00.080735  8726 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:00.081266  8863 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:00.045113  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.002863  9210 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.077422  9051 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:00.055040  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.004221  8769 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:04:00.024645  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.055956  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.097121  9210 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.056367  1555 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:04:00.113718  9210 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:04:00.078630  9175 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.379s	user 0.005s	sys 0.011s
I20260709 13:04:00.048007  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:04:00.118530  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 70609us
I20260709 13:04:00.059396  1554 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:00.060070  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2735) took 1203 ms. Trace:
I20260709 13:04:00.072538  9142 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:00.119503  1558 rpcz_store.cc:276] 0709 13:03:58.856741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.856893 (+   152us) service_pool.cc:224] Handling call
0709 13:03:58.857188 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.806277 (+949089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.026632 (+220355us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540036d3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:00.060008 (+ 33376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089438592}
I20260709 13:04:00.021957  2291 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:59.878965  1450 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:00.062590  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 56683.8us
I20260709 13:04:00.063373  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.064766  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:00.124185  1629 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23.
I20260709 13:04:00.003216  8612 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:00.071029  1634 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:04:00.072788  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48066.5us
W20260709 13:04:00.076929  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 57479.6us
I20260709 13:04:00.134135  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.077494  1705 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:03:59.992354  8872 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.020408  8914 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.020614  9059 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:00.080076  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 88294.6us
W20260709 13:04:00.144153  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.119081  8769 consensus_queue.cc:1243] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:00.090931  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:00.092075  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:00.020534  8951 raft_consensus.cc:740] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:00.094928  9220 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:00.095150  9051 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.084285  9117 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:00.096642  9080 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.055902  9104 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:00.057861  9120 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.048332  9020 consensus_queue.cc:1048] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:04:00.066913  9201 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.113564  1551 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:00.136250  9055 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.128866  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:00.126921  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 117750us
I20260709 13:04:00.135560  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.138266  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:00.140846  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.073340  9207 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 13:04:00.089931  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:00.183691  2295 raft_consensus.cc:2368] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 25. Current term is 26.
W20260709 13:04:00.183583  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 48134.4us
I20260709 13:04:00.149880  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.184423  1634 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:00.184509  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.180541  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:00.155138  2289 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 24. Current term is 25. Ops: [20.13-24.15]
I20260709 13:04:00.122759  9095 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.127779  9159 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:00.251997  9159 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.252576  9159 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.097795  9119 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
W20260709 13:04:00.124433  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 61121.1us
I20260709 13:04:00.131922  8612 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:00.260450  8612 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.261723  8612 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:00.183104  8951 consensus_queue.cc:260] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.265543  8951 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.182994  9117 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.269003  9117 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.272866  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.273710  1551 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:00.135959  9210 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:00.173166  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:00.180624  2290 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:00.141902  9054 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.144194  9059 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:00.183385  9104 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:00.226816  9167 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.190056  1599 leader_election.cc:400] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 25. Current term is 26.
I20260709 13:04:00.134666  9052 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 13:04:00.188143  2292 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:00.188856  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.244644  9095 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.191061  1705 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:00.275498  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.169451  9051 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.135835  9049 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:04:00.277887  9075 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:00.287107  9221 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:04:00.315590  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 126831us
I20260709 13:04:00.358031  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.279557  9210 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:00.315191  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 125323us
I20260709 13:04:00.292743  9117 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.187520  9055 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.393657  9055 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.394457  2297 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:00.243072  1450 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 20, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 20 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:00.397707  9229 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 13:04:00.397236  1450 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.401132  9095 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.345546  9075 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.409502  9075 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.356938  9167 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:00.392246  8914 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.412281  9167 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.412703  9167 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.417253  9159 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.446791  1525 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23.
I20260709 13:04:00.452237  1525 leader_election.cc:403] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 13:04:00.464615  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 189201us
I20260709 13:04:00.174467  8863 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.263s
I20260709 13:04:00.368471  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.372732  8933 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.319094  8958 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.319001  8709 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.289343  9142 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.289657  9144 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.384004  8710 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:00.384419  9228 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 13:04:00.385058  9059 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.388090  9210 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:00.442828  8984 consensus_queue.cc:1048] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:04:00.366817  9213 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.363734  2290 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:00.487550  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:00.440603  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52620.1us
I20260709 13:04:00.498060  1629 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:00.498337  1598 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:00.441869  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 79986.3us
W20260709 13:04:00.472538  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 343780us
I20260709 13:04:00.499722  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:00.500365  1628 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:00.173888  8872 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.269s
I20260709 13:04:00.437986  9095 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.438228  9119 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:00.167970  9220 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:00.443706  9049 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.399144  1450 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.448359  9185 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:00.468353  9160 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23.
I20260709 13:04:00.483251  2292 raft_consensus.cc:1240] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:00.506285  8949 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.442948  9187 consensus_queue.cc:1059] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:04:00.557320  9187 raft_consensus.cc:3055] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:00.557832  9187 raft_consensus.cc:740] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:00.244402  9201 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:00.540323  9119 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.502840  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.508133  1707 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:00.469581  8682 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:04:00.564421  8682 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:00.564980  8682 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:00.542583  9090 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:04:00.566045  8682 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.568157  9196 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:00.475519  8726 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.537900  1631 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:00.568477  1450 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 24, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 24 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:00.485479  8958 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:00.481794  8933 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.570158  1450 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 20, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 20 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:00.497164  9057 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:00.571458  9057 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.417382  9104 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.572063  9220 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.573520  9057 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.485754  9064 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.574643  1630 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:00.545271  9096 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.576226  9096 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.578189  9167 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.485319  8709 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.573189  9185 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:00.590989  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 297180us
I20260709 13:04:00.593211  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.511358  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:00.570825  8726 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.585410  1630 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
W20260709 13:04:00.599496  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.558876  9187 consensus_queue.cc:260] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.610884  1629 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:00.540246  9100 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.589828  8933 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.578598  9096 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.547850  8998 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.614815  1628 pending_rounds.cc:85] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 13
I20260709 13:04:00.510632  9213 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.547640  8914 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.621402  1628 pending_rounds.cc:107] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:04:00.626920  1628 raft_consensus.cc:2889] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:00.617013  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 109318us
I20260709 13:04:00.571700  8682 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:00.582459  9059 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.632249  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:00.632669  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:00.633036  2296 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:00.633458  2297 raft_consensus.cc:2368] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 25. Current term is 26.
I20260709 13:04:00.635064  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:00.635812  2297 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:00.658666  2297 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:04:00.513223  9095 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.666226  1599 leader_election.cc:400] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 25. Current term is 26.
I20260709 13:04:00.666690  1599 leader_election.cc:403] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:04:00.620647  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:00.631494  8933 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:00.667418  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46848.4us
I20260709 13:04:00.667568  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:00.673415  8958 consensus_queue.cc:1059] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:00.680184  9236 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.681727  9144 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.682559  9144 raft_consensus.cc:3055] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:00.682989  9144 raft_consensus.cc:740] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:04:00.685896  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.686820  8933 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.689630  9119 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.690698  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:00.691468  1555 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.692073  9080 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 25. Current term is 26.
I20260709 13:04:00.698683  8709 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.612174  1450 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 18, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:00.704519  1450 catalog_manager.cc:5697] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.582617  9049 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.706453  1450 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 22, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 22 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.707952  9049 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.664093  9074 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.710567  9074 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.671044  9241 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.713129  9213 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:00.715422  9054 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:00.716096  9054 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.718716  9054 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.721042  9187 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:00.721858  9074 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:00.725731  9226 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.706s	user 0.003s	sys 0.000s
I20260709 13:04:00.726770  9095 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.613127  9051 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.728219  9064 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:00.729317  9051 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.730098  9167 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.731194  9144 consensus_queue.cc:260] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.623548  9201 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:00.749703  1555 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:04:00.736491  2293 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:04:00.712730  9241 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:00.740624  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 73147.4us
I20260709 13:04:00.734859  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.705245  9185 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.755683  1630 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:00.681017  9236 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.731477  9075 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.623926  8726 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.133s
I20260709 13:04:00.750082  9201 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:00.772487  1626 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:00.752765  9242 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.623937  9100 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.754223  9159 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:00.767620  1524 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:00.763680  9167 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.764855  9168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.033s	sys 0.046s
I20260709 13:04:00.790124  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:00.791988  9168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.034s	sys 0.046s
I20260709 13:04:00.797317  8914 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.797501  9082 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:00.798167  9082 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:00.798000  8914 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.799088  9082 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:00.778330  8612 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:04:00.804736  1450 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 23, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:00.810024  9075 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:00.830459  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.813437  1555 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:04:00.814213  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.777312  9201 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:00.823503  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.824800  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.812224  9092 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:00.826452  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.837430  1557 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
I20260709 13:04:00.827474  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:00.827735  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 459362us
I20260709 13:04:00.829797  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:00.839923  1553 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.834060  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:00.842060  1555 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.843216  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.751914  9059 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:00.835790  1552 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:04:00.825800  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:00.835165  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.845646  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:00.847029  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.851887  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.855875  1555 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:00.857607  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:00.858928  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 360677us
I20260709 13:04:00.859899  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:00.862998  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.863638  1555 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:00.886389  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:00.887535  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:00.888208  1559 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:04:00.890053  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 244630us
I20260709 13:04:00.892004  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:00.819001  9167 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:00.902840  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.897975  1553 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:00.787281  9242 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:00.840827  9120 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:00.856730  9168 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:00.840278  9159 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:00.900748  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.858384  9092 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:00.863898  9054 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:00.897038  8998 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:00.879793  9117 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:00.913223  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.879951  9059 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:00.889986  8709 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:00.886260  8710 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:00.911967  1675 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:00.889552  9078 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.936681  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:00.944569  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 58277us
I20260709 13:04:00.882790  9113 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:04:00.964219  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52348.1us
W20260709 13:04:00.956602  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.956665  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:00.967252  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.935407  9043 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:00.964383  1601 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:00.971007  9250 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:00.935895  9054 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.975939  1707 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:00.950888  9055 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:00.977576  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:00.979142  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.980324  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.002369  1701 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:01.008498  1709 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:00.981343  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.010460  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.011168  1559 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:00.984824  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.941187  8998 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.988720  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.989368  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:00.990302  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 53753.8us
I20260709 13:04:00.976652  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:00.998471  1702 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:00.983302  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:00.962888  9168 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:00.969408  8863 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.022004  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:00.988853  9043 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.026472  1709 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:00.968780  9117 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:00.972363  8984 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:01.044803  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.045449  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:01.046550  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.047303  1706 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:04:01.049049  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.020095  1700 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.020539  1708 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:01.012928  1703 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:04:01.051390  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.011185  1553 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:00.962611  8914 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.065121  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:01.023250  1701 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:01.021760  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.021724  8998 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:01.029428  9049 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:00.969050  8710 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.030130  9253 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:01.030416  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:01.030995  9244 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:01.032174  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.043390  8933 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.036156  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.037132  9242 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.026089  9185 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.038139  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:01.023986  9168 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:01.040426  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:01.042040  9220 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.000530  9054 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.049847  1702 raft_consensus.cc:2376] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 20. Re-sending same reply.
I20260709 13:04:01.015007  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:01.002070  9142 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.011508  9001 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:01.011314  9055 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.049518  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.020815  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:01.022970  9210 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 13:04:00.969180  9078 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.065550  1705 raft_consensus.cc:2368] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23.
I20260709 13:04:01.013103  9250 raft_consensus.cc:697] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:01.045679  9043 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:01.011018  8709 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:04:01.025241  9241 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.126227  9242 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:01.103094  9253 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.093528  9059 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.139925  8992 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:01.140339  8992 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:01.134796  9075 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.141050  9075 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.141274  8992 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.145184  1635 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:01.144948  8992 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.034157  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.146869  1631 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:04:01.105319  1629 raft_consensus.cc:2393] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:01.125038  8933 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:01.151957  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.152434  8933 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.152729  1559 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:01.152878  8933 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:01.154368  8984 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:01.154866  8984 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.157032  8710 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.127007  9057 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.127564  9055 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.135859  9210 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.164875  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.166544  9210 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.168221  9167 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.172966  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.174767  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.113428  9168 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.042176  9092 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.180195  9049 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:01.180730  9049 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:01.184379  9260 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:01.101073  1626 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:04:01.111323  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.187451  1703 raft_consensus.cc:2410] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 13:04:01.188557  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.178603  9078 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.189949  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.090807  9064 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:01.191504  9064 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.191998  9064 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:01.190824  1703 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:01.062474  9207 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
W20260709 13:04:01.159583  9251 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:04:01.044950  8863 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:01.112660  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47802.2us
I20260709 13:04:01.116513  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.114338  1560 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: [22.10-22.10]
W20260709 13:04:01.119539  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 104611us
W20260709 13:04:01.229206  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.936352  1450 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 20 to 21, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:01.124804  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 104068us
I20260709 13:04:01.125181  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.230376  1709 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:01.104049  9100 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:01.125684  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 76231.8us
I20260709 13:04:01.237004  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.132223  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:01.132925  1708 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:01.131379  1700 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:01.145567  9250 consensus_queue.cc:237] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.185995  9144 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.062199  9239 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:04:01.062768  9246 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:01.175482  1701 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:01.133803  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:01.202746  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.204936  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.207999  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:01.209259  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.143435  9075 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.092338  9236 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:01.209350  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:01.264675  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55416.3us
I20260709 13:04:01.265287  1601 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:01.222101  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:01.266803  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 44830.4us
I20260709 13:04:01.223292  9104 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:01.269554  2292 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:01.270028  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:01.224843  1706 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:01.114364  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.231534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.232358  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.227497  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:01.233124  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.275662  1555 raft_consensus.cc:2393] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:01.233619  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:01.226125  8682 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 13:04:01.257207  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.278595  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:01.257795  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:01.140352  9253 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:01.255393  9271 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:01.253394  9210 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:04:01.279268  2296 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
W20260709 13:04:01.280893  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 76053.4us
W20260709 13:04:01.297395  9269 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:04:01.299881  9062 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:01.302402  9257 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 13:04:01.218453  9120 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.283382  1554 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:01.287895  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:01.288957  1673 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:01.268898  8872 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:01.239115  9096 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.297433  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.254658  9090 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.239387  9247 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.239354  9119 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:04:01.278384  9142 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.283218  9236 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:01.361826  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64471us
I20260709 13:04:01.284101  9117 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.245887  8958 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.093s
W20260709 13:04:01.340255  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 51460.9us
I20260709 13:04:01.292900  8951 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:01.268602  9104 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.313078  9253 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:01.395068   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:01.361079  9167 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:01.382349  9187 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.000s
I20260709 13:04:01.359722  8872 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:01.369360  8710 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:01.286751  8998 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.361367  9120 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:01.431274  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 165087us
I20260709 13:04:01.282888  9246 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.412222  9059 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:01.459831  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 182818us
I20260709 13:04:01.426018  8951 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.382015  9096 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.397109  9051 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:01.474493  1700 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:01.397456  9247 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.478297  1627 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:04:01.478899  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.452567  8872 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:01.485200  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.492038  1626 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:01.492925  1634 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:01.488210  1635 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:01.475354  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:01.454360  9142 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:01.453922  9119 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:01.501600  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.495651  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:01.462268  9117 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.500422  1635 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 13:04:01.503253  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:01.503746  1635 pending_rounds.cc:85] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 13
I20260709 13:04:01.504274  1635 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:04:01.504825  1635 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:01.505335  1635 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 13:04:01.505698  1635 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:01.466150  9051 consensus_queue.cc:1243] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:01.465452  8951 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:01.462001  9059 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:01.510263  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 219732us
I20260709 13:04:01.509543  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:01.513725  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 260341us
I20260709 13:04:01.436131  9253 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.510087  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.528518  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.463124  9054 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.542774  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.543056  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.544131  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.565659  1626 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:04:01.567229  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.514340  1599 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:01.568610  1631 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:01.650803  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:01.648908  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 228140us
I20260709 13:04:01.522424  9243 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:01.497285  9142 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:01.544441  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:01.653021  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:01.655102  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 140867us
I20260709 13:04:01.651677  1704 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:01.674844  1702 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:01.519582  9001 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:01.693719  1703 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:01.517580  9119 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.517786  9117 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.551951  8863 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.552731  8710 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:01.552718  9266 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:01.785043  9256 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:04:01.669214  9236 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:01.679657  9159 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.017s	sys 0.016s
I20260709 13:04:01.665498  9243 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:01.697634  8726 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.671880  9078 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:01.842070  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 188902us
I20260709 13:04:01.843338  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:01.833490  9159 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.017s	sys 0.026s
I20260709 13:04:01.668352  9075 consensus_queue.cc:1059] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.132s
W20260709 13:04:01.719198  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 172433us
I20260709 13:04:01.799973  9283 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.846789  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 302413us
I20260709 13:04:01.858668  1523 leader_election.cc:400] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23.
I20260709 13:04:01.818396  9236 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.832902  9243 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:01.847666  1702 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:01.836349  9078 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.835904  8998 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.420744  1450 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 18, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 18 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:01.837602  9100 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.808218  9266 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.837677  9104 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.795324  9059 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.805200  8710 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.864188  9120 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:01.870750  9078 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.850096  9159 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:01.867592  9236 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.808794  9142 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.903404  9142 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:01.905968  9064 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.000s
I20260709 13:04:01.906422  9064 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.907685  1634 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 26: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:01.908406  1443 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f9d92cdb4b624c3bbc074afe1340b761 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2579bae7bf1543bf9b10d68e93a6ebbb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:01.836404  9280 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:01.867172  9267 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.376s	user 0.001s	sys 0.001s
W20260709 13:04:01.881112  9273 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
W20260709 13:04:01.930536  8933 consensus_meta.cc:287] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.777s	user 0.008s	sys 0.002s
I20260709 13:04:01.930883  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.930982  8933 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.839035  8726 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.965032  8726 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.839085  9001 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.965749  9246 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.966472  9246 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.966903  9246 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:01.968791  9264 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.800s	user 0.004s	sys 0.007s
I20260709 13:04:01.971468  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:01.951524  8951 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.972433  9247 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.974009  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1589) took 1229 ms. Trace:
I20260709 13:04:01.974455  2297 rpcz_store.cc:276] 0709 13:04:00.745020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.745203 (+   183us) service_pool.cc:224] Handling call
0709 13:04:00.745617 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.745963 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.745991 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.747859 (+  1868us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.080694 (+332835us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:01.080758 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.080861 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.968119 (+887258us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:01.973669 (+  5550us) raft_consensus.cc:1685] finished
0709 13:04:01.973698 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.973938 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":990,"raft.queue_time_us":4434,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6983,"spinlock_wait_cycles":257364736,"thread_start_us":332418,"threads_started":1,"wal-append.queue_time_us":419956}
I20260709 13:04:01.975750  8951 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.923527  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.979066  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:01.924446  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.924875  9286 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.894345  8769 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.980538  9286 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.980597  8769 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:01.982697  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:01.932471  9291 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:01.932655  9054 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:01.983212  8769 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.900463  9120 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 13:04:01.984037  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:01.891662  9096 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:01.983067  9286 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.984943  9096 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.985364  9096 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:01.985718  9096 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.986064  8682 raft_consensus.cc:3055] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:01.986455  8682 raft_consensus.cc:740] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:01.936152  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:01.987465  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:01.988016  1707 raft_consensus.cc:2376] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 444b494bb53842b59408a7230884398a in term 21. Re-sending same reply.
W20260709 13:04:01.936237  9185 consensus_meta.cc:287] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.000s
W20260709 13:04:01.936918  9276 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:04:01.887297  9100 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.941032  9266 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:01.941174  9220 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.000s
I20260709 13:04:01.942035  9117 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:01.918169  9275 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.365s	user 0.001s	sys 0.001s
I20260709 13:04:01.943396  9119 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.868062  9283 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:01.944331  9305 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:01.946888  9168 consensus_meta.cc:287] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.770s	user 0.008s	sys 0.001s
W20260709 13:04:01.950104  9043 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.786s	user 0.006s	sys 0.002s
I20260709 13:04:01.950795  1450 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 18, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:01.953470  9252 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.700s	user 0.010s	sys 0.013s
I20260709 13:04:01.953557  8710 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.867743  9292 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.958379  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:01.959285  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:01.895262  9288 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:04:01.960739  8612 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:01.960438  9159 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:01.917277  9144 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.003s
I20260709 13:04:01.962781  9064 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:01.896848  9274 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:04:01.981170  9167 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:01.933192  9253 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:01.990224  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:01.935222  9201 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.085s	user 0.008s	sys 0.001s
I20260709 13:04:01.992722  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.017234  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.020813  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:01.936403  8984 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.000s
W20260709 13:04:01.930527  9049 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.001s
I20260709 13:04:02.059473  1701 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:01.987298  8682 consensus_queue.cc:260] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.012048  9278 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.060344  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 206288us
I20260709 13:04:02.013883  9185 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:02.039332  9250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.016s	sys 0.039s
W20260709 13:04:02.081537  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 238280us
I20260709 13:04:02.089071  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.083458  1627 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
W20260709 13:04:02.128199  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.082391  9119 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.097477  9300 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.121999  1709 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:02.074618  9168 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.128306  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:02.078953  9043 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:02.140091  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52420us
I20260709 13:04:02.137073  1709 pending_rounds.cc:85] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 10
I20260709 13:04:02.145363  1709 pending_rounds.cc:107] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 18.11
W20260709 13:04:02.131273  9250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.017s	sys 0.040s
I20260709 13:04:02.110220  9144 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.145668  1709 raft_consensus.cc:2889] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.123148  9051 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.103031  8612 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.132885  9201 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.125833  9305 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:02.151935  2297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:04:02.155205  2297 raft_consensus.cc:606] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:04:02.155697  2297 raft_consensus.cc:614] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:02.157344  2291 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:02.143098  9167 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:02.148164  8682 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.119551  9236 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.134711  9278 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.130115  8984 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.153743  9144 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.141880  9300 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.155843  9051 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.165037  9294 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:04:02.166113  9308 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:02.156383  9305 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.150045  9185 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:04:02.153148  9055 consensus_meta.cc:287] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.992s	user 0.004s	sys 0.008s
I20260709 13:04:02.133122  9049 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.137975  9293 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 13:04:02.136686  9119 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:02.175683  9113 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.000s	sys 0.016s
I20260709 13:04:02.202178  9290 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:02.204394  9119 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.173350  9095 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:04:02.204962  9168 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.206316  9100 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:02.205564  9113 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.000s	sys 0.017s
I20260709 13:04:02.290539  1702 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:02.208953  9144 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.217198  9236 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.005924  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.330564  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.006893  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.041850  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.332902  1560 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
I20260709 13:04:01.990108  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:02.336045  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 345874us
I20260709 13:04:02.003010  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.270776  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3197) took 1658 ms. Trace:
I20260709 13:04:02.339042  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:02.339277  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 336260us
I20260709 13:04:02.337687  1628 rpcz_store.cc:276] 0709 13:04:00.612672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.613974 (+  1302us) service_pool.cc:224] Handling call
0709 13:04:00.614467 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:00.627619 (+ 13152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.627666 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:00.628370 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 13:04:00.966406 (+338036us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:00.966443 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.967177 (+   734us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.952884 (+985707us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:02.269644 (+316760us) raft_consensus.cc:1685] finished
0709 13:04:02.269737 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.270471 (+   734us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1556,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":46799,"spinlock_wait_cycles":52095872,"thread_start_us":337679,"threads_started":1,"wal-append.queue_time_us":495473}
I20260709 13:04:02.340307  2290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41b83284fac147339ac4124c2eedee11"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:04:02.341300  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.341864  9280 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.342338  1628 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:02.342998  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.342876  9280 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.008159  1557 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:02.112285  1635 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:02.349350  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3252) took 1319 ms. Trace:
I20260709 13:04:02.349902  9220 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:02.350086  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.351716  9168 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:02.211747  9096 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.352365  9168 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.352870  9168 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:02.353020  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.354905  9096 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.154260  1705 raft_consensus.cc:1240] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:02.271538  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.301734  9160 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.357139  1632 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:02.286820  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.357991  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3193) took 1375 ms. Trace:
W20260709 13:04:02.284097  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 138326us
I20260709 13:04:02.358474  1551 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:02.254233  1450 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 17, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.361820  8914 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:02.362720  8914 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:02.178603  9243 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.364575  8914 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.365559  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1685) took 1234 ms. Trace:
I20260709 13:04:02.366077  2291 rpcz_store.cc:276] 0709 13:04:01.131445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.131567 (+   122us) service_pool.cc:224] Handling call
0709 13:04:02.365032 (+1233465us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400610154. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:02.365326 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113045888}
I20260709 13:04:02.366678  9243 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.288322  9043 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.170032  9078 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.199307  9055 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.210096  9293 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:02.368377  9293 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:02.369383  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3192) took 1386 ms. Trace:
I20260709 13:04:02.370363  9055 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.369998  9293 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.197036  9294 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.207841  9246 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.216183  9278 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.221902  9305 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.331748  9286 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:02.219506  9300 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.007737  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.322834  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.207719  9283 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.210203  9290 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.359352  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.211232  9201 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.199467  9185 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.371737  9293 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:02.292570  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.294397  9213 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:02.302325  8984 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.292892  9049 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.295619  8612 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.292701  9051 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.325021  9297 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.325961  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.351990  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.357831  9160 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.007149  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.435983  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.378157  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.395779  9144 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:02.349924  1630 rpcz_store.cc:276] 0709 13:04:01.029926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.030088 (+   162us) service_pool.cc:224] Handling call
0709 13:04:02.348843 (+1318755us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400296dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:02.349078 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706311040}
I20260709 13:04:02.443914  8863 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.444818  8863 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.447857  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:02.448969  2297 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:02.449782  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.453785  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.455193  9144 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:02.459980  2293 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:02.461378  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.462183  1556 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.462466  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2785) took 1403 ms. Trace:
I20260709 13:04:02.462936  1705 rpcz_store.cc:276] 0709 13:04:01.058553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.273200 (+214647us) service_pool.cc:224] Handling call
0709 13:04:01.273444 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.356191 (+1082747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.462022 (+105831us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540010e5d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:02.462297 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025229568}
I20260709 13:04:02.467903  1557 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:02.469491  9160 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.472257  9095 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.473191  9095 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.474882  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.476786  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.397708  9280 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.398391  1702 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.477483  9095 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:02.480304  9255 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.345s	user 0.010s	sys 0.014s
I20260709 13:04:02.480656  9168 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.482337  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3188) took 2009 ms. Trace:
I20260709 13:04:02.482728  1633 rpcz_store.cc:276] 0709 13:04:00.472669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.472808 (+   139us) service_pool.cc:224] Handling call
0709 13:04:00.473203 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.709033 (+235830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.709080 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.709632 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.965120 (+255488us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:00.965180 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.148982 (+183802us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54002977d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:01.149034 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.479331 (+1330297us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:02.481907 (+  2576us) raft_consensus.cc:1685] finished
0709 13:04:02.481930 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.482164 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3103,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":65,"spinlock_wait_cycles":873866368,"thread_start_us":255116,"threads_started":1,"wal-append.queue_time_us":397962}
I20260709 13:04:02.460125  9297 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.487118  9297 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.488246  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:04:02.489872  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:02.380560  9300 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:02.358480  1703 rpcz_store.cc:276] 0709 13:04:00.982658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.256095 (+273437us) service_pool.cc:224] Handling call
0709 13:04:01.268142 (+ 12047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.694222 (+426080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.357414 (+663192us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003e8ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:02.357810 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978228480}
I20260709 13:04:02.491153  9095 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:02.402318  9078 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.501348  1560 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:04:02.403277  8872 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.009s
I20260709 13:04:02.384300  9185 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:02.381582  8933 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.376130  9281 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.369881  1700 rpcz_store.cc:276] 0709 13:04:00.982382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.255001 (+272619us) service_pool.cc:224] Handling call
0709 13:04:01.255309 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.475008 (+219699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.368841 (+893833us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540016ba54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:02.369188 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486784256}
I20260709 13:04:02.392171  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:04:02.408893  9270 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.232s	user 0.010s	sys 0.006s
I20260709 13:04:02.376080  9286 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:02.409660  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52738.5us
I20260709 13:04:02.374073  9294 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.410970  2292 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:02.380462  9290 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:02.412086  9272 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 1.235s	user 0.001s	sys 0.003s
I20260709 13:04:02.411837  9213 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.385980  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:02.389309  1628 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:02.413993  1701 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:04:02.362360  1450 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.378564  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.467430  1599 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.477491  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.377310  1553 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:02.380456  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.496873  1556 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:02.461014  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:02.502903  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 69303.4us
I20260709 13:04:02.504963  1551 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:02.512089  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.507436  9095 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:02.520980  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:02.526588  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 59450.7us
I20260709 13:04:02.530716  1708 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.531278  2290 raft_consensus.cc:606] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:02.532002  2290 raft_consensus.cc:614] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:02.532922  1559 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:02.528669  1554 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:04:02.509150  8872 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.520627  8863 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.541196  9168 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.542068  1702 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
I20260709 13:04:02.538383  1703 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:02.537653  2263 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.536640  2296 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:02.538094  1631 raft_consensus.cc:1275] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:04:02.546814  9290 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:02.547884  9297 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:02.548693  9297 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:02.550226  9297 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.552017  2260 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:02.553776  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:02.554490  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 162453us
I20260709 13:04:02.555476  9293 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:02.556041  9292 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.555482  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.546118  1631 pending_rounds.cc:85] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 9
I20260709 13:04:02.558606  9292 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.560957  8984 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:02.561589  8682 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:02.562075  8682 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.562470  8682 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.563498  9292 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.564466  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.564635  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.565750  9051 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:02.566012  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.566406  9051 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:02.566160  9095 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.566876  1555 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:04:02.558928  1631 pending_rounds.cc:107] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 22.10
I20260709 13:04:02.571837  1631 raft_consensus.cc:2889] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.573668  8872 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.564399  9300 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:02.576372  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:02.576808  9300 raft_consensus.cc:697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:02.577052  1558 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:02.577255  9286 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:02.564296  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 184059us
I20260709 13:04:02.574286  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.537719  9283 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:04:02.579159  9300 consensus_queue.cc:237] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.577724  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:02.536821  9281 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.585929  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.566053  9213 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:02.587945  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43622.1us
I20260709 13:04:02.578433  9051 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.589668  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.590468  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.591431  1703 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 15.
I20260709 13:04:02.591924  9287 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.586196  2291 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.592793  9287 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.595220  9287 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:02.579687  9322 log.cc:927] Time spent T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:04:02.600214  1557 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 26.
I20260709 13:04:02.603571  1708 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:02.608640  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.611483  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:02.613907  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 78259.3us
I20260709 13:04:02.617025  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.604444  8933 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.625165  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.586025  9281 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.609498  9290 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:02.634219  9057 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:02.628343  8933 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.632339  9290 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.628919  8914 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:02.644387  1708 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:02.633380  9287 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:02.647194  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.648809  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.649876  1557 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:02.642122  9057 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:02.646545  8914 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.644829  8933 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.664013  9318 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:02.664196  1702 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.665217  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.646148  9290 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:02.674265  9287 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:02.675696  9287 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.677537  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:02.679540  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.666898  9318 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.682878  9318 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.656646  8914 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:02.675509  9279 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.829s	user 0.004s	sys 0.000s
I20260709 13:04:02.696833  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.701561  2291 raft_consensus.cc:2468] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:02.698292  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.687713  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.705163  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.707801  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.708719  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:02.705799  1559 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:04:02.710837  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.709350  1552 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:02.711982  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.713614  8872 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:02.710022  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.707538  9213 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.687865  9078 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:02.717203  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.715412  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:02.722404  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.722457  1675 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.722739  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:02.715265  8872 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.725220  1628 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:02.705888  9051 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.726683  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:04:02.729555  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44110.4us
I20260709 13:04:02.694805  9113 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.733080  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.734784  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.728106  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3259) took 1519 ms. Trace:
I20260709 13:04:02.737047  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:02.737679  1557 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:02.740561  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:02.741559  1632 raft_consensus.cc:2393] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 26: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:02.742026  9059 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:02.740604  9299 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.742834  9299 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.743211  9299 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.743887  9055 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.744071  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.744894  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.745057  9283 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.744350  9055 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:02.690056  8863 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:04:02.748543  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.748438  9092 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:04:02.744918  1627 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:02.729032  8872 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:02.751204  8992 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:02.751631  1552 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:02.752175  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.752746  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.754055  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.756013  2296 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:02.749780  1554 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:02.751426  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.757699  1559 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:02.749508  9055 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.727911  9213 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.747005  9283 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.768077  2293 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:02.775758  9241 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.776664  9241 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.768095  9092 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:02.778635  9092 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:02.779136  9241 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.736653  1629 rpcz_store.cc:276] 0709 13:04:01.209015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.209573 (+   558us) service_pool.cc:224] Handling call
0709 13:04:01.209885 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.267973 (+ 58088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.268005 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.268495 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.492779 (+224284us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:01.492824 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.492884 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.674483 (+1181599us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:02.725437 (+ 50954us) raft_consensus.cc:1685] finished
0709 13:04:02.725469 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.727933 (+  2464us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59960,"raft.queue_time_us":77579,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":3808,"spinlock_wait_cycles":7216896,"thread_start_us":223878,"threads_started":1,"wal-append.queue_time_us":522941}
I20260709 13:04:02.760571  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:02.756620  9057 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.760352  8992 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.768030  9305 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.781330  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.787221  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.789984  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.790640  1705 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:02.788584  8992 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.796116  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.797538  9281 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.804268  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.804421  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:02.804993  2296 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:02.806300  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.788147  9113 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.809113  1599 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.809597  9113 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.809787  1553 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:02.810262  9113 raft_consensus.cc:2804] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:02.779729  9241 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.821368  1551 raft_consensus.cc:1240] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:02.819694  8710 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.818190  8726 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:02.824752  1635 raft_consensus.cc:1240] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:02.824782  8710 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:02.825433  8710 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.828634  2297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:04:02.829346  2297 raft_consensus.cc:606] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
W20260709 13:04:02.829564  9327 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 13:04:02.830615  9241 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:02.833014  9241 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:02.833630  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 199883us
I20260709 13:04:02.833801  8710 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.835039  1557 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 27.
I20260709 13:04:02.835717  1559 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:02.836310  8682 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.844836  1628 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:02.846804  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.845774  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:02.849403  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:02.853068  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:02.861328  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.861719  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.870957  2291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4"
dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
mode: GRACEFUL
new_leader_uuid: "00f3fbc60b884078a3c219909b44f64c"
 from {username='slave'} at 127.0.0.1:41766
I20260709 13:04:02.860846  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:02.838580  8682 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.876912  2291 raft_consensus.cc:606] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Received request to transfer leadership to TS 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:02.874702  1552 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:02.872977  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.877683  2291 raft_consensus.cc:614] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:02.876276  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.824905  8726 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.878787  1703 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:04:02.880885  1702 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:02.764032  9051 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.887172  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.888214  2290 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:02.861419  9113 raft_consensus.cc:697] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:02.890221  1702 raft_consensus.cc:1240] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 13:04:02.851624  9241 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.869346  9283 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.891989  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.876471  9305 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:02.894308  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.881726  8726 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:02.894894  1705 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:04:02.901443  1708 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:04:02.895740  9305 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:02.900174  1633 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:02.904268  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.906749  9305 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:02.907572  9332 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:04:02.908589  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.908784  9159 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.909655  9330 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:04:02.910043  9159 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.912619  9159 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:02.917829  9213 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:02.920037  9334 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:04:02.921290  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 102332us
I20260709 13:04:02.922684  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:02.922863  9092 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.876705  9290 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.914224  1598 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:02.852191  9281 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.928879  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:02.932919  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:02.934639  1703 raft_consensus.cc:2393] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:02.930589  2288 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:02.946873  9064 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:02.953217  9280 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.953931  9280 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.956341  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:02.961679  1707 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:02.964046  9292 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:02.964911  9292 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.965574  9292 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:02.951571  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 128203us
I20260709 13:04:02.966835  9290 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.967399  1525 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:02.964692  9185 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.924405  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:02.927062  8710 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:02.977876  2294 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:02.980221  2289 raft_consensus.cc:2368] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22.
I20260709 13:04:02.974520  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:02.924944  9326 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:04:02.984944  1633 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:02.932149  9318 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:02.985872  9092 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.986510  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2793) took 1444 ms. Trace:
I20260709 13:04:02.986927  1706 rpcz_store.cc:276] 0709 13:04:01.542368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.543874 (+  1506us) service_pool.cc:224] Handling call
0709 13:04:02.986091 (+1442217us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003dc554. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:02.986261 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2342661888}
I20260709 13:04:02.978482  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:02.993397  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.989931  1630 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:02.996299  2292 raft_consensus.cc:2368] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:02.996767  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:02.992619  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:02.927940  9241 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:02.930505  8872 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.006789  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:03.009390  9318 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:02.985018  9051 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:03.010483  1708 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:04:03.010480  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.012801  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:03.019347  9055 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.019812  9299 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:03.021803  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 131237us
I20260709 13:04:03.023172  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.005833  9064 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:03.028740  2293 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.030447  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.031848  9213 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:03.032874  9335 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:04:02.926676  9113 consensus_queue.cc:237] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.034857  9281 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.034986  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.035709  2295 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:03.036307  9286 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.005115  9078 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:03.036871  9286 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.037211  9078 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.037652  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.039386  9078 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.036718  1709 raft_consensus.cc:2368] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:04:03.041404  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.042256  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.042788  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.046554  1555 raft_consensus.cc:2368] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:04:03.039608  9283 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.048213  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.048954  2297 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:03.049568  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:03.049809  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.050006  1675 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:03.050740  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.051117  2297 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:03.045727  9055 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.052583  2295 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:03.053087  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.006067  9292 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.054082  2294 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:03.055405  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.058357  1630 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:03.052677  8710 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:03.059335  2293 raft_consensus.cc:2393] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:03.012611  8863 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:03.060326  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.016788  9059 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.031862  9210 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:03.053118  8682 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:03.066751  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.072263  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.053892  9280 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.073412  2295 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:03.074206  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.059221  9144 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:03.077706  1700 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:03.062217  9113 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.080724  1629 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:03.077744  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.065562  8863 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.089470  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.067746  9059 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.076668  9294 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.090898  9049 consensus_queue.cc:1059] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:03.045979  8992 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.090418  9078 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:03.093850  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.074550  9210 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.077194  8682 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.088570  1601 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.087291  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.023747  9305 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.102062  1633 raft_consensus.cc:2393] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:03.094485  8863 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.093704  9144 raft_consensus.cc:697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:03.112685  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.113492  8992 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.111292  8984 raft_consensus.cc:3055] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:03.114123  8984 raft_consensus.cc:740] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:03.115033  1635 raft_consensus.cc:2368] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:04:03.098757  9241 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:03.115453  8984 consensus_queue.cc:260] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.117211  8984 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:03.120327  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 64406.5us
I20260709 13:04:03.121335  2262 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.122321  9290 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.128923  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.129771  1705 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.133774  9055 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:04:03.134884  9055 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:03.135304  9055 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:03.097757  9059 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.106415  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.109570  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:03.142066  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53552.5us
I20260709 13:04:03.104663  9210 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:03.106840  9281 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.148494  1700 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:03.106738  8682 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.151167  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:03.146298  9049 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:03.146536  9078 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:03.168732  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.167012  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:03.169687  1708 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:03.152102  9113 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:03.104897  9294 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.138095  8933 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.169170  9185 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.141386  8914 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.213202  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.207428  8992 consensus_queue.cc:1059] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:03.173609  9113 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:03.230527  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135119us
I20260709 13:04:03.231314  2296 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:03.230280  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.232939  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.240126  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:03.237987  2292 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 15.
W20260709 13:04:03.246822  9341 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:04:03.247414  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.248140  1553 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:03.249004  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.238770  9144 consensus_queue.cc:237] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.249886  8992 raft_consensus.cc:3055] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:03.250304  8992 raft_consensus.cc:740] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:03.255806  1626 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26.
I20260709 13:04:03.208055  9049 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:04:03.251143  8992 consensus_queue.cc:260] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:03.263249  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 175304us
I20260709 13:04:03.233724  9113 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:03.267623  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:03.271349  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:03.259176  9159 raft_consensus.cc:2764] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:04:03.261538  8710 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.196823  9055 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.279855  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.280144  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:03.280431  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.283912  2297 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 24: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:03.262748  9064 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:03.292184  9337 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:04:03.286737  1673 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.303146  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.309300  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:03.313112  9340 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:04:03.313460  1551 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:03.316543  8933 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.318967  9213 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.320092  9062 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:03.321764  9297 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:03.329692  9253 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:03.329768  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.333647  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.335135  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:03.335497  9142 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.339802  9253 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.340377  9253 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.340806  9253 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.341720  9349 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.343889  2292 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.344918  8872 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.315971  9064 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.346773  9292 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.347182  9292 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.347225  9064 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.347671  9292 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.348430  9286 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.309933  1708 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:03.365922  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.368597  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:03.370282  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 82133.2us
I20260709 13:04:03.374418  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:03.377774  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:03.306890  9159 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.287381  8710 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:03.365000  9341 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:03.358855  9305 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:03.385915  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:03.364180  8992 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:03.388356  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53311.3us
I20260709 13:04:03.392599  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.319645  9213 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.394743  1705 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:03.392908  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:03.403903  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.381507  9286 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:03.406720  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 97472.3us
I20260709 13:04:03.387732  8872 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.300158  9287 consensus_queue.cc:1048] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:04:03.391060  8984 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.296582  9078 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.403064  9277 consensus_queue.cc:1059] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:04:03.425242  9305 raft_consensus.cc:2764] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:04:03.414252  8933 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.422375  9290 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:03.456588  8710 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:03.438056  9236 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:03.446597  9286 consensus_queue.cc:1059] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:03.436956  8726 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.453323  9213 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.469695  9092 consensus_queue.cc:1059] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:03.471956  8914 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.473544  9055 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:03.474161  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:03.488901  2294 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:03.480929  9305 raft_consensus.cc:2764] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
W20260709 13:04:03.487612  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 116115us
I20260709 13:04:03.463099  9062 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:04:03.466609  9210 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:03.482578  9345 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.276s	user 0.000s	sys 0.005s
I20260709 13:04:03.483122  9286 raft_consensus.cc:3055] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:03.488162  1708 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:03.481292  9299 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.496603  1632 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:04:03.496016  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:03.493046  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.493310  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.498032  1709 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:03.478252  9294 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:03.494315  1707 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:03.509891  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:03.518132  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 93724.5us
I20260709 13:04:03.490623  9236 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:03.527426  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.528987  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:03.530519  1706 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:03.524471  1635 raft_consensus.cc:1240] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:03.525170  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.504621  9213 raft_consensus.cc:3055] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:03.496796  9286 raft_consensus.cc:740] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:03.539404  1708 raft_consensus.cc:2410] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 24 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:04:03.540014  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.505813  9290 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:03.544312  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 119956us
I20260709 13:04:03.480346  9253 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.546885  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:03.489223  9292 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.548517  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:03.561074  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 135105us
I20260709 13:04:03.494829  8863 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.562000  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.563035  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.490424  9059 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.572439  1703 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:03.533933  9213 raft_consensus.cc:740] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:03.575245  1523 leader_election.cc:400] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:04:03.575608  1523 leader_election.cc:403] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:03.574983  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.578068  1674 leader_election.cc:400] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16.
I20260709 13:04:03.557818  9299 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:03.554451  9185 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:04:03.584627  1631 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:04:03.582118  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.586172  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:03.573005  9294 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.589684  1701 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:03.567369  9354 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.577428  9344 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.544570  9051 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:04:03.610025  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:03.616508  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55693.4us
I20260709 13:04:03.617502  1523 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:03.627382  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.631588  2289 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:03.601234  9294 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:03.647986  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.604681  8984 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.651363  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.607702  9354 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:03.651712  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.659703  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.660586  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:03.649227  9236 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.616772  9364 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:03.633607  9297 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.679603  1556 raft_consensus.cc:1275] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:03.647292  9064 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:03.690280  9344 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:03.690990  1559 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:03.691563  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.692791  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:04:03.695732  9355 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:04:03.696130  9356 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.651664  9294 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.696424  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:03.648929  9281 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.656996  9366 consensus_queue.cc:1059] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:04:03.705736  8863 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:03.671411  9213 consensus_queue.cc:260] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.708698  9292 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.657311  9286 consensus_queue.cc:260] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.709738  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.711100  8872 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.712745  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.715715  1627 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:04:03.721560  1560 raft_consensus.cc:2393] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:03.722895  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:03.651464  8726 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.659062  9354 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.681308  9280 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:03.774295  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.798846  1709 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:03.775013  2289 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:03.717553  9064 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.802309  1703 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
W20260709 13:04:03.783318  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 165927us
W20260709 13:04:03.785349  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 350627us
I20260709 13:04:03.723503  9290 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:03.788787  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.789323  1553 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:03.651443  9299 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
W20260709 13:04:03.799281  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 89646.6us
I20260709 13:04:03.824347  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.790132  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:04:03.800997  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 154301us
I20260709 13:04:03.826017  1554 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:03.827099  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 193971us
I20260709 13:04:03.827654  1552 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.844139  1632 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:03.845603  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.866216  1673 leader_election.cc:400] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:03.863746  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:03.867141  1674 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:03.806828  9064 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.829926  9213 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.869421  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.874634  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:03.806728  9362 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.784404  9318 consensus_queue.cc:1048] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:04:03.863416  2262 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26.
I20260709 13:04:03.875382  1633 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 26: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:03.884443  2262 leader_election.cc:403] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:04:03.869699  1675 leader_election.cc:400] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:04:03.878234  1558 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:03.765861  9356 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:03.875026  9299 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:03.875627  9253 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:03.881628  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.827266  9280 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:03.878098  9359 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:03.821410  9144 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
W20260709 13:04:03.893816  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 212078us
I20260709 13:04:03.864300  9349 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.781136  9344 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.907020  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.867409  9159 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:03.829432  9095 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:04:03.903689  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:03.908236  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 63820.7us
I20260709 13:04:03.910496  1633 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:03.906276  1558 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:03.911504  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.893381  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 86011.6us
I20260709 13:04:03.809549  9367 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
W20260709 13:04:03.910549  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 107440us
I20260709 13:04:03.913292  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.914711  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.915019  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.893029  9286 raft_consensus.cc:3060] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.844604  9361 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:04:03.919166  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.925873  8710 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:03.929235  8710 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.929771  8710 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.926234  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:03.916960  9363 log.cc:927] Time spent T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:04:03.940621  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:03.948604  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 106440us
I20260709 13:04:03.953395  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.954850  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:03.954701  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:03.956976  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:03.957348  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.959025  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:03.961607  2292 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:03.926080  9253 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.965078  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:03.965771  1632 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.925297  9299 raft_consensus.cc:697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:03.967120  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:03.969763  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.969907  2296 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:03.974740  1631 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:03.981168  2290 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:04:03.976634  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 191748us
I20260709 13:04:03.994707  1704 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 11. (term mismatch)
I20260709 13:04:03.996966  1633 raft_consensus.cc:1240] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.958494  9168 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:04.005685  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:03.977985  9365 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 13:04:04.008527  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.995241  1704 pending_rounds.cc:85] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 10
I20260709 13:04:03.922446  9349 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.008860  1673 leader_election.cc:400] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22.
I20260709 13:04:04.010202  9100 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:04.010602  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:04.013319  9389 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:04.070068  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9744e1e31604277953315481c56b171"
dest_uuid: "8b2c3934407d422495728f84374c6b53"
mode: GRACEFUL
new_leader_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:04:04.070926  1627 raft_consensus.cc:606] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Received request to transfer leadership to TS 444b494bb53842b59408a7230884398a
I20260709 13:04:03.976333  9353 raft_consensus.cc:3055] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:03.968077  9362 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.022354  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3216) took 1067 ms. Trace:
I20260709 13:04:04.089229  1702 rpcz_store.cc:276] 0709 13:04:02.955267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.004453 (+ 49186us) service_pool.cc:224] Handling call
0709 13:04:03.026321 (+ 21868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.581455 (+555134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.581513 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.619898 (+ 38385us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.833389 (+213491us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:03.871331 (+ 37942us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.874773 (+  3442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.874861 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.022067 (+147206us) raft_consensus.cc:1685] finished
0709 13:04:04.022096 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.022272 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7642,"prepare.queue_time_us":41319,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":12087,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4067,"spinlock_wait_cycles":20825984,"thread_start_us":651625,"threads_started":2,"wal-append.queue_time_us":283641}
W20260709 13:04:04.025134  9113 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.004s
I20260709 13:04:04.092223  9113 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:03.962951  9092 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:04:04.028180  9078 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:04.029583  9059 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.037966  1552 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:04.041616  1706 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:04.042224  1704 pending_rounds.cc:107] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 21.11
W20260709 13:04:04.009467  2294 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:04:04.108379  1704 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.108623  2294 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:04.083408  1632 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:04.015290  1554 raft_consensus.cc:2468] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:04:04.084193  8710 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.087286  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:03.992036  8863 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:04.100689  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:04.103247  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 146155us
W20260709 13:04:04.104775  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 106332us
I20260709 13:04:03.942071  9290 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.069454  9185 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:04.137282  8710 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.139817  1632 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:04:04.111799  9372 log.cc:927] Time spent T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:04:04.088958  9353 raft_consensus.cc:740] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:03.832635  9062 consensus_queue.cc:1175] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b2c3934407d422495728f84374c6b53" responder_term: 20 status { last_received { term: 20 index: 16 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:04.143064  1632 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:04.152293  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:04.152629  9055 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.003s
I20260709 13:04:04.153116  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:04.176261  2295 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.153949  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:04.177290  2288 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.113780  8933 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26.
I20260709 13:04:04.160734  9343 raft_consensus.cc:3055] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:04.161721  9375 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:04:04.161918  8682 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.995s	user 0.005s	sys 0.008s
I20260709 13:04:04.164342  9299 consensus_queue.cc:237] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.168526  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:04.169790  2289 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:04.172010  2290 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.178885  8710 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:04.156522  9095 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.977030  9280 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.188753  1552 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:04:04.189320  1627 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:04:04.202541  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 247756us
I20260709 13:04:04.203481  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.207484  2294 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26. Ops: [20.13-25.14]
I20260709 13:04:04.169149  9370 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:04.214689  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:04.214380  8710 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.192713  9343 raft_consensus.cc:740] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:04.141549  9294 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.233397  2288 raft_consensus.cc:2468] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:04.235437  8710 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.195983  9375 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:04.140789  9380 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 13:04:04.238679  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.244100  9371 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:04.219158  9359 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:04.244691  9375 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.219975  9095 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.212514  8682 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:04.248436  9377 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:04:04.254462  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.215957  9055 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:04.233924  9366 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:04.215359  9159 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:04:04.235402  9092 consensus_queue.cc:1059] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:04:04.222230  9344 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.259202  9142 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:04.276783  2289 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:04.284665  1628 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:04.314093  9373 consensus_queue.cc:1059] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 13:04:04.313639  9395 log.cc:927] Time spent T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:04:04.312260  8682 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.250468  9281 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 13:04:04.322682  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 84101.6us
I20260709 13:04:04.259300  9236 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:04.328073  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:04.312274   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:04.269647  9366 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:04.337136  2295 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:04:04.345340  1673 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:04.351867  9142 raft_consensus.cc:2804] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:04.352449  9142 raft_consensus.cc:697] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:04.352567  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:04.355849  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:04.342294  9354 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 13:04:04.354216  9236 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:04.366530  9392 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:04.366828   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:04.336099  9343 consensus_queue.cc:260] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.370630  9343 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:04.375841  9349 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.376260  9382 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.347287  9375 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.346112  9361 raft_consensus.cc:3055] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:04.355366  9366 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:04.408568  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 123732us
I20260709 13:04:04.409742  1601 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:04.422912  8682 raft_consensus.cc:2804] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:04.423663  8682 raft_consensus.cc:697] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:04.424607  8682 consensus_queue.cc:237] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.427105  2290 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:04.403928  9349 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:04.441404  2296 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:04.353488  9142 consensus_queue.cc:237] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:04.432283  9213 consensus_meta.cc:287] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 13:04:04.406234  9361 raft_consensus.cc:740] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:04.448999  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.286473  9353 consensus_queue.cc:260] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.411346  9399 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:04.470157  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:04.377066  9354 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.470085  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.476727  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:04.406108  9381 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.238s
I20260709 13:04:04.453044  9394 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:04.482676  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:04.405887  9382 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.378023  9359 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.454908  9349 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.481213  9399 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:04.486641  9294 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.504760  1554 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:04.488507  9210 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.517822  9349 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:04.522616  9064 consensus_meta.cc:287] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 13:04:04.502939  9159 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:04.506390  9380 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:04.528950  9382 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:04.530330  9286 consensus_meta.cc:287] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.002s
I20260709 13:04:04.580729  9380 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.581388  1558 raft_consensus.cc:1275] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:04.539860  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:04.582118  9064 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.583236  1554 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:04:04.583364  9380 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.533450  9294 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.553802  9353 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.584146  9294 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.588243  9294 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.543365  1556 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:04.555091  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.497898  9393 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:04.556833  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2823) took 1011 ms. Trace:
I20260709 13:04:04.563735  1675 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.592052  1553 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:04.592231  9393 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.592082  1703 rpcz_store.cc:276] 0709 13:04:03.545099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.614181 (+ 69082us) service_pool.cc:224] Handling call
0709 13:04:03.800648 (+186467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.821099 (+ 20451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.556590 (+735491us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400096d50. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:04.556763 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428055296}
I20260709 13:04:04.564180  2263 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:04.578621  2290 raft_consensus.cc:1240] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:04.580579  9159 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.508926  9361 consensus_queue.cc:260] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:04.522652  9297 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.797s	user 0.008s	sys 0.004s
I20260709 13:04:04.613528  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:04.630187  2290 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:04.631433  2297 raft_consensus.cc:2393] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 26: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:04.632558  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68452.4us
I20260709 13:04:04.634582  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.609313  9294 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:04.637049  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.594614  9349 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:04.617751  9297 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21.
I20260709 13:04:04.636915  9405 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:04.650815  9369 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:04.498623  9394 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.629598  9305 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.642787  9381 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:04.639355  9382 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:04.642979  9361 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.672899  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.677675  1554 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 26.
I20260709 13:04:04.691588  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.693288  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.678114  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:04.689292  9399 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.695554  1559 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:04.696130  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46273.4us
I20260709 13:04:04.681006  9159 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:04.704583  1559 raft_consensus.cc:2468] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:04.706187  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:04.676113  9142 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:04:04.661448  9349 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.708714  9354 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.712172  9393 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:04.712478  9411 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:04.728222  9366 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.729439  9095 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.730628  9366 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.732795  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3228) took 1102 ms. Trace:
I20260709 13:04:04.733244  1628 rpcz_store.cc:276] 0709 13:04:03.630424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.763288 (+132864us) service_pool.cc:224] Handling call
0709 13:04:03.788275 (+ 24987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.329170 (+540895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.732515 (+403345us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54003435d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:04.732725 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134441344}
I20260709 13:04:04.737030  9368 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:04.737617  9368 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:04.737910  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:04.738556  9368 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.670698  9359 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.663331  9405 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.657475  9318 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 13:04:04.748559  9368 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:04.675266  9305 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.721491  9394 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.723202  9349 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.707324  9299 consensus_queue.cc:1059] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:04:04.675968  9343 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:04:04.669023  9369 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.715080  9382 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:04.726155  9362 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.001s
I20260709 13:04:04.726431  9285 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:04:04.858996  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 111729us
I20260709 13:04:04.859270  9405 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:04.860963  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:04.864200  1555 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:04.865136  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.865840  9359 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:04.867367  9423 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.008s
I20260709 13:04:04.868742  9381 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.869320  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.720005  9297 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.952276  9297 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.930569  9285 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.954288  9285 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:04.857844  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:04.955750  9078 raft_consensus.cc:3055] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:04.956284  9078 raft_consensus.cc:740] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:04.956436  1628 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:04.941421  9349 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:04.871229  8710 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:04.957602  9349 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:04.957943  8710 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.958657  9349 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.880739  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:04.960161  9113 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.918767  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.962313  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:04.964080  9389 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:04.964331  9381 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:04.965799  9389 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:04.966218  9389 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:04.966588  9389 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.919860  9305 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:04.967758  9049 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:04.968082  9382 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:04.874670  9356 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.909525  8992 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.969771  8992 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.920778  9299 raft_consensus.cc:3055] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:04.921502  9406 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:04.911827  8682 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.920796  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.987844  2290 raft_consensus.cc:2368] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16.
I20260709 13:04:04.934738  1559 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:04.935454  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.869774  9354 consensus_queue.cc:1059] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 13:04:04.949523  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.990872  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.991226  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.951367  9277 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.953042  1560 raft_consensus.cc:1240] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:04.957228  9078 consensus_queue.cc:260] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.911985  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:04.909559  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.968276  9049 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:04.917416  2297 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:04:04.907810  8863 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.998980  9078 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:04.960202  8710 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:04.978134  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:05.003820  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.963662  1601 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:04.980654  9299 raft_consensus.cc:740] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:04.985245  8992 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:04.933669  9362 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:05.011339  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 75975.8us
I20260709 13:04:04.955458  9297 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:05.014549  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 50986.9us
I20260709 13:04:05.016317  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:05.021239  1628 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.002545  9049 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:05.023070  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 162228us
I20260709 13:04:05.023718  9049 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.024252  1599 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:05.025758  9393 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:05.028231  1703 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:05.028510  1601 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:05.034797  1599 leader_election.cc:400] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16.
I20260709 13:04:05.035255  1599 leader_election.cc:403] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:05.036067  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:04.999087  8863 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:04.986362  9305 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:05.045148  2290 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:04:05.049767  1559 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:04.985997  9406 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.031700  9390 raft_consensus.cc:2804] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:05.062341  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 356222us
I20260709 13:04:05.030833  9354 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:05.069837  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:05.071735  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:05.072994  1557 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:04:05.033958  9185 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:05.070052  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:04.995666  8682 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.088382  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:05.007542  9277 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.046150  9343 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:05.084697  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:05.083894  1560 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 24: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:05.097644  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:05.103588  9100 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.104278  9369 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:05.108265  8992 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:05.092264  9185 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:05.118429  1706 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:05.119613  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.120344  1709 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:05.059489  9362 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:04:05.086560  9287 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:05.125540  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.128190  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:05.130064  1631 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:05.109674  9406 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.101641  9283 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:05.140468  1706 raft_consensus.cc:1240] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.085638  9305 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:05.162665  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 134257us
I20260709 13:04:05.076764  9299 consensus_queue.cc:260] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:05.167831  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.172554  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:05.173599  1705 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.183290  9390 raft_consensus.cc:697] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:05.117323  9185 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.186106  2262 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:05.184970  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:05.166316  9359 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:05.157459  9393 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:05.198889  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.200218  1709 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:05.202611  9064 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.160094  9287 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.208654  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.212404  1631 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:04:05.195912  9421 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:04:05.180698  9406 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:04:05.217643  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.199441  9281 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:05.110554  9383 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.116s
I20260709 13:04:05.228269  9393 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.235868  9287 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:05.235992  8863 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.236562  1709 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:05.248240  1633 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:05.258710  1628 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:04:05.210023  9210 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:05.237690  9281 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.260030  9210 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:05.266633  9430 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:04:05.272074  9277 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:05.273494  9429 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:04:05.289350  9210 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.290221  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.290953  1709 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:05.264183  9357 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.292024  9357 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.293708  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2812) took 1256 ms. Trace:
I20260709 13:04:05.294263  9357 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.294896  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.296370  9389 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.294116  1552 rpcz_store.cc:276] 0709 13:04:04.037648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.107457 (+ 69809us) service_pool.cc:224] Handling call
0709 13:04:04.107944 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.210728 (+102784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.293446 (+1082718us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b540036c9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:05.293610 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146274176}
I20260709 13:04:05.298482  9389 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.265374  1705 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:05.282562  9281 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:05.303303  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43790.7us
W20260709 13:04:05.318706  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51928 (request call id 3183) took 1662 ms (client timeout 1897 ms). Trace:
W20260709 13:04:05.319257  1555 rpcz_store.cc:269] 0709 13:04:03.656552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.656906 (+   354us) service_pool.cc:224] Handling call
0709 13:04:05.318452 (+1661546us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001379d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:05.318594 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364725504}
I20260709 13:04:05.310268  9390 consensus_queue.cc:237] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.336552  9305 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:05.338390  9387 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 1.067s	user 0.000s	sys 0.009s
I20260709 13:04:05.338591  9292 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:05.339250  9292 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:05.339818  1551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3184) took 1683 ms. Trace:
I20260709 13:04:05.340451  9292 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.341048  9285 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.342072  9292 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:05.342787  2289 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:05.343947  9285 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:05.344631  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.346903  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1415) took 1002 ms. Trace:
I20260709 13:04:05.351583  1704 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.352586  8984 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:05.353174  8984 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.359552  9353 raft_consensus.cc:3055] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:05.360095  9353 raft_consensus.cc:740] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:05.360961  9353 consensus_queue.cc:260] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.362071  9353 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:05.361753  9064 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:05.364907  9442 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:05.369588  9287 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.370229  9287 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.371362  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2810) took 1334 ms. Trace:
I20260709 13:04:05.371847  1558 rpcz_store.cc:276] 0709 13:04:04.037091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.037716 (+   625us) service_pool.cc:224] Handling call
0709 13:04:04.038099 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.581779 (+543680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.371087 (+789308us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540036f6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:05.371297 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248653056}
I20260709 13:04:05.340204  1551 rpcz_store.cc:276] 0709 13:04:03.670013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.680550 (+ 10537us) service_pool.cc:224] Handling call
0709 13:04:03.680870 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.777393 (+ 96523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.777440 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.822273 (+ 44833us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.962666 (+140393us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:03.962804 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.962953 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.298996 (+1336043us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:05.339467 (+ 40471us) raft_consensus.cc:1685] finished
0709 13:04:05.339495 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.339725 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103311,"raft.queue_time_us":80929,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18910,"spinlock_wait_cycles":14686464,"thread_start_us":85739,"threads_started":1,"wal-append.queue_time_us":328968}
I20260709 13:04:05.380345  1704 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:05.384317  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.386160  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:05.386350  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:05.388659  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.347437  2288 rpcz_store.cc:276] 0709 13:04:04.344175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.349244 (+  5069us) service_pool.cc:224] Handling call
0709 13:04:04.349600 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.368020 (+ 18420us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:04:04.455573 (+ 87553us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.643343 (+187770us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.643543 (+   200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.643600 (+    57us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:04:04.643699 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.961532 (+317833us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800085480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:05.325805 (+364273us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b1800085480. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:05.346540 (+ 20735us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54007eb8d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:05.346611 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.346819 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24066,"prepare.queue_time_us":133649,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":111,"raft.queue_time_us":30277,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":11946,"spinlock_wait_cycles":112533120,"thread_start_us":250094,"threads_started":2,"wal-append.queue_time_us":268244}
I20260709 13:04:05.399528  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.404213  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:05.407166  1709 raft_consensus.cc:1240] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:05.408030  9415 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.404410  1673 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:05.409096  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:05.415436  1633 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:04:05.408355  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:05.416167  1558 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:05.414893  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:05.417289  1553 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:05.417219  1673 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:05.410426  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.424422  1631 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:05.426759  1706 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:05.430662  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.432194  1626 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:05.399057  9059 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:05.441133  8710 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:05.439287  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:05.444432  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:05.442559  1673 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:05.448436  1704 raft_consensus.cc:1275] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:05.450707  8710 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:05.451195  8710 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:05.451524  8710 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:05.452378  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.455211  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:05.447942  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.409873  9305 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:05.471340  9297 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:05.474897   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:05.475124  9055 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:05.504467  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.504815  1553 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
I20260709 13:04:05.505414  1631 raft_consensus.cc:2410] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:04:05.506392  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.507557  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.507480  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.508812  1674 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:05.509080  1635 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:05.509202  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.509888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.476553  9100 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 15. Current term is 16.
W20260709 13:04:05.476938  9425 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.432s	user 0.002s	sys 0.000s
I20260709 13:04:05.396683  9277 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:04:05.472308  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.429153  9358 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:05.472867  9426 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:05.484800  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3286) took 1856 ms. Trace:
I20260709 13:04:05.516366  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.523342  1631 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:05.524053  1628 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:05.501497  9297 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.431980  9415 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.681360  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.682204  1706 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
W20260709 13:04:05.692209  9445 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:04:05.499825  9059 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:04:05.703898  9444 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.704538  9444 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.706799  9444 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.522496  1634 rpcz_store.cc:276] 0709 13:04:03.628438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.628555 (+   117us) service_pool.cc:224] Handling call
0709 13:04:03.656433 (+ 27878us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:03.920087 (+263654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.920118 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:03.936473 (+ 16355us) log.cc:844] Serialized 52 byte log entry
0709 13:04:04.104432 (+167959us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:04.188421 (+ 83989us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.209690 (+ 21269us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.209897 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.969041 (+759144us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18002b77a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:05.332255 (+363214us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b18002b77a0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:05.484374 (+152119us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54000dded4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:05.484452 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.484520 (+    68us) raft_consensus.cc:1685] finished
0709 13:04:05.484538 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.484721 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66472,"prepare.queue_time_us":111580,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":3902,"raft.queue_time_us":148580,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":24841,"spinlock_wait_cycles":625752704,"thread_start_us":127648,"threads_started":1,"wal-append.queue_time_us":216499,"wal-append.run_cpu_time_us":5111,"wal-append.run_wall_time_us":1183712}
I20260709 13:04:05.733897  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.564381  9297 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:05.517887  9359 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.196s
I20260709 13:04:05.514590  9305 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:05.713541  9415 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:05.739850  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 267810us
I20260709 13:04:05.522303  9281 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.780612  1627 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:05.781445  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 329891us
W20260709 13:04:05.785303  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 342806us
I20260709 13:04:05.789227  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:05.803627  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1409) took 1630 ms. Trace:
I20260709 13:04:05.816746  9287 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.806579  2294 rpcz_store.cc:276] 0709 13:04:04.151460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.179082 (+ 27622us) service_pool.cc:224] Handling call
0709 13:04:04.207102 (+ 28020us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:04.211422 (+  4320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.781431 (+1570009us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b540040a950. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:05.781613 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63338240}
I20260709 13:04:05.828300  8710 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.827440  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:05.832403  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:05.835218  1634 raft_consensus.cc:1275] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 13:04:05.835655  1634 pending_rounds.cc:85] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 14
I20260709 13:04:05.826701  8933 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:04:05.845341  9299 raft_consensus.cc:3060] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.856140  9390 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:05.856873  9390 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.857858  9305 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:05.859190  9390 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.857182  9280 raft_consensus.cc:3055] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:05.860030  9280 raft_consensus.cc:740] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:05.863472  9390 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:05.864415  1627 raft_consensus.cc:1275] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:05.864197  9390 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.866389  9390 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.869526  9236 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.870926  9281 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.874922  9381 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.803988  9357 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:05.877210  8682 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:05.876942  9357 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.882854  1706 raft_consensus.cc:2393] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:05.884507  1634 pending_rounds.cc:107] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 13:04:05.885033  1634 raft_consensus.cc:2889] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:05.897722  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.905647  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.832588  9287 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:05.914448  9305 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:05.915135  1599 leader_election.cc:400] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:05.824779  9290 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 13:04:05.915524  1599 leader_election.cc:403] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:05.916419  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:05.927752  8710 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:05.932122  1630 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:04:05.909461  9358 raft_consensus.cc:697] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:05.915046  9305 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:05.943069  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.944393  9362 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:05.912089  8682 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.912787  9452 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.929693  9449 log.cc:927] Time spent T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.057s	user 0.000s	sys 0.004s
W20260709 13:04:05.941923  9447 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.212s	user 0.002s	sys 0.002s
I20260709 13:04:05.944023  9287 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.959380  1628 raft_consensus.cc:1240] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:05.958613  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.945004  9437 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:04:05.945984  8933 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.961205  1634 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:05.964198  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:05.971875  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:05.998870  1627 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:05.970424  8933 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:05.965476  9287 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.003129  8933 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.003594  9287 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.004305  9287 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.005244  9401 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:04:06.005496  9305 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:05.962402  9358 consensus_queue.cc:237] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:06.010701  9450 log.cc:927] Time spent T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:06.010902  8984 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.011837  9374 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:06.012318  9374 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:05.957561  8682 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.013262  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3227) took 2382 ms. Trace:
I20260709 13:04:06.013185  9374 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.014498  9374 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.014946  8682 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.013727  1629 rpcz_store.cc:276] 0709 13:04:03.630277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.763425 (+133148us) service_pool.cc:224] Handling call
0709 13:04:03.774410 (+ 10985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.809590 (+ 35180us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:03.887013 (+ 77423us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.386497 (+1499484us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b18003ff700. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70ce9dc060 00007f70ce9ee716 00007f70ce9ff287
0709 13:04:05.386598 (+   101us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.387093 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.801420 (+414327us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:05.801484 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.801582 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.012920 (+211338us) raft_consensus.cc:1685] finished
0709 13:04:06.013036 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.013176 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32059,"prepare.queue_time_us":133983,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":68086,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"spinlock_wait_cycles":17612416,"thread_start_us":657233,"threads_started":4,"wal-append.queue_time_us":666495,"wal-append.run_cpu_time_us":1395,"wal-append.run_wall_time_us":1037743}
I20260709 13:04:06.030913  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:05.960237  9452 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.091647  9359 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.092552  9359 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.093956  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 3241) took 1117 ms. Trace:
I20260709 13:04:06.094437  1635 rpcz_store.cc:276] 0709 13:04:04.976692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.977743 (+  1051us) service_pool.cc:224] Handling call
0709 13:04:06.093729 (+1115986us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54000de3d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:06.093880 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091730816}
I20260709 13:04:06.069196  9381 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:06.004945  9287 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:06.082373  9292 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.003s
I20260709 13:04:06.068907  8933 consensus_queue.cc:1048] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:04:06.118247  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.121737  1702 raft_consensus.cc:1275] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:06.122207  1702 pending_rounds.cc:85] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 10
I20260709 13:04:06.122485  1702 pending_rounds.cc:107] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:06.122752  1702 raft_consensus.cc:2889] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.125300  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:06.125598  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.129961  1601 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:06.131058  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:06.133101  1635 raft_consensus.cc:1240] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:06.135061  1635 raft_consensus.cc:1240] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:06.136103  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.137140  1707 raft_consensus.cc:2410] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:06.136001  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.137931  1705 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:06.137077  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.139027  2289 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:06.138983  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.140502  2294 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:06.140774  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:06.141533  1708 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:06.142091  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:06.096892  9281 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:04:06.144840  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.156239  1702 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:06.135175  9055 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:06.146679  9362 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:06.135277  9290 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.147905  9359 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.148344  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.149732  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.150362  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.158722  1701 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:04:06.159667  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:06.129712  9415 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.162743  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.163661  1708 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
W20260709 13:04:06.165733  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.168119  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.171303  1707 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:06.157840  9362 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:06.194439  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.196626  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:06.197602  2288 raft_consensus.cc:2368] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:06.199681  2288 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:04:06.202139  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.203161  2288 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:06.207871  2291 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:06.208216  9390 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.208930  9390 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.209295  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.210597  1673 leader_election.cc:400] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:06.210970  1673 leader_election.cc:403] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:06.211328  9390 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.212056  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.166870  9394 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:06.215427  1701 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 13:04:06.216140  1701 pending_rounds.cc:85] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 13
I20260709 13:04:06.216426  1701 pending_rounds.cc:107] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 13:04:06.216727  1701 raft_consensus.cc:2889] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.227635  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.229524  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:06.236161  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 45510.9us
I20260709 13:04:06.240754  9359 consensus_queue.cc:1059] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:06.241969  9359 consensus_queue.cc:1243] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer 8b2c3934407d422495728f84374c6b53 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:06.242700  9359 raft_consensus.cc:3055] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:06.242578  9358 consensus_queue.cc:1048] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:06.240852  9457 consensus_queue.cc:1048] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:06.244678  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:06.244992  9457 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:06.246013  9457 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:06.246366  1673 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.247977  9457 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:06.248348  9457 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.248207  9292 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:06.250938  9292 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:06.251668  9285 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:06.251994  9460 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.252908  9460 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.256109  1633 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:04:06.256457  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.252239  9285 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:06.243356  9359 raft_consensus.cc:740] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:04:06.260236  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 471127us
I20260709 13:04:06.212419  9362 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:06.248986  9457 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:06.163863  9055 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.166590  9290 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.212954  9443 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.270895  1673 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.183923  9415 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.228786  9459 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:06.226893  9281 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.270507  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:04:06.284930  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.265990  9361 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.287573  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.294363  2295 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:06.288079  8914 raft_consensus.cc:993] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: : Instructing follower 444b494bb53842b59408a7230884398a to start an election
I20260709 13:04:06.293751  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:06.276341  9394 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.302817  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:06.303727  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:06.283051  8863 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:06.283972  9354 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.281659  9453 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:06.290387  9292 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:06.330632  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:06.289086  9277 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:06.319506  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 189624us
I20260709 13:04:06.305263  8863 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.371429  8863 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:06.328910  9394 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.378436  9394 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.321785  8914 raft_consensus.cc:1081] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Signalling peer 444b494bb53842b59408a7230884398a to start an election
I20260709 13:04:06.385155  9415 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.385946  9415 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:06.388439  9462 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:06.299891  9361 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.353211  9185 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:06.391310  9185 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.391755  9185 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:06.324633  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 85987.8us
I20260709 13:04:06.370455  8984 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:06.370765  9466 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:06.321246  9444 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:06.301736  9460 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:06.401181  8984 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.296325  9281 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.410446  9287 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:06.383625  9297 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.003s
I20260709 13:04:06.411785  9055 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.366271  9381 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:06.428035  9381 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:06.427649  9297 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.396082  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:06.358670  9353 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.418787  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.357374  9470 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:06.367017  9343 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:06.333073  9452 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.354306  9359 consensus_queue.cc:260] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.442807  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.289479  9443 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.424206  9286 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.448333  1700 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:06.429011  9381 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.425843  8984 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.400292  9277 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.452987  1702 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:04:06.438741  9059 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.450934  9062 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:06.463905  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:06.464529  9062 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:06.465184  9210 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.465723  9210 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.329197  9285 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.353680  9453 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.441195  9287 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:06.469365  9287 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:06.469869  9287 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:06.470161  9475 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:06.470737  9475 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:06.454808  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:06.471599  9475 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.476377  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.477322  1555 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:06.480383  9453 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.481158  9453 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:06.481534  9437 consensus_meta.cc:287] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 13:04:06.482136  9437 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:06.482324  9290 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:06.443589  9470 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:06.483754  9470 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:06.485049  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.486066  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.486199  8863 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.488130  9359 raft_consensus.cc:3060] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.489233  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.491431  1554 raft_consensus.cc:1240] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:06.492085  9297 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.492810  9297 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:06.342270  9354 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.495606  8710 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.496192  8710 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:06.498183  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 140575us
W20260709 13:04:06.501817  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 196368us
I20260709 13:04:06.504765  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.444943  9353 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.507637  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:06.509146  8710 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.494694  9059 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:06.512152  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.501447  9285 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:06.514930  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.537321  1553 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 13: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:06.516347  8863 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.517225  9059 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.519169  9287 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.520617  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.521716  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.521911  9353 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.522094  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.543496  1556 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:06.461484  9286 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.491576  9185 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.528530  9366 consensus_queue.cc:1059] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:06.530386  9362 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:06.532740  9448 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.754s	user 0.009s	sys 0.000s
I20260709 13:04:06.533882  8710 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.535080  9452 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.524785  9469 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:06.538509  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.541846  1559 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:06.512007  9437 consensus_queue.cc:1048] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:04:06.549590  8710 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.546030  1598 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:06.549746  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.554171  1551 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:06.567724  1555 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:06.568775  1598 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:06.576638  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 243013us
I20260709 13:04:06.577658  1705 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:06.578724  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:06.583032  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.547514  9467 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:04:06.588246  9468 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:04:06.589071  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.550338  9469 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.591243  9469 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.592898  9354 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:06.593995  9464 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:04:06.594131  9357 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:06.595597  9473 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:04:06.601080  9358 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:06.601774  9358 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.604417  9358 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.613528  9362 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:06.615936  8984 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.618490  8984 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.620239  9286 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.624300  9281 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:06.550561  9366 consensus_queue.cc:1243] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [LEADER]: Peer 00f3fbc60b884078a3c219909b44f64c log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:04:06.629136  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50541.3us
W20260709 13:04:06.629976  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 52609.9us
I20260709 13:04:06.633170  8710 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:06.629202  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 46258.7us
I20260709 13:04:06.636624  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.638700  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.644325  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:06.647156  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 183359us
I20260709 13:04:06.556813  9343 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:06.650359  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.579639  9389 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:06.660632  8710 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.661098  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.661854  1559 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:06.663129  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.663985  2293 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:06.665021  2289 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:04:06.557113  9452 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.640372  9362 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:06.632627  9390 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:06.671864  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.670756  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.634317  9481 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.687708  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:06.672917  2290 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:06.693143  1553 raft_consensus.cc:1240] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:06.694996  1675 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:06.696506  8710 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:06.696775  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63486.6us
I20260709 13:04:06.698271  1559 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:06.697618  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:06.700067  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:06.701324  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:06.701686  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:06.702495  2293 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:06.705953  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.708252  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.698731  9476 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.663380  9354 consensus_queue.cc:1059] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 13:04:06.710261  9460 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.693537  9362 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:06.713915  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:06.716493  9049 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:06.711954  9389 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.717272  1524 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:06.717821  9389 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.718565  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:06.719321  1559 raft_consensus.cc:2368] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
I20260709 13:04:06.706775  9358 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.721212  9358 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.721627  9358 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.722832  8863 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.723362  8863 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.723800  8863 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.715281  9460 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.725601  9460 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.723372  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.699487  9287 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.744474  9479 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:06.809454  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:38031 timed out after 1.621s (SENT) [suppressed 8 similar messages]
I20260709 13:04:06.785701  1675 leader_election.cc:400] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
I20260709 13:04:06.736630  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:06.811249  1675 leader_election.cc:403] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:04:06.703370  9486 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.811851  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75325.4us
I20260709 13:04:06.815337  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:06.811823  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74591.1us
I20260709 13:04:06.729112  9362 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:06.825711  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.826906  2296 raft_consensus.cc:1240] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:04:06.827425  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.730121  9470 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:06.817592  9457 consensus_meta.cc:287] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.500s	user 0.007s	sys 0.004s
I20260709 13:04:06.805528  9461 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.833165  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.836900  1560 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:06.710816  9476 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.829391  8984 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:06.822590  9479 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:06.822377  9415 consensus_queue.cc:1048] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:04:06.835093  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.822141  9287 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:06.829324  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:06.842677  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:06.733847  9481 consensus_queue.cc:1059] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
I20260709 13:04:06.833477  9389 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.859683  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.859405  9241 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:06.865167  9452 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:06.865005  9389 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:06.859428  9305 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.866652  9305 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:06.867084  9305 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.867606  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:06.826085  9486 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.869748  9486 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.865013  9241 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:06.873318  9241 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:06.874308  9241 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:06.874449  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.875133  1705 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:06.876252  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.876521  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:06.876955  1706 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:06.871577  9406 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:06.878356  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:06.878890  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.879011  1555 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:06.845638  9479 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.810876  9343 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:06.887085  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.893087  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:06.897106  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 87706.1us
I20260709 13:04:06.840979  9457 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.863276  9185 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:06.893743  1706 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:04:06.887197  9374 consensus_meta.cc:287] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.872s	user 0.005s	sys 0.010s
I20260709 13:04:06.909761  1553 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:06.878010  9406 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.884305  9368 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.879799  9487 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:06.887847  9479 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:06.913457  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:06.936726  1556 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:06.937000  1635 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:06.919018  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:06.886766  9389 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.921685  1632 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:04:06.939705  1702 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:04:06.940011  1632 pending_rounds.cc:85] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 15
I20260709 13:04:06.940411  1632 pending_rounds.cc:107] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:04:06.940703  1632 raft_consensus.cc:2889] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.934391  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:06.935199  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:06.891826  8984 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:06.942682  1559 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:06.916854  1560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9744e1e31604277953315481c56b171"
dest_uuid: "444b494bb53842b59408a7230884398a"
 from {username='slave'} at 127.0.0.1:51916
I20260709 13:04:06.904302  9452 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:06.843590  9461 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:06.933094  9406 raft_consensus.cc:3060] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:06.931138  9368 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.067114  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.123291  1635 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:07.123920  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.126811  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.131721  9280 consensus_queue.cc:260] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:07.132860  9485 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:04:07.133606  9280 raft_consensus.cc:3060] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:07.134588  9494 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:04:07.132076  9496 log.cc:927] Time spent T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:04:07.173336  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:07.036798  9357 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.023134  9500 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.192651  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:07.088955  9368 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.043447  9389 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.242278  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.243109  1556 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:07.023016  9487 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.220243  9500 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.261199  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.198064  9415 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:07.067051  8984 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.267747  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
W20260709 13:04:07.269493  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 303140us
I20260709 13:04:07.093429  9452 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:07.222962  9469 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:07.276547  1635 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:07.195488  9292 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:04:07.292872  9457 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.302006  9477 raft_consensus.cc:3055] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:07.302608  9477 raft_consensus.cc:740] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:07.200130  9357 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.270990  9415 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:07.313090  1558 raft_consensus.cc:1275] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:04:07.313445  1558 pending_rounds.cc:85] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Aborting all ops after (but not including) 15
I20260709 13:04:07.313647  1558 pending_rounds.cc:107] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:04:07.313853  1558 raft_consensus.cc:2889] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:07.314013  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.277073  9470 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.285660  9469 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.372239  9469 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.376731  9461 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.381502  9479 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.337569  8984 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.338835  9500 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.339632  9357 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:07.341439  8863 consensus_meta.cc:287] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
I20260709 13:04:07.303663  9477 consensus_queue.cc:260] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.312639  9415 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:07.311913  9460 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.003s
I20260709 13:04:07.356700  9354 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.318009  9476 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.357481  9487 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.358992  9389 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.360978  9361 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.362677  9406 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.429528  9406 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.364693  9452 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:07.357447  9368 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.367978  9297 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.432320  9368 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.432785  9297 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.435657  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.436304  9277 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.436564  1628 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:07.370195  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.437350  1627 raft_consensus.cc:1240] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:07.438166  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.397863  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.438591  2294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1419) took 1246 ms. Trace:
I20260709 13:04:07.438951  2294 rpcz_store.cc:276] 0709 13:04:06.192497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.192672 (+   175us) service_pool.cc:224] Handling call
0709 13:04:06.193055 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.294178 (+101123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.294211 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.294256 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:06.294288 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.438373 (+1144085us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400771ad4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:07.438422 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.438535 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232110976}
I20260709 13:04:07.440302  9297 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.440917  9368 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:07.441486  9368 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.401721  1634 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
I20260709 13:04:07.444053  9368 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.445228  1709 raft_consensus.cc:1275] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:04:07.445626  1709 pending_rounds.cc:85] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 12
I20260709 13:04:07.445850  1709 pending_rounds.cc:107] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:04:07.446115  1709 raft_consensus.cc:2889] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.445930  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.446554  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:07.402662  9287 consensus_meta.cc:287] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 13:04:07.412050  9500 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.422938  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:07.368989  1560 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:07.460773  1560 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:07.441263  9049 raft_consensus.cc:3055] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:07.468763  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:07.469870  1627 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:07.472121  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 213634us
I20260709 13:04:07.477121  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.480517  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.481323  1557 raft_consensus.cc:2368] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:07.401943  9305 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:07.489380  8710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.022s	sys 0.021s
I20260709 13:04:07.491442  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.492373  1557 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:07.496840  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.497251  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:07.500303  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.501534  1627 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 13.
I20260709 13:04:07.466684  9287 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
I20260709 13:04:07.410454  8863 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.416906  9460 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.424664  9354 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:07.503113  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 299989us
I20260709 13:04:07.470803  9461 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:07.468043  9049 raft_consensus.cc:740] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:07.523335  1598 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:07.525365  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.531657  1628 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:07.532410  1551 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:07.472870  9500 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.533375  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:07.494926  9479 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.540772  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.541944  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.542316  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.542851  1551 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.542917  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.543663  1703 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:07.544260  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.544790  1704 raft_consensus.cc:3060] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.545548  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.548759  8984 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:07.493983  9509 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 13:04:07.530997  9358 consensus_meta.cc:287] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.004s
I20260709 13:04:07.550944  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3248) took 1087 ms. Trace:
I20260709 13:04:07.557964  1702 rpcz_store.cc:276] 0709 13:04:06.463889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.499491 (+ 35602us) service_pool.cc:224] Handling call
0709 13:04:06.508663 (+  9172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.023106 (+514443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.539113 (+516007us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54001347d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:07.550872 (+ 11759us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984917248}
I20260709 13:04:07.551646  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.559660  1707 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:07.552083  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.544807  1560 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.556090  1555 raft_consensus.cc:2393] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:07.565995  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.536396  9064 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:07.535051  9406 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:07.578694  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.586078  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.586793  1555 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:07.530903  9461 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:07.587788  9510 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:07.582074  9479 consensus_queue.cc:1059] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:04:07.597734  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:07.599663  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:07.600538  2297 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:07.582119  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:07.602948  2294 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:07.583027  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.609973  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:07.574452  9277 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:07.581571  9064 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.580991  9113 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.622323  1551 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:07.622941  1705 raft_consensus.cc:2468] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:07.627200  1598 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:07.635675  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.640365  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:07.579322  9358 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.625941  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.636708  1629 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:07.645943  1555 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:07.654126  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.626045  9064 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:07.639333  9362 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.797s	user 0.012s	sys 0.000s
I20260709 13:04:07.672510  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3205) took 1185 ms. Trace:
I20260709 13:04:07.673168  1552 rpcz_store.cc:276] 0709 13:04:06.487458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.487592 (+   134us) service_pool.cc:224] Handling call
0709 13:04:06.487906 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.256628 (+768722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.256676 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.256738 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:07.256785 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.672075 (+415290us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540036f1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:07.672151 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.672401 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737009408}
I20260709 13:04:07.657024  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.660269  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:07.717406  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 90302.8us
I20260709 13:04:07.721829  1555 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 26.
W20260709 13:04:07.722919  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 69687.9us
I20260709 13:04:07.724361  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:07.729794  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.730619  1628 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:04:07.733106  9394 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:07.733698  9394 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.734129  9394 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.736896  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.738567  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.606559  9049 consensus_queue.cc:260] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.744915  9049 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:07.752584  9512 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:04:07.752820  9062 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:07.756693  9358 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:07.759120  9241 consensus_meta.cc:287] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.884s	user 0.009s	sys 0.005s
W20260709 13:04:07.762069  9507 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:04:07.627210  9406 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 13:04:07.773974  1707 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:07.724432  9059 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.764057  9514 consensus_queue.cc:1175] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" responder_term: 18 status { last_received { term: 18 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 18 index: 15 } }
W20260709 13:04:07.793247  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.794811  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:07.637419  9113 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.726560  9390 raft_consensus.cc:3055] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:07.800706  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:07.799467  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.806486  1632 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 13.
I20260709 13:04:07.719091  9210 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.804247  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
W20260709 13:04:07.808372  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.049s	sys 0.071s
I20260709 13:04:07.725859  9509 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:07.767431  9401 raft_consensus.cc:2804] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:07.809725  1551 raft_consensus.cc:3060] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:07.810348  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.049s	sys 0.071s
W20260709 13:04:07.813510  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.093s	sys 0.162s
W20260709 13:04:07.813957  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.093s	sys 0.162s
I20260709 13:04:07.818725  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.819661  1632 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:07.791213  9461 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:07.719331  9362 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.874981  1551 raft_consensus.cc:2468] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:07.804785  9390 raft_consensus.cc:740] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:07.799474  9059 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:07.897673  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.927794  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:07.930853  1706 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:07.826409  9210 consensus_queue.cc:1059] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
I20260709 13:04:07.942998  1704 raft_consensus.cc:2468] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
W20260709 13:04:07.943621  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.979681  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:08.000918  1555 raft_consensus.cc:2368] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:07.817456  9509 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:07.977702  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.979410  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a" is_pre_election: true
I20260709 13:04:07.976588  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.990152  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.992151  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.981619  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:07.961256  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2835) took 1260 ms (client timeout 1593 ms). Trace:
I20260709 13:04:07.977183  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:07.980633  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:07.819334  9305 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.043376  1705 raft_consensus.cc:2393] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:08.043932  1706 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:08.024777  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.989270  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.069753  1709 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 12.
I20260709 13:04:08.069829  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.070947  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.046813  1708 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:07.820540  9443 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:07.988615  9513 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.395s	user 0.000s	sys 0.005s
I20260709 13:04:08.047675  1700 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:04:08.044301  1553 rpcz_store.cc:269] 0709 13:04:06.700832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.701049 (+   217us) service_pool.cc:224] Handling call
0709 13:04:07.961014 (+1259965us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54003fb454. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:07.961152 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365521792}
I20260709 13:04:08.078733  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51928 (request call id 3206) took 1590 ms. Trace:
I20260709 13:04:07.952860  9477 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:08.079732  1558 rpcz_store.cc:276] 0709 13:04:06.488338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.491859 (+  3521us) service_pool.cc:224] Handling call
0709 13:04:06.492154 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.314265 (+822111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.073761 (+759496us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54000e9054. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:08.078668 (+  4907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887135488}
I20260709 13:04:08.051052  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:08.052528  1707 raft_consensus.cc:2393] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:07.824328  8863 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.058013  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:07.973955   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:04:08.061241  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.970639  9318 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:08.089329  9318 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.090394  1560 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:07.986121  9362 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.097060  1633 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:07.951066  9210 raft_consensus.cc:3055] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:08.075073  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.075722  1675 leader_election.cc:400] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:08.100390  1675 leader_election.cc:403] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.102113  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:07.827409  9401 raft_consensus.cc:697] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:07.974454  9515 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.974349  9358 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.083333  1632 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:07.982820  9113 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:07.943816  9059 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.130378  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.065568  1551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "444b494bb53842b59408a7230884398a"
I20260709 13:04:08.132380  1551 raft_consensus.cc:2368] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:08.084872  1628 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:07.994231  9290 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:07.949662  9486 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:08.068501  9452 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.088431  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.147192  1708 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:08.002712  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.092901  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:08.081714  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 498832us
I20260709 13:04:07.980583  9460 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:07.904306  9354 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:08.096880  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.046s	sys 0.084s
W20260709 13:04:08.156014  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.046s	sys 0.084s
I20260709 13:04:08.044428  9353 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:08.074482  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:07.984774  9390 consensus_queue.cc:260] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:07.990619  1598 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.104036  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.100440  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
W20260709 13:04:08.164011  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 173465us
W20260709 13:04:08.106820  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.110966  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.074234  9474 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.119442  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:08.164836  2297 consensus_meta.cc:287] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 13:04:08.074613  9461 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:08.074259  9361 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.123627  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51916 (request call id 2846) took 1227 ms. Trace:
I20260709 13:04:08.123792  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:08.172892  1702 raft_consensus.cc:1240] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 13:04:08.124032  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 323429us
I20260709 13:04:08.173302  9474 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.173873  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:08.174587  1634 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:08.158267  1523 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:07.984239  9500 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.172458  9361 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:08.176532  9517 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:04:08.177395  9533 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.177754  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42818 (request call id 1317) took 1243 ms. Trace:
I20260709 13:04:08.177925  9533 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.172434  1560 rpcz_store.cc:276] 0709 13:04:06.889467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.896613 (+  7146us) service_pool.cc:224] Handling call
0709 13:04:07.957478 (+1060865us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54002bb1d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9dba3d 00007f70d39abcd0 00007f70ccf30254 00007f70ccf301e2 00007f70ccf30111 00007f70ccf3008c 00007f70ccf2f342 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:08.090235 (+132757us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b4c00129b80. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cccb0a20 00007f70cccc1a29 00007f70cccc213f 00007f70ccc862ab 00007f70cccb3f3c 00007f70cccb43fe 00007f70ccf4188b 00007f70ce940495 00007f70ce99130f 00007f70ce9dba55 00007f70d39abcd0
0709 13:04:08.090948 (+   713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828899584}
I20260709 13:04:08.133538  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.099931  9210 raft_consensus.cc:740] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:08.180967  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:08.181362  9210 consensus_queue.cc:260] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.182554  9210 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:08.182722  9533 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.117472  9362 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:08.148923  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:08.163605  9452 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.185935  9452 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.186328  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:08.049290  9366 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.177647  9361 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.188045  9366 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.189153  9443 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:08.190614  9366 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.190797  9474 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.190639  9443 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:08.192371  9443 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:08.193277  9443 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.150071  9486 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.196275  9486 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.197247  9509 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.198436  9064 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.200879  9064 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.135227  9290 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.201442  9290 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.178146  1630 rpcz_store.cc:276] 0709 13:04:06.934462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.934634 (+   172us) service_pool.cc:224] Handling call
0709 13:04:07.038042 (+103408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.050734 (+ 12692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.050788 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.180875 (+130087us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.813374 (+632499us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:07.813429 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.813524 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.177556 (+364032us) raft_consensus.cc:1685] finished
0709 13:04:08.177580 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.177701 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":24990,"thread_start_us":565026,"threads_started":1,"wal-append.queue_time_us":842331}
I20260709 13:04:08.166242  9353 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.187938  9406 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:08.205981  9353 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.206580  9514 raft_consensus.cc:2804] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:08.207219  9514 raft_consensus.cc:697] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:08.208218  9514 consensus_queue.cc:237] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:08.217222  9415 consensus_meta.cc:287] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.000s
W20260709 13:04:08.218104  9520 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:04:08.217830  9415 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.218981  1631 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:08.218611  9486 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.221004  1627 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:08.167976  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:08.168773  8914 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:08.138471  9358 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.169907  9305 consensus_queue.cc:1048] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:08.234349  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.170902  9318 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.137821  9515 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.172561  9461 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.188637  9361 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:08.196244  9394 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.169981  1703 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:08.165254  2297 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 27.
I20260709 13:04:08.136628  1555 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:08.165280  1709 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:08.166162  1598 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.165380  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:08.236304  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.029s	sys 0.065s
I20260709 13:04:08.236267  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:08.235298  1633 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
W20260709 13:04:08.236635  2296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.039s	sys 0.042s
W20260709 13:04:08.265307  2296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.039s	sys 0.042s
I20260709 13:04:08.240387  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:08.243976  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.046s
W20260709 13:04:08.261673  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.029s	sys 0.066s
W20260709 13:04:08.266170  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.002s	sys 0.047s
W20260709 13:04:08.266759  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 542471us
I20260709 13:04:08.261754  1708 raft_consensus.cc:2468] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:08.164883  1707 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:08.244498  9389 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.019s	sys 0.028s
W20260709 13:04:08.271409  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 83920.6us
W20260709 13:04:08.272965  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 106916us
I20260709 13:04:08.273175  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.273578  1601 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:08.257464  9361 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.274777  9361 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.274922  9475 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:08.275296  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.275975  1704 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:08.277204  9394 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:08.280165  9389 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.019s	sys 0.028s
I20260709 13:04:08.274711  9527 raft_consensus.cc:2804] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:08.283327  9527 raft_consensus.cc:697] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:08.280202  1599 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:08.284320  9527 consensus_queue.cc:237] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.262329  8933 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:08.286104  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:08.286614  9461 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.237416  9286 consensus_queue.cc:1059] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:04:08.288683  9286 raft_consensus.cc:3055] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:08.289048  9286 raft_consensus.cc:740] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:08.289150  9515 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.284701  9357 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:08.290059  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:08.289986  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:08.289966  9286 consensus_queue.cc:260] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.292773  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.245404  9500 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:08.296983  1706 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:08.303344  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:08.317507  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 141067us
I20260709 13:04:08.331837  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:08.325418  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:08.325587  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52088.2us
I20260709 13:04:08.327790  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.343691  1635 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 27.
I20260709 13:04:08.344725  1627 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:08.345000  1709 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:08.345239  1627 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:08.323690  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.353091  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.362946  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:08.364598  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.296237  9343 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:08.316020  9357 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:08.320608  9353 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:08.388152  1631 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:08.401840  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:08.403008  1706 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 19: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:08.403220  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.403596  1630 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:08.321518  9286 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:08.406373  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 82777.5us
I20260709 13:04:08.374168  1627 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.363127  9475 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:08.345842  8914 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:08.371081  9532 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:08.435887  1707 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:08.421260  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.439112  2290 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:08.421806  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.440323  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:08.443964  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.444834  1635 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:08.476527  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.415933  9366 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:08.414832  8984 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:08.447413  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.446885  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:08.475008  1599 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.376819  9406 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 13:04:08.405450  9415 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.479638  9532 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.512753  9415 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.513815  9532 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.513819  9357 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.485805  9343 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.517979  9343 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.520083  9343 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.490631  9062 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:08.520893  9062 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.521246  9062 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:08.524032  9531 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:04:08.415441  9210 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:08.443339  9354 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:08.404943  9500 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:08.490336  9469 consensus_meta.cc:287] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.117s	user 0.011s	sys 0.003s
I20260709 13:04:08.444083  9486 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.479336  9358 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.485767  9353 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:08.446462  9059 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:04:08.536250  9389 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.541626  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51916 (request call id 2853) took 1299 ms. Trace:
I20260709 13:04:08.542531  9290 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.444834  9401 consensus_queue.cc:237] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.546461  9406 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:08.509706  1704 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
W20260709 13:04:08.574721  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.575939  8984 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.574738  9354 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.594043  9389 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.577744  9469 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.591126  1559 rpcz_store.cc:276] 0709 13:04:07.242321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.275562 (+ 33241us) service_pool.cc:224] Handling call
0709 13:04:07.275971 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.389944 (+113973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.389995 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.390605 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.266359 (+875754us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:08.266401 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.266486 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.541290 (+274804us) raft_consensus.cc:1685] finished
0709 13:04:08.541325 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.541543 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113418,"raft.queue_time_us":113642,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":875281,"threads_started":1,"wal-append.queue_time_us":1117890}
I20260709 13:04:08.548952  9500 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.513300  9415 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.532548  9366 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:04:08.548086  9210 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:08.627364  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.608299  9287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.010s
I20260709 13:04:08.603503  9528 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.668606  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 481073us
W20260709 13:04:08.612780  9477 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.002s
I20260709 13:04:08.608520  9406 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:04:08.664583  9287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.007s	sys 0.010s
I20260709 13:04:08.614089  9500 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.742233  2263 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:08.743729  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:04:08.762806  2262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.003s	sys 0.023s
W20260709 13:04:08.765982  2262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.004s	sys 0.024s
I20260709 13:04:08.609476  8984 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:08.767856  1626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.007s	sys 0.015s
W20260709 13:04:08.768337  1626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.007s	sys 0.015s
W20260709 13:04:08.681718  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 499742us
W20260709 13:04:08.685127  1629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.018s	sys 0.022s
W20260709 13:04:08.772074  1629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.019s	sys 0.022s
W20260709 13:04:08.672250  9470 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.003s	sys 0.020s
I20260709 13:04:08.688815  1557 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:04:08.762929  9241 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.026s	sys 0.038s
I20260709 13:04:08.675235  9361 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.777974  8984 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.673054  9528 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:08.764319  9479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.025s	sys 0.009s
I20260709 13:04:08.790752  8933 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:08.800956  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51916 (request call id 2857) took 1304 ms. Trace:
W20260709 13:04:08.793077  1674 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.002s	sys 0.020s
W20260709 13:04:08.808959  1674 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.003s	sys 0.020s
W20260709 13:04:08.787602  9479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.026s	sys 0.010s
I20260709 13:04:08.809510  9389 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.809648  9290 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.810276  9290 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:08.810609  9479 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.810753  9290 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.811089  9479 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:08.780972  9241 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.027s	sys 0.038s
I20260709 13:04:08.812739  9518 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:08.806979  1557 rpcz_store.cc:276] 0709 13:04:07.496433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.496590 (+   157us) service_pool.cc:224] Handling call
0709 13:04:08.800679 (+1304089us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400309dd4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:08.800856 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2739550592}
I20260709 13:04:08.813149  9518 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:04:08.813613  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 684553us
W20260709 13:04:08.778997  9470 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.003s	sys 0.020s
I20260709 13:04:08.814011  9518 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.815405  9518 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:08.793028  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.015s	sys 0.030s
W20260709 13:04:08.819701  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.015s	sys 0.030s
I20260709 13:04:08.771023  9113 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:08.820130  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 706948us
I20260709 13:04:08.813163  9479 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.822124  1675 leader_election.cc:400] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:08.822460  1675 leader_election.cc:403] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.825708  9361 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.828948  9486 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:08.829000  9358 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 22. Current term is 23.
I20260709 13:04:08.829774  9486 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:08.834008  9354 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.808291  9406 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.831565  1675 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:08.830892  9486 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.857394  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.860160  9354 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.860548  1701 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:08.851814  9390 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:08.861698  1708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.010s	sys 0.022s
W20260709 13:04:08.863914  1708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.022s
I20260709 13:04:08.791741  8933 raft_consensus.cc:697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:08.865860  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:08.866195  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:08.864312  9527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.023s	sys 0.035s
I20260709 13:04:08.869689  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:08.861279  9460 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:08.873185  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.874491  1701 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:08.868880  9527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.023s	sys 0.036s
I20260709 13:04:08.871222  9361 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:08.864491  9358 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.877642  9358 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.880914  9290 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.882071  8933 consensus_queue.cc:237] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.882596  9361 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:08.884404  9528 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.885062  9536 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:08.886353  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:08.887394  9343 raft_consensus.cc:2804] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:08.887910  9343 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.888383  9343 raft_consensus.cc:3060] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.860994  9477 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.873085  8863 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:08.862978  9059 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.869424  9353 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:08.891278  9460 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:08.896210  9358 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.898015  9062 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.898689  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1740) took 1337 ms. Trace:
W20260709 13:04:08.899402  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 412546us
I20260709 13:04:08.892796  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:08.901151  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:08.928452  1704 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.860781  9354 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.926604  2296 rpcz_store.cc:276] 0709 13:04:07.561636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.563033 (+  1397us) service_pool.cc:224] Handling call
0709 13:04:07.563543 (+   510us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:08.295293 (+731750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.295341 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:08.322130 (+ 26789us) log.cc:844] Serialized 52 byte log entry
0709 13:04:08.741933 (+419803us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:08.741977 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.742049 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.898432 (+156383us) raft_consensus.cc:1685] finished
0709 13:04:08.898463 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.898624 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15464,"raft.queue_time_us":378607,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":110,"thread_start_us":980998,"threads_started":2,"wal-append.queue_time_us":521140}
I20260709 13:04:08.938748  1704 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
W20260709 13:04:08.890687  9443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.025s	sys 0.034s
W20260709 13:04:08.931389  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45337.3us
I20260709 13:04:08.899657  8863 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.901346  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:08.908834  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.033s	sys 0.034s
I20260709 13:04:08.939620  1709 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:04:08.924108  9059 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:08.924305  9533 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:04:08.959724  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.034s	sys 0.034s
I20260709 13:04:08.937587  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:08.940296  1703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.022s
W20260709 13:04:08.964027  1703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.002s	sys 0.024s
I20260709 13:04:08.929042  9415 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:08.940625  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.012s
W20260709 13:04:08.968706  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.014s
I20260709 13:04:08.965618  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:08.963315  9443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.026s	sys 0.035s
I20260709 13:04:08.930627  9358 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:08.964267  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:04:08.943202  1598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.936779  9210 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.908180  9477 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:08.969098  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 562741us
I20260709 13:04:08.981557  1523 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:08.969605  9361 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:08.941409  9353 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:09.001991  9353 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:09.002470  9353 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.003968  9361 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:09.004436  9361 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:08.994041  9486 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.012861  9210 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:09.013563  9210 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:09.014029  9210 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:08.995802  9469 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:08.962898  8863 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.993315  9277 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:09.016095  9277 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.016971  9277 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.017798  9477 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:08.990542  9358 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.020955  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3312) took 1096 ms. Trace:
I20260709 13:04:09.022774  9358 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.021487  1626 rpcz_store.cc:276] 0709 13:04:07.924922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.925066 (+   144us) service_pool.cc:224] Handling call
0709 13:04:08.009521 (+ 84455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.010886 (+  1365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.010936 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.117155 (+106219us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b2c00046b18. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cd2dda11 00007f70cd2ed372 00007f70cea1fa01 00007f70ce9e443a 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2
0709 13:04:08.124727 (+  7572us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.768840 (+644113us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:08.768893 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.769071 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.020584 (+251513us) raft_consensus.cc:1685] finished
0709 13:04:09.020615 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.020880 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32304,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":19422,"spinlock_wait_cycles":93967104,"thread_start_us":608767,"threads_started":1,"wal-append.queue_time_us":802130}
W20260709 13:04:09.003463  1450 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 214) took 15275 ms (client timeout 15000 ms). Trace:
I20260709 13:04:08.997033  9533 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.028237  9533 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:09.003320  9353 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.029389  9460 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:08.998214  9354 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.005059  9366 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.032487  9366 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.034709  9366 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:08.996007  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.031579  9062 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.038565  1701 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
W20260709 13:04:09.042584  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.004s	sys 0.017s
W20260709 13:04:09.043887  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.004s	sys 0.018s
W20260709 13:04:09.043179  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54788.4us
I20260709 13:04:09.052089  1598 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:09.044348  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.057538  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:09.059870  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.060093  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.059542  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.061969  1628 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.069625  1601 leader_election.cc:400] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031) with higher term. Message: Invalid argument: T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27.
I20260709 13:04:09.071051  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:09.042169  9357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.023s
I20260709 13:04:09.086750  9361 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.088776  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.124400  1626 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:09.090514  9357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.023s
I20260709 13:04:09.058594  9469 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.103749  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.105582  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.088375  9443 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.107277  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.133965  1630 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:09.080603  9527 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:09.109648  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.097050  9538 consensus_queue.cc:1059] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.072s
I20260709 13:04:09.111132  1627 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:09.089967  9415 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:09.128834  1633 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:04:09.106945  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.075819  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:09.164773  9361 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.167709  9460 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.079530  9049 consensus_queue.cc:1048] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:09.167945  9533 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.168846  9059 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:09.201000  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 457321us
I20260709 13:04:09.111692  9358 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.187140  1632 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:09.110152  1628 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:04:09.222990  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.226641  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.239905  2291 raft_consensus.cc:1275] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:09.241868  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.243948  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.244835  2291 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:09.178889  9527 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:04:09.218943  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 143233us
W20260709 13:04:09.187600  9394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.015s	sys 0.036s
W20260709 13:04:09.260691  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.009s	sys 0.017s
W20260709 13:04:09.261186  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.009s	sys 0.017s
I20260709 13:04:09.262920  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.263998  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.285216  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2876) took 1011 ms. Trace:
I20260709 13:04:09.135710  9469 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.294868  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.306565  2294 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:09.178700  9500 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.294735  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.273380  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.298426  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:09.265688  9394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.015s	sys 0.036s
I20260709 13:04:09.217094  9415 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:09.307123  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.310425  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.257225  9528 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:09.263932  9527 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.320366  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.333315  2288 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:09.178521  9443 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.323052  2293 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:09.311720  2289 raft_consensus.cc:1240] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:09.324249  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.328071  2295 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 19.
I20260709 13:04:09.200999  9460 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.338784  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.279268  9536 raft_consensus.cc:697] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:09.248690  9358 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.260327  9534 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:09.347625  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 85943.9us
I20260709 13:04:09.354591  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.355273  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:09.351289  9565 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:09.351890  1456 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:09.360563  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.357690  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.367406  2294 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:09.366715  2293 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:09.358562  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.360163  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:09.376699  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.375533  2288 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:09.378158  1601 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:09.383111  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.383849  1628 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:09.384552  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.385247  1631 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:09.361889  9533 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:09.363783  9460 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:09.367492  8914 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:09.390779  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.391935  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.421985  1632 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:09.386631  2260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.004s	sys 0.008s
W20260709 13:04:09.424242  2260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.004s	sys 0.012s
W20260709 13:04:09.424988  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 617966us
I20260709 13:04:09.394694  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36806 (request call id 3316) took 1308 ms. Trace:
I20260709 13:04:09.435825  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:09.426555  1629 rpcz_store.cc:276] 0709 13:04:08.085802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.085997 (+   195us) service_pool.cc:224] Handling call
0709 13:04:08.105597 (+ 19600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.106020 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:08.772463 (+666443us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.342146 (+569683us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.342716 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.342891 (+   175us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:09.352280 (+  9389us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.367313 (+ 15033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.367404 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.394394 (+ 26990us) raft_consensus.cc:1685] finished
0709 13:04:09.394427 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.394600 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77728,"prepare.queue_time_us":38604,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":86,"raft.queue_time_us":99300,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":23510,"thread_start_us":1115464,"threads_started":2,"wal-append.queue_time_us":485871}
I20260709 13:04:09.440436  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:09.442267  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.014s	sys 0.038s
I20260709 13:04:09.384773  9049 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:09.442690  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.014s	sys 0.038s
I20260709 13:04:09.442216  1628 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:09.450284  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 548079us
I20260709 13:04:09.391165  9559 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.452526  9559 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.412724  9343 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.454844  9559 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.454140  1456 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 10 to 11, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 11 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:09.393874  9415 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.439669  9366 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.456902  9443 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:09.454885  9343 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.457065  9366 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.459650  9366 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:09.455480  9541 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.462978  2288 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:04:09.465047  9358 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.466370  1628 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:09.466801  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:09.347664  9536 consensus_queue.cc:237] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.469882  8933 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:09.470784  8933 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:09.472131  9560 log.cc:927] Time spent T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:04:09.473532  8933 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:09.477567  9527 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.435789  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.392833  9528 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:09.480162  9528 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.480288  9277 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:09.480579  9528 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:09.480870  9277 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.481801  9277 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:09.483858  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.485545  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 50682.2us
I20260709 13:04:09.486394  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.386677  9390 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:09.488080  9555 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:04:09.488824  9578 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:09.489287  9578 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:09.489663  9578 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:09.488173  9390 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:09.494081  9549 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 13:04:09.495973  9390 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.297155  1708 rpcz_store.cc:276] 0709 13:04:08.274171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.280872 (+  6701us) service_pool.cc:224] Handling call
0709 13:04:08.281199 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.281403 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:08.864698 (+583295us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.284746 (+420048us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.284824 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:09.284898 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.284991 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.285140 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":538693,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":17389,"thread_start_us":997118,"threads_started":2,"wal-append.queue_time_us":412679}
I20260709 13:04:09.499853  9578 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.502162  9578 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.503125  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.503350  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.411638  8863 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.505034  9287 raft_consensus.cc:3055] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:09.505503  9287 raft_consensus.cc:740] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:09.507042  9361 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:09.515067  1627 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:09.520010  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 3269) took 1189 ms. Trace:
I20260709 13:04:09.520537  1627 rpcz_store.cc:276] 0709 13:04:08.330671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.331540 (+   869us) service_pool.cc:224] Handling call
0709 13:04:09.110713 (+779173us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b580074d2d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9400aa 00007f70ce9c2447 00007f70ce9dafe7 00007f70ce9dc7de 00007f70ce9e827c 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c
0709 13:04:09.519925 (+409212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285157248}
I20260709 13:04:09.508195  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.523950  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.524688  1628 raft_consensus.cc:2468] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:04:09.506742  9287 consensus_queue.cc:260] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.526851  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.534372  1633 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.526914  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.535955  1631 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:09.527276  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.507146  8863 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:09.535883  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 436279us
I20260709 13:04:09.507642  9361 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.466419  9500 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:09.538278  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.522444  9049 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:09.526654  9059 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:04:09.534173  9210 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.002s
I20260709 13:04:09.564265  1632 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 28.
W20260709 13:04:09.558990  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 209085us
W20260709 13:04:09.574568  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.575291  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:09.368672  9534 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.572952  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.539175  9406 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.024037  1450 rpcz_store.cc:269] 0709 13:03:53.731123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.745768 (+ 14645us) service_pool.cc:224] Handling call
0709 13:04:09.003330 (+15257562us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:02.996545 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:03.028516 (+ 31971us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:03.028570 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:04:03.028619 (+    49us) tablet.cc:662] Decoding operations
0709 13:04:03.030400 (+  1781us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:03.030494 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:03.030523 (+    29us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:04:03.112154 (+ 81631us) tablet.cc:701] Row locks acquired
0709 13:04:03.112217 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:04:03.112408 (+   191us) write_op.cc:270] Start()
0709 13:04:03.112570 (+   162us) write_op.cc:276] Timestamp: P: 1783602243112333 usec, L: 0
0709 13:04:03.112615 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:04:03.113538 (+   923us) log.cc:844] Serialized 8256 byte log entry
0709 13:04:03.480732 (+367194us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:04.010725 (+529993us) op_driver.cc:464] REPLICATION: finished
0709 13:04:04.264056 (+253331us) write_op.cc:301] APPLY: starting
0709 13:04:04.264186 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:04.693749 (+429563us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:04.693801 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:05.107526 (+413725us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:05.122673 (+ 15147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:04:05.122758 (+    85us) write_op.cc:312] APPLY: finished
0709 13:04:05.263897 (+141139us) log.cc:844] Serialized 188 byte log entry
0709 13:04:05.438680 (+174783us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:05.485103 (+ 46423us) write_op.cc:454] Released schema lock
0709 13:04:05.510221 (+ 25118us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:05.740192 (+229971us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:05.778413 (+ 38221us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1660442,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":227171,"apply.run_cpu_time_us":51177,"apply.run_wall_time_us":1308553,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91991,"num_ops":21,"prepare.queue_time_us":204739,"prepare.run_cpu_time_us":13660,"prepare.run_wall_time_us":223172,"raft.queue_time_us":324459,"raft.run_cpu_time_us":34402,"raft.run_wall_time_us":257966,"replication_time_us":862164,"spinlock_wait_cycles":6709632,"thread_start_us":707577,"threads_started":5,"wal-append.queue_time_us":332185,"wal-append.run_cpu_time_us":27787,"wal-append.run_wall_time_us":2882814}]]}
I20260709 13:04:09.545755  9361 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:09.559677  9287 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:09.559876  9353 raft_consensus.cc:2804] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:09.596225  9541 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:09.614418  2297 raft_consensus.cc:1240] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:09.596570  8863 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:09.586203  9059 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.628719  1633 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:04:09.576160  9210 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.606168  9353 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.612591  9559 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.612289  9578 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.642287  9562 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.598117  9406 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.627282  9486 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.643659  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:09.627935  9556 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.635375  9353 raft_consensus.cc:3060] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.625416  9394 consensus_queue.cc:1059] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 13:04:09.642616  9357 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.643908  8863 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.672155  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.673326  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.681260  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.682379  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.682570  9406 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:09.673559  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:09.683434  9586 log.cc:927] Time spent T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:04:09.659132  9318 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:09.684262  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.644026  9559 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.685321  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.686374  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.686582  9559 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.687388  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.676055  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.692315  2290 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:09.693176  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.677063  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.646873  9578 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.694871  9528 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.640439  9049 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.696193  2294 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 22: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:09.689585  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.675107  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.698961  2291 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.699035  9357 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.699450  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.653328  9486 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.701222  9357 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:09.704371  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.706858  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.690862  9534 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:09.683478  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.696915  9578 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.688115  8863 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.712594  1633 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:09.652612  9562 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.717712  2291 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:09.720191  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.733984  1629 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 27. Current term is 28. Ops: []
I20260709 13:04:09.734202  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.737417  1632 raft_consensus.cc:2393] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:09.738351  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.730190  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.730703  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.741282  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.748235  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.749737  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.751135  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.753405  9579 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.753191  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.754817  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.755034  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.650369  9059 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.657835  9556 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.755640  1628 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 16.
I20260709 13:04:09.759341  1631 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:09.762540  1633 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:09.764254  1525 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:09.765198  9559 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:09.765667  9559 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:09.766505  9559 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.768287  1629 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:09.768451  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:09.769402  9581 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:04:09.769713  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.770543  2293 raft_consensus.cc:2410] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:09.770663  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:09.771335  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.771965  2289 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:09.772179  9528 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.754763  9579 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.773938  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:09.727622  9486 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:09.796125  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 218348us
I20260709 13:04:09.722604  9578 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:04:09.799885  1629 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:04:09.803216  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.803897  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:09.810348  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:09.811151  1631 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:09.808838  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:09.820947  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 163407us
I20260709 13:04:09.840430  2296 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:09.843089  9565 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:09.843593  9565 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.843997  9565 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:09.844129  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 81759.3us
I20260709 13:04:09.874975  1631 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 27.
I20260709 13:04:09.874784  1632 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:04:09.878284  9318 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:09.881440  9443 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:09.811133  8933 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.046s
W20260709 13:04:09.889211  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 52154.4us
I20260709 13:04:09.895920  2289 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:09.854661  9401 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:04:09.886725  9210 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.911330  2289 raft_consensus.cc:2468] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
W20260709 13:04:09.912252  9591 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:04:09.914911  9525 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:09.918066  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.921371  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.922854  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.942836  1703 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:09.923592  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:09.925057  9588 log.cc:927] Time spent T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:04:09.925956  9580 consensus_queue.cc:1059] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:09.927532  9305 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.927997  9460 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:09.928498  9538 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.921408  9575 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:09.931107  9579 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:09.933329  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 466597us
I20260709 13:04:09.936072  1633 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:09.906261  9062 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.943686  1709 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:04:09.945070  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 92334.8us
I20260709 13:04:09.974786  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:09.948738  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.976028  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
W20260709 13:04:09.964339  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 227217us
I20260709 13:04:09.965124  1629 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:09.977685  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:09.963343  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.978557  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:09.979235  1701 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:09.969185  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.982450  1705 raft_consensus.cc:2393] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:09.982030  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:09.946482  9318 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:09.946393  9292 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:09.992615  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.987349  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:09.987152  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:09.995134  1706 raft_consensus.cc:1275] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:10.119077  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:09.985050  9579 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:09.989425  9287 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:10.177227  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 194686us
W20260709 13:04:10.119095  9594 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:04:09.990226  9318 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:10.186576  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:10.189914  9500 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:10.190996  9582 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:10.193516  9361 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.585s	user 0.001s	sys 0.010s
I20260709 13:04:10.278501  9582 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.278952  9582 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.278679  9361 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.280746  9500 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.281323  9361 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.196519  9610 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:04:10.282953  9500 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:09.978835  9575 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.285152  1704 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:10.216620  9358 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.976913  9538 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.976145  9460 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:09.984208  9062 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:09.976732  9536 consensus_queue.cc:1048] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.198s
I20260709 13:04:09.970484  9305 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.002985  9559 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:10.277833  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 91325.4us
I20260709 13:04:10.368930  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:10.127388  9572 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:10.218580  9597 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 13:04:10.232239  9556 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.297590  9358 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.315551  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.180272  9574 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:10.339699  1701 raft_consensus.cc:2468] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
W20260709 13:04:10.386675  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 411967us
I20260709 13:04:10.284850  9575 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:10.259364  9579 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.375483  9572 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:10.399137  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 278576us
I20260709 13:04:10.382998  9470 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:10.425863  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:10.431942  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.392549  9049 raft_consensus.cc:3055] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:10.427269  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:10.399183  9394 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:10.445785  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:10.444219  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:10.451191  9559 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:10.451781  9559 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.452323  9559 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:10.406380  9572 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.460796  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.461335  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.462433  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.463155  9527 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.463526  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:10.466125  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.466713  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.469262  9574 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:10.468385  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.479682  2292 raft_consensus.cc:2468] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:04:10.484596  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.439105  9470 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:10.442251  9287 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:10.442752  9049 raft_consensus.cc:740] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:10.447185  9292 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:04:10.478987  9059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.005s	sys 0.039s
I20260709 13:04:10.458791  9500 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:10.511415  9460 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.520488  9049 consensus_queue.cc:260] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.500486  9287 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.477610  9277 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.462459  9469 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:10.525329  9614 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:10.477365  9366 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.528291  9358 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:10.481014  9574 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:10.530052  9305 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.533353  9062 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:10.534646  9538 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:10.536724  8863 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:10.443876  9318 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.500284  9470 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:10.499637  9353 consensus_meta.cc:287] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.830s	user 0.011s	sys 0.001s
I20260709 13:04:10.492424  9605 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.541146  2296 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:04:10.570420  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 567921us
I20260709 13:04:10.610766  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.551867  9292 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:10.545184  9578 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.546089  9210 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.539690  9579 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.576495  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:10.632654  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:10.582764  9574 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:10.580751  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 211966us
I20260709 13:04:10.634011  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.568948  9287 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.583235  9049 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:10.578271  9562 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.634155  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:10.612460  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:10.568595  9277 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:10.636052  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59609us
W20260709 13:04:10.674568  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:10.675153  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.675236  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:10.619146  9292 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:10.637396  9578 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.680819  9277 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.680961  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 47028.2us
I20260709 13:04:10.681427  9277 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:10.681725  1524 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:10.684003  9343 raft_consensus.cc:2804] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:10.684053  9401 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:10.684535  9343 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:10.684628  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:10.685081  9343 raft_consensus.cc:3060] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.686872  9562 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:10.618323  8863 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.687978  9579 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:10.689875  8863 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.631846  9565 consensus_meta.cc:287] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.008s
W20260709 13:04:10.695852  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 83500.2us
W20260709 13:04:10.696866  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:10.704339  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:10.574066  9366 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.616315  9538 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:04:10.690318  9618 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:04:10.719074  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:10.574548  9614 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.673213  9318 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:10.612816  9353 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.671954  9358 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:10.684664  9401 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:10.722710  9562 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:10.762634  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 43656.9us
W20260709 13:04:10.761236  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:10.618940  9605 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:10.776180  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:10.667409  9210 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:10.714958  9460 raft_consensus.cc:2764] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:04:10.785725  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.792356  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.798164  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:10.799520  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.800324  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.805541  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.807092  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.811522  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:10.814380  1673 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:10.816107  1705 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:10.779657  9401 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:10.759521  9622 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:10.825529  9605 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:10.827544  9533 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:04:10.817710  9358 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:10.997012  9533 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:10.997570  1700 raft_consensus.cc:1240] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:04:10.998087  9533 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:10.999271  9533 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:10.837218  9614 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:10.838532  9292 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.003263  9292 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.004030  1703 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:11.006234  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.728987  9565 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.010273  9565 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.853037  9620 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:04:10.854929  9353 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:10.787670  9305 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:11.021968  9305 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:11.022603  9305 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:10.861580  9366 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:10.863006  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.028209  1706 raft_consensus.cc:2393] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 22: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:11.029675  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.821362  9617 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:10.868347  9606 raft_consensus.cc:3055] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:10.880345  9612 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.032052  9612 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.035141  9612 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:11.040558  9606 raft_consensus.cc:740] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:11.041615  9606 consensus_queue.cc:260] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.042806  9606 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:11.049238  9564 consensus_queue.cc:1059] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:11.050724  9534 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:11.052898  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.054977  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.056149  1703 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:04:10.892591   431 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 1541)
I20260709 13:04:11.119848  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.120698  1703 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.892637  9578 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:10.901507  9582 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.004s
I20260709 13:04:10.908308  9538 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.930893  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:10.932513  9589 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:10.953193  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1765) took 1179 ms. Trace:
I20260709 13:04:10.955502  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:10.958150  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.966331  9609 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.488s	user 0.001s	sys 0.000s
I20260709 13:04:10.969806  9575 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:11.000730  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:10.881923  9210 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:11.147625  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.148169  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.148659  1627 raft_consensus.cc:2393] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:11.149746  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.150837  1633 raft_consensus.cc:2368] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:04:11.152000  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.058161  1706 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:11.058702  9564 raft_consensus.cc:3055] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:11.076785  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.085829  9559 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:11.089208  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.108712  2288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.104s	user 0.033s	sys 0.085s
W20260709 13:04:11.169261  2288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.034s	sys 0.086s
W20260709 13:04:11.169345  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.170073  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1767) took 1238 ms. Trace:
W20260709 13:04:11.112718  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.141s	sys 0.228s
W20260709 13:04:11.117697  9572 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.614s	user 0.012s	sys 0.000s
I20260709 13:04:11.124135  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:11.182085  2296 raft_consensus.cc:2393] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:11.127930  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 313612us
I20260709 13:04:11.135332  2289 rpcz_store.cc:276] 0709 13:04:09.774020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.776354 (+  2334us) service_pool.cc:224] Handling call
0709 13:04:10.952923 (+1176569us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b5400625f54. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:10.953104 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210109184}
I20260709 13:04:11.156955  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.185823  1631 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:11.158138  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.188537  1635 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:11.189903  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.159544  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.191243  1627 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:11.192494  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:11.159984  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:11.161274  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.171343  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.141s	sys 0.228s
I20260709 13:04:11.194917  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2891) took 1227 ms. Trace:
I20260709 13:04:11.195017  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.173369  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.179629  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.170446  2288 rpcz_store.cc:276] 0709 13:04:09.933036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.933233 (+   197us) service_pool.cc:224] Handling call
0709 13:04:09.933560 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.933882 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.933910 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.949776 (+ 15866us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.169720 (+1219944us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:11.169767 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.169835 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.169877 (+    42us) raft_consensus.cc:1685] finished
0709 13:04:11.169888 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.170013 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25157,"raft.queue_time_us":161,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6296,"thread_start_us":1176019,"threads_started":1,"wal-append.queue_time_us":1134766}
I20260709 13:04:11.187342  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.203660  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.206207  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.209407  1599 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:11.212098  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.213248  1627 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:11.215227  1674 leader_election.cc:400] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:04:11.215791  1674 leader_election.cc:403] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.219937  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:11.225988  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:11.228886  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:11.231680  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.220935  2296 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:11.140828  9563 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.134124  9406 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:11.243297  1629 raft_consensus.cc:2368] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:11.245347  1628 raft_consensus.cc:1275] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:11.249198  1700 raft_consensus.cc:2393] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.171171  9415 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.167656  9564 raft_consensus.cc:740] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:11.255273  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.201944  9617 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.256215  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:11.259382  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.123s	user 0.086s	sys 0.172s
W20260709 13:04:11.259727  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.086s	sys 0.172s
I20260709 13:04:11.258145  9628 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:11.261224  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.263113  1700 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 19.
I20260709 13:04:11.264092  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.252748  9406 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.265367  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:11.251559  9287 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.560s	user 0.005s	sys 0.006s
I20260709 13:04:11.241111  9358 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 13. Current term is 14.
I20260709 13:04:11.153728  9582 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.166244  9538 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.271463  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:11.262575  9470 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.000s
I20260709 13:04:11.172255  9575 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:11.286190  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.206138  9572 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.259598  9536 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.255144  9612 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.284493  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:11.285431  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:11.171099  9589 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.279681  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:11.294591  1598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.021s	sys 0.028s
W20260709 13:04:11.296550  1598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.021s	sys 0.028s
W20260709 13:04:11.297358  2295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.035s
W20260709 13:04:11.297776  2295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.013s	sys 0.035s
W20260709 13:04:11.299064  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.299523  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.300591  9617 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:11.299719  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 613660us
W20260709 13:04:11.306805  1675 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.307191  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.283529  9210 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.278180  9366 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.309351  1709 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:11.310047  1675 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.305958  1598 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:11.315187  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.013s	sys 0.046s
W20260709 13:04:11.338056  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.013s	sys 0.046s
I20260709 13:04:11.263365  9415 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.295449  9562 raft_consensus.cc:2804] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:11.305902  9343 consensus_meta.cc:287] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.001s
W20260709 13:04:11.338743  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.301317  9574 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.000s
I20260709 13:04:11.275911  9287 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.345521  1702 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
W20260709 13:04:11.346242  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.253690  9563 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.385057  1628 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:11.387287  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.422792  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.305714  9318 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.355304  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.308264  9605 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:11.314020  9460 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:11.439782  9614 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:11.440825  1708 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.442770  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.444531  9578 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.445278  9563 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:11.423311  1631 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:11.290026  9470 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.446439  2289 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 13:04:11.446810  2289 pending_rounds.cc:85] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 19
I20260709 13:04:11.447064  2289 pending_rounds.cc:107] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 13:04:11.447316  2289 raft_consensus.cc:2889] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.297199  9612 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:11.427644  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 81437.9us
W20260709 13:04:11.451458  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.452198  1675 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.431514  9624 consensus_queue.cc:1048] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.118s
I20260709 13:04:11.431761  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:11.305409  9564 consensus_queue.cc:260] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.460497  9564 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:11.464915  9486 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.465683  9486 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.420194  9623 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:11.468760  9623 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.357215  9574 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.472990  1700 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:11.474519  9574 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.475040  1700 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:11.469972  9394 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:11.475999  9394 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:11.476753  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2898) took 1482 ms. Trace:
I20260709 13:04:11.477133  1709 rpcz_store.cc:276] 0709 13:04:09.994400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.994528 (+   128us) service_pool.cc:224] Handling call
0709 13:04:10.125226 (+130698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.335317 (+1210091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.476522 (+141205us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540010d6d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:11.476691 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2722815232}
W20260709 13:04:11.487346  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44866 (request call id 2888) took 1521 ms (client timeout 1606 ms). Trace:
W20260709 13:04:11.487910  1708 rpcz_store.cc:269] 0709 13:04:09.966088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.969320 (+  3232us) service_pool.cc:224] Handling call
0709 13:04:10.930767 (+961447us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfeee 0000000000375939 00007f70d39331e0 00007f70d39fc0c4 00007f70d39fc131 00007f70d39aa0a0 00007f70d39a9984 00007f70ccf246d4 00007f70ccf24662
0709 13:04:11.487099 (+556332us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400167954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:11.487268 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705511040}
I20260709 13:04:11.489410  9470 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.195328  1707 rpcz_store.cc:276] 0709 13:04:09.967078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.969209 (+  2131us) service_pool.cc:224] Handling call
0709 13:04:09.969600 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.969892 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:10.471323 (+501431us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.194568 (+723245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.194623 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:11.194670 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.194738 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.194859 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":707362,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":1390,"thread_start_us":1203393,"threads_started":2,"wal-append.queue_time_us":641627}
I20260709 13:04:11.497531  1707 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:04:11.499053  1707 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:11.500927  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.502523  1631 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:04:11.340876  9210 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.505645  9210 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:11.508652  2262 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.510650  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:11.432533  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.356889  9343 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.345297  9366 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:11.297011  9536 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.436761  9542 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.438551  9287 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:11.421877  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:11.455062  1675 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.512430  1626 raft_consensus.cc:2468] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:11.517905  1703 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:11.519454  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.522414  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.533277  1635 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:11.531394  9542 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.534214  9612 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:11.532011  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 76982.4us
W20260709 13:04:11.535220  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:11.535712  9305 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.005s
W20260709 13:04:11.535890  1675 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.536196  9305 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.536798  9366 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:11.537623  1701 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.471474  9623 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:11.536669  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.538930  1601 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.524166  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.539371  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:11.542745  1601 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.543212  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.546996  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:11.532405  9406 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:11.530900  9401 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.003s
W20260709 13:04:11.548152  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.535982  9601 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 1.543s	user 0.003s	sys 0.013s
I20260709 13:04:11.548518  1704 raft_consensus.cc:2468] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:11.521385  9486 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:11.529520  9357 raft_consensus.cc:2804] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:11.551120  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.564297  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.565323  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.532621  9559 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.568789  9394 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.571667  9394 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.572793  9415 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.573635  9357 raft_consensus.cc:697] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:11.574811  9305 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.574568  9357 consensus_queue.cc:237] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:11.548657  9623 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:11.548144  9366 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.576824  1629 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:11.579715  9575 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:11.506965  9318 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.582242  9318 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:11.585012  1708 raft_consensus.cc:1240] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:11.588901  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 418746us
W20260709 13:04:11.585235  1601 leader_election.cc:336] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.594954  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.596079  1631 raft_consensus.cc:2410] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
W20260709 13:04:11.590641  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.608821  2263 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.621500  2263 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.549582  9626 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:04:11.625528  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.593768  9415 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.665350  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43900.2us
W20260709 13:04:11.666437  2263 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.667001  2263 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.595382  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.610776  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.562844  9470 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:11.613889  1630 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 13:04:11.669355  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 84152.2us
I20260709 13:04:11.562325  9401 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:11.670254  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.670842  1601 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:11.615720  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.686043  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.688920  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.689574  2263 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:11.692972  2263 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.693723  2263 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:11.697691  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.698789  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.699317  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 152399us
W20260709 13:04:11.689117  1601 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:11.718166  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.717912  9574 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.707674  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:11.703727  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 494430us
I20260709 13:04:11.675367  9623 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:11.701999  2263 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.728276  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.727314  1627 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 13:04:11.711014  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.701683  9606 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:11.704185  9533 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:11.720091  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.705216  9486 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:11.673149  9318 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:11.669387  9415 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:11.734782  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.713542  9538 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:11.734743  2263 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.740723  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:11.740228  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:11.724812  9582 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.735606  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:11.712095  9559 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 13:04:11.713243  9572 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.720470  9542 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:11.740414  9634 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:04:11.742977  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:11.749011  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 508525us
W20260709 13:04:11.745754  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.750368  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.750499  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:11.749650  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 443756us
I20260709 13:04:11.734970  9486 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:11.754400  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53714.9us
I20260709 13:04:11.751961  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:11.738968  9343 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:11.759047  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:11.762441  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.769805  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:11.771572  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:11.760663  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:11.741851  9635 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:11.746784  9533 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.732412  9574 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:11.776932  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:11.780213  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.784423  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:11.785092  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:11.761847  9486 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.781700  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:11.793558  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:11.794330  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:11.787936  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 395122us
I20260709 13:04:11.778568  9572 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:11.779503  9582 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:11.759287  9605 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:11.778980  9575 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:11.779683  9623 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.802351  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.815150  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.815877  1703 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:11.816903  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.792714  9394 consensus_queue.cc:1059] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:04:11.779055  9049 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:11.818881  9641 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.819005  9049 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.820144  9469 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.816962  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.822089  1701 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:11.783871  9343 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:04:11.810247  9572 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.810125  9582 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:11.831475  2260 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45021 timed out after 2.166s (SENT) [suppressed 77 similar messages]
I20260709 13:04:11.834537  1635 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:11.820713  9469 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.814709  9626 raft_consensus.cc:2804] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:11.837073  1706 raft_consensus.cc:1275] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:11.854830  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:11.812217  9575 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.839490  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.799088  9636 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:11.843384  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.819561  9049 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:11.824914  9401 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.848031  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:11.822971  9559 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.826310  9361 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:11.833973  9556 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:11.884871  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.887903  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.895135  1701 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 21: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.896303  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:11.847577  9582 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:11.829293  9532 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:11.832100  9536 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.837797  9572 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:11.835608  9533 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:11.865461  9631 log.cc:927] Time spent T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:04:11.869701  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.906677  1627 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 13:04:11.848901  9635 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 13:04:11.910621  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:11.911445  1631 raft_consensus.cc:1240] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:11.950484  1631 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:11.913246  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3286) took 1085 ms. Trace:
I20260709 13:04:11.951962  1708 rpcz_store.cc:276] 0709 13:04:10.827464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.489007 (+661543us) service_pool.cc:224] Handling call
0709 13:04:11.584784 (+ 95777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.585568 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.913018 (+327450us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b5400097250. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:11.913169 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219391232}
I20260709 13:04:11.851310  9626 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.953384  9626 raft_consensus.cc:3060] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:11.819527  9641 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.958158  9641 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:11.883317  9575 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:11.960263  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:11.960968  2290 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:11.889204  9361 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.964339  9361 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:11.838315  9366 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.069s
I20260709 13:04:11.968376  9636 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.969138  9636 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:11.976049  1635 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
W20260709 13:04:11.977819  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 108195us
I20260709 13:04:11.978940  1674 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:11.980388  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:11.891757  9538 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.891625  9643 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:11.849287  9318 consensus_queue.cc:1059] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:04:11.988541  1674 leader_election.cc:400] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:11.989219  1674 leader_election.cc:403] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:11.998250  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 202814us
I20260709 13:04:11.945998  9559 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.919812  9556 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:12.016652  1708 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:04:12.020723  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:12.022816  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:12.032240  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:12.070680  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 276412us
I20260709 13:04:12.071132  1708 raft_consensus.cc:2393] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 26: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:12.035543  1634 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:11.892751  9562 raft_consensus.cc:697] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:12.038687  9589 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:11.983918  9536 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.898950  9639 consensus_queue.cc:1059] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.120s
I20260709 13:04:12.041399  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.044212  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:12.094146  1701 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.999289  9614 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:11.995151  9401 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:12.148341  9401 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.148954  9401 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.010455  9538 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:12.070926  9574 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:11.926205  9532 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.064592  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1674) took 1365 ms. Trace:
I20260709 13:04:12.154129  2295 rpcz_store.cc:276] 0709 13:04:10.699268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.699446 (+   178us) service_pool.cc:224] Handling call
0709 13:04:10.699835 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.700278 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.700329 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.700907 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.298225 (+597318us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:11.298283 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.298374 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.064358 (+765984us) raft_consensus.cc:1685] finished
0709 13:04:12.064382 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.064520 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":881,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":2896,"thread_start_us":590039,"threads_started":1,"wal-append.queue_time_us":908774}
I20260709 13:04:12.156243  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:12.068222  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 585351us
W20260709 13:04:12.156922  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 115605us
I20260709 13:04:11.920001  9605 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.071851  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.157490  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:12.158054  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:12.158336  1633 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:12.158231  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 86479.2us
I20260709 13:04:12.159060  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.159948  9647 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.159219  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:12.160437  9406 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:12.160871  9406 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.160913  9646 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:12.161373  9406 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:12.161823  9646 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:12.162441  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:12.163136  1701 raft_consensus.cc:2368] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20.
I20260709 13:04:12.073092  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:12.164213  1708 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:11.999503  9460 consensus_queue.cc:1048] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:04:12.164958  9565 raft_consensus.cc:3055] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:11.999094  9542 consensus_queue.cc:1059] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.061s
I20260709 13:04:12.054095  9612 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.127875  9415 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.187338  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:12.188079  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:12.189213  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 392558us
W20260709 13:04:12.132390  9653 log.cc:927] Time spent T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:04:12.133863  9636 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:12.137341  9564 consensus_meta.cc:287] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.001s
I20260709 13:04:12.138209  9469 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.031440  9559 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.270358  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44862 (request call id 3292) took 1441 ms. Trace:
I20260709 13:04:12.270644  1702 raft_consensus.cc:1275] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:12.140065  9556 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.065723  9643 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.075s
I20260709 13:04:12.144711  9361 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.145069  9562 consensus_queue.cc:237] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.147343  1700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3281) took 1321 ms. Trace:
I20260709 13:04:12.270879  1704 rpcz_store.cc:276] 0709 13:04:10.829358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.261787 (+432429us) service_pool.cc:224] Handling call
0709 13:04:12.270111 (+1008324us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54000424d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:12.270285 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580697344}
I20260709 13:04:12.278934  1700 rpcz_store.cc:276] 0709 13:04:10.826003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.476840 (+650837us) service_pool.cc:224] Handling call
0709 13:04:11.477138 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.477559 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:11.477587 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.478055 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.785977 (+307922us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:11.787077 (+  1100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.787153 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.147082 (+359929us) raft_consensus.cc:1685] finished
0709 13:04:12.147106 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.147264 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6752,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":307612,"threads_started":1,"wal-append.queue_time_us":607745}
I20260709 13:04:12.057830  9277 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.090s
I20260709 13:04:11.978611  9533 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.160236  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.153877  9532 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:12.165473  9565 raft_consensus.cc:740] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:12.157419  1673 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:12.185662  9633 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:12.159214  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:12.193435  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.194367  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.198871  2297 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:04:12.314932  2295 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:12.220435  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.221006  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:12.221726  2294 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:12.234316  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:12.316937  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 156775us
I20260709 13:04:12.275519  2296 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 13:04:12.317704  2296 pending_rounds.cc:85] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting all ops after (but not including) 15
I20260709 13:04:12.317965  2296 pending_rounds.cc:107] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 13:04:12.318197  2296 raft_consensus.cc:2889] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:12.275947  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 50819.9us
I20260709 13:04:12.290623  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:12.292915  1627 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:12.294242  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1386) took 1473 ms. Trace:
I20260709 13:04:12.319290  1702 rpcz_store.cc:276] 0709 13:04:10.820923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.424835 (+603912us) service_pool.cc:224] Handling call
0709 13:04:12.270271 (+845436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.271020 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.294010 (+ 22990us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54000424d0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e38ad 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:12.294174 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943606528}
I20260709 13:04:12.295706  1629 raft_consensus.cc:1240] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 25. Current term is 26. Ops: []
W20260709 13:04:12.005102  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 237224us
W20260709 13:04:12.307367  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 150061us
W20260709 13:04:12.323253  1601 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:12.324414  1601 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:12.324904  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:12.166708  9644 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:12.309100  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 249745us
I20260709 13:04:12.318461  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:12.226835  9563 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.333606  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.334422  2290 raft_consensus.cc:3060] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.328881  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.336504  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:12.330263  2291 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:12.294152  9636 raft_consensus.cc:3055] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:12.331183  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:12.331425  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.331575  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.362165  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.312142  9460 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:12.338052  9353 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:12.338312  9646 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:12.334105  9665 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.259907  9612 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.294765  9623 consensus_queue.cc:1048] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:04:12.372225  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 368996us
I20260709 13:04:12.343182  9636 raft_consensus.cc:740] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:12.383916  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:12.397325  2295 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:12.398561  2290 raft_consensus.cc:2468] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 20.
I20260709 13:04:12.367587  9578 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:12.368484  9343 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:12.342806  9563 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:12.432318  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.375113  9645 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:12.486225  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:12.511443  2290 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:12.487799  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:12.512827  2295 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 28.
W20260709 13:04:12.532002  2292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.012s
W20260709 13:04:12.533124  2292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.004s	sys 0.012s
I20260709 13:04:12.535501  2288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1461) took 1312 ms. Trace:
I20260709 13:04:12.536109  2288 rpcz_store.cc:276] 0709 13:04:11.223476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.224195 (+   719us) service_pool.cc:224] Handling call
0709 13:04:11.224675 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.690161 (+465486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.690203 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.690714 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.010313 (+319599us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:12.534766 (+524453us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.535043 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.535146 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.535202 (+    56us) raft_consensus.cc:1685] finished
0709 13:04:12.535219 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.535422 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":210121,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":1252987,"threads_started":3,"wal-append.queue_time_us":411936}
I20260709 13:04:12.423707  9460 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:12.423563  9343 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:04:12.524358  9606 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.787s	user 0.012s	sys 0.003s
I20260709 13:04:12.369117  9565 consensus_queue.cc:260] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:12.543503  9318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.009s	sys 0.013s
I20260709 13:04:12.529024  9564 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.425611  9575 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.544840  9415 raft_consensus.cc:3055] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:12.540236  9534 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:12.556557  9343 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.567556  9606 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:12.585537  9667 log.cc:927] Time spent T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:04:12.565325  9563 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.587059  9415 raft_consensus.cc:740] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:04:12.601176  9670 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:04:12.632640  9657 log.cc:927] Time spent T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 13:04:12.601159  9663 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 13:04:12.629778  9533 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:12.633497  9668 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.266s	user 0.000s	sys 0.000s
W20260709 13:04:12.634430  9572 consensus_meta.cc:287] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.003s
I20260709 13:04:12.635025  9572 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.636591  9612 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.637992  9572 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:12.638302  9559 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:12.642889  9662 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:12.645682  9575 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:12.551607  9636 consensus_queue.cc:260] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.647059  9636 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:12.652586  9626 consensus_meta.cc:287] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260709 13:04:12.653196  9626 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.586700  9564 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.658397  9564 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:12.659256  9658 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:04:12.600871  9664 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.119s
I20260709 13:04:12.600848  9679 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.242s
I20260709 13:04:12.663180  9679 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:12.630329  9669 log.cc:927] Time spent T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:04:12.631261  9582 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.002s
I20260709 13:04:12.664417  9582 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.666395  9676 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:04:12.663666  9679 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:12.666865  9582 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.563031  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:12.631933  9486 consensus_meta.cc:287] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.832s	user 0.010s	sys 0.005s
I20260709 13:04:12.611303  9361 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:12.734925  1707 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.037s	sys 0.076s
I20260709 13:04:12.781646  9486 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.670969  9626 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:12.740449  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3308) took 1352 ms. Trace:
I20260709 13:04:12.745852  2289 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:12.786001  1632 rpcz_store.cc:276] 0709 13:04:11.388263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.388422 (+   159us) service_pool.cc:224] Handling call
0709 13:04:11.388809 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.748122 (+359313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.748156 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.748666 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.054854 (+306188us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:12.054894 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.054970 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.739717 (+684747us) raft_consensus.cc:1685] finished
0709 13:04:12.739798 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.740189 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30711,"raft.queue_time_us":140874,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8537,"thread_start_us":633957,"threads_started":2,"wal-append.queue_time_us":632720}
W20260709 13:04:12.742528  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 358710us
I20260709 13:04:12.668004  9564 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:12.790643  9564 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:12.792824  9564 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.793784  9415 consensus_queue.cc:260] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.794783  9564 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:12.768253  9318 consensus_queue.cc:1175] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 24 status { last_received { term: 23 index: 16 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:12.742260  1626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3319) took 1148 ms. Trace:
I20260709 13:04:12.797888  9672 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.798540  1626 rpcz_store.cc:276] 0709 13:04:11.593996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.594144 (+   148us) service_pool.cc:224] Handling call
0709 13:04:11.615478 (+ 21334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.615704 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:12.269654 (+653950us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b18004d8e40. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70ce58c4bc 00007f70cb8da48b 00007f70d4734ab9 00007f70cb8f669b 00007f70cb8f493f 00007f70cb8f4455 00007f70d23dc3c7 00007f70ce9ee9a7 00007f70ce9fe0bf
0709 13:04:12.287577 (+ 17923us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.741235 (+453658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.741374 (+   139us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:12.741557 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.741701 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.742013 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":221908,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":16668,"raft.queue_time_us":38252,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":393,"spinlock_wait_cycles":17834752,"thread_start_us":641172,"threads_started":2,"wal-append.queue_time_us":389312}
I20260709 13:04:12.802191  9287 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:12.740432  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 368777us
W20260709 13:04:12.803323  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 240209us
I20260709 13:04:12.804730  1673 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:12.807107  1524 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:12.748391  9353 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:12.807519  9361 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:12.809834  9353 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.810334  9361 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:04:12.743019  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 422803us
I20260709 13:04:12.812476  9292 raft_consensus.cc:2804] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:12.813314  9292 raft_consensus.cc:697] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:12.813205  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.815325  9292 consensus_queue.cc:237] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:12.819926  1707 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.044s	sys 0.080s
I20260709 13:04:12.821812  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2903) took 1019 ms. Trace:
I20260709 13:04:12.824167  9528 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:12.668087  9679 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.835182  9679 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:12.779027  9305 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.836946  9305 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.840922  9644 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:12.822288  1707 rpcz_store.cc:276] 0709 13:04:11.802223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.804121 (+  1898us) service_pool.cc:224] Handling call
0709 13:04:11.804548 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.804846 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.804906 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.805453 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.065681 (+260228us) raft_consensus.cc:1645] Marking committed up to 20
0709 13:04:12.820679 (+754998us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.821224 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.821359 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.821460 (+   101us) raft_consensus.cc:1685] finished
0709 13:04:12.821479 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.821716 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":618205,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":1515,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"thread_start_us":1006004,"threads_started":2,"wal-append.queue_time_us":527827}
I20260709 13:04:12.874372  9305 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.876119  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:12.877471  2297 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
W20260709 13:04:12.888103  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 494090us
I20260709 13:04:12.889634  9486 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:12.890740  9486 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:12.891165  1523 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
W20260709 13:04:12.892812  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 86000.9us
I20260709 13:04:12.896214  9691 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:12.896553  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.897539  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.898958  2263 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:12.900184  2263 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:12.865662  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75610us
I20260709 13:04:12.903738  9533 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:12.904711  1598 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.905929  9644 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:04:12.906193  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:12.799093  9672 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.910521  9672 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:12.915974  9644 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:12.896275  9666 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:12.931645  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:12.941231  9692 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:12.942049  9692 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:12.927631  9666 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:12.944669  1598 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:12.951679  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 60715.9us
I20260709 13:04:12.958206  2260 leader_election.cc:400] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20.
I20260709 13:04:12.958765  2260 leader_election.cc:403] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:12.960830  9666 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:12.957971  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:12.968499  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:12.943287  9692 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:12.967614  2263 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:12.973426  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.979872  9644 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:12.967269  9486 consensus_queue.cc:1059] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.013s
W20260709 13:04:12.981494  9049 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.084s	user 0.010s	sys 0.014s
I20260709 13:04:12.982116  9049 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:12.994171  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.993402  9305 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:12.994993  9305 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:12.995379  9287 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:12.996109  9287 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.001363  9287 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.001768  9305 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:13.004283  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 81505.1us
I20260709 13:04:13.008126  9361 raft_consensus.cc:2804] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:13.008734  9361 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.009145  9361 raft_consensus.cc:3060] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:13.010540  9672 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:13.011400  9672 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.011904  9542 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20.
I20260709 13:04:13.014613  9353 raft_consensus.cc:3055] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:13.014482  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:13.015384  2297 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:13.016664  9353 raft_consensus.cc:740] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:13.023077  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 54792.2us
I20260709 13:04:13.025117  9699 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:13.025995  9699 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.026842  9636 consensus_queue.cc:1048] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:13.029016  9699 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:13.034396  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55897.7us
W20260709 13:04:13.037643  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.042155  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.043064  1706 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
I20260709 13:04:13.043725  9528 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:13.044157  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.045006  1707 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 17.
I20260709 13:04:13.045109  9672 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:13.025139  9536 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:13.046579  9536 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.049721  9536 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:13.049713  9534 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:13.051107  9406 consensus_meta.cc:287] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.011s
I20260709 13:04:13.051643  9406 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.054561  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.057760  2293 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:13.058265  2293 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:13.059374  2293 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.067235  2293 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:13.067262  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:13.071802  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.073743  1706 raft_consensus.cc:2393] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:13.075260  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57934 (request call id 1781) took 1294 ms. Trace:
W20260709 13:04:13.030988  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.003s
I20260709 13:04:13.075749  2289 rpcz_store.cc:276] 0709 13:04:11.780484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.780635 (+   151us) service_pool.cc:224] Handling call
0709 13:04:13.075021 (+1294386us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540040c254. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:13.075178 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187371264}
W20260709 13:04:13.077548  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 77312.8us
I20260709 13:04:13.078155  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1777) took 1346 ms. Trace:
I20260709 13:04:13.101150  2296 rpcz_store.cc:276] 0709 13:04:11.731394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.731534 (+   140us) service_pool.cc:224] Handling call
0709 13:04:11.731957 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.318651 (+586694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.077674 (+759023us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:13.077964 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249105152}
I20260709 13:04:13.085007  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.085569  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.104449  9049 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:13.104565  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.090289  1703 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:04:13.105484  1630 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
I20260709 13:04:13.090502  9565 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:13.107313  1702 raft_consensus.cc:1275] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:13.107993  9343 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.092658  9633 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:13.109777  9633 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:13.110438  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:13.111328  2297 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.104460  1634 raft_consensus.cc:2368] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
W20260709 13:04:13.086912  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.011s	sys 0.004s
I20260709 13:04:13.100553  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:13.086632  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.086207  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.129982  1673 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:13.141633  2293 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:04:13.149065  9460 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:13.149701  9460 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.150076  9460 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:13.153715  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 829275us
I20260709 13:04:13.121101  9343 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.155346  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.150707  9633 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.155097  1632 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:13.018242  9353 consensus_queue.cc:260] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:13.164820  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 43894.1us
I20260709 13:04:13.168778  1626 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:13.185094  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 227376us
I20260709 13:04:13.194242  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.193645  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.183311  9305 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:13.206982  1631 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.215159  2262 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.216076  9635 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:04:13.218022  9415 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:13.218436  9572 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.219125  9572 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.220427  9633 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.221626  9572 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:13.221755  9705 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:13.204332  9305 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:13.222106  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 177863us
I20260709 13:04:13.215492  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:13.241138  9589 raft_consensus.cc:3055] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:13.241703  9589 raft_consensus.cc:740] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:13.242973  9589 consensus_queue.cc:260] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.249830  9635 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:13.230674  9305 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.252920  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.256537  9675 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:13.275416  9675 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:13.275713  1674 leader_election.cc:400] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
I20260709 13:04:13.276193  1674 leader_election.cc:403] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:04:13.276649  9675 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.277643  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.281785  9681 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:13.282321  9681 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.282769  9681 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.261061  9533 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:13.283968  9533 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:13.284405  9533 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:13.288007  9679 raft_consensus.cc:3060] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:13.290017  9575 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:04:13.291502  9640 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 13:04:13.295585  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:13.270349  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.296139  1601 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:13.224462  9542 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.298672  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:13.270557  9695 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.309650  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.302392  2296 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 27.
I20260709 13:04:13.356045  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.358644  2297 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:13.360052  9675 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:13.361052  9361 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.283576  9406 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:13.397562  9401 consensus_meta.cc:287] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.248s	user 0.013s	sys 0.013s
I20260709 13:04:13.398141  9401 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.400264  1635 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:13.404094  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.407294  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.408771  9542 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.413136  1626 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:13.424321  1631 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
W20260709 13:04:13.424321  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 209414us
I20260709 13:04:13.428097  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.411535  9542 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:13.434399  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.437026  9647 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.437705  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.437978  9533 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.439425  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.445541  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.445979  9469 consensus_queue.cc:1048] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:13.447666  1631 raft_consensus.cc:1240] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:13.449959  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:13.451583  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3326) took 1407 ms. Trace:
W20260709 13:04:13.493464  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 46559.5us
I20260709 13:04:13.511816  2293 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:13.513439  1708 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 13:04:13.500413  9564 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.705s	user 0.017s	sys 0.000s
I20260709 13:04:13.518110  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.489009  1627 rpcz_store.cc:276] 0709 13:04:12.043605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.319594 (+275989us) service_pool.cc:224] Handling call
0709 13:04:12.319939 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.912279 (+592340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.912341 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.916850 (+  4509us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.927018 (+ 10168us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:12.927066 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.933099 (+  6033us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.451107 (+518008us) raft_consensus.cc:1685] finished
0709 13:04:13.451129 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.451367 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8627,"raft.queue_time_us":42940,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5884,"thread_start_us":457622,"threads_started":2,"wal-append.queue_time_us":80989}
W20260709 13:04:13.528044  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 145848us
I20260709 13:04:13.528579  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.534472  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.535606  1709 raft_consensus.cc:2393] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 20: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:13.535452  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.536600  1703 raft_consensus.cc:2468] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:13.536988  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.537650  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.538519  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.538946  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.539469  1709 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:13.540218  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.532100  1706 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.510849  9695 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.545295  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.549197  1708 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:13.544864  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:13.551893  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 370219us
I20260709 13:04:13.503569  9647 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.560001  1701 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:13.560303  1525 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.561585  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.562331  9692 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:13.562997  9692 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:13.563318  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.564136  9692 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.565523  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.566809  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.561712  9664 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:13.567818  9664 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:13.569293  1700 raft_consensus.cc:1240] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:04:13.569108  9664 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:13.574859  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.575979  1700 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:04:13.563246  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.543681  9534 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:13.559212  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:13.584911  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 288819us
W20260709 13:04:13.589233  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 139679us
I20260709 13:04:13.583981  9575 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.594820  9401 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.612273  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:13.624626  1708 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:13.598664  9564 raft_consensus.cc:2764] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:04:13.623409  1706 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:13.631613  1632 raft_consensus.cc:1240] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:13.634531  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.630424  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.640108  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:13.641027  1708 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:13.616926  9415 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:13.622464  9361 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.638523  9533 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:13.647821  9401 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:13.647874  1599 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.653295  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:13.643421  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:13.661209  9401 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.661878  9401 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:13.664270  9646 consensus_queue.cc:1059] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:13.665516  9701 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:04:13.665568  9366 raft_consensus.cc:3055] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:13.666229  9701 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.666296  9366 raft_consensus.cc:740] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:13.666713  9675 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.666874  9701 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:04:13.667534  9366 consensus_queue.cc:260] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.669106  9366 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.670766  1599 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:13.673869  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 72518.8us
I20260709 13:04:13.622965  9575 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:13.676213  2261 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.676908  9575 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:13.677603  9575 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:13.622815  9460 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.680459  1633 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: []
W20260709 13:04:13.683099  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.633808  9572 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.682039  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.699206  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.679359  9415 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.720276  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.676963  9695 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.684577  9469 raft_consensus.cc:2764] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 13:04:13.753769  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.755065  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.756027  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.699697  9460 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.678236  9575 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.702504  9564 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:13.686074  9640 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.074s
W20260709 13:04:13.789371  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 66718.1us
I20260709 13:04:13.745095  9049 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 13:04:13.789160  9697 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.963s	user 0.007s	sys 0.005s
W20260709 13:04:13.789237  9679 consensus_meta.cc:287] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.005s
I20260709 13:04:13.743990  9572 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:13.800037  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 210586us
I20260709 13:04:13.801262  1708 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
W20260709 13:04:13.804018  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.807139  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.790181  9460 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:13.789556  9361 raft_consensus.cc:3060] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:13.810477  1705 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:04:13.792521  9647 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.794044  9646 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:13.812933  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:13.818923  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 133881us
W20260709 13:04:13.821400  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.821868  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.830596  9486 consensus_queue.cc:1059] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:13.831676  9486 raft_consensus.cc:3055] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:13.832046  9486 raft_consensus.cc:740] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:13.832418  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:13.833209  1707 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:13.832854  9486 consensus_queue.cc:260] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.834157  9486 raft_consensus.cc:3060] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:13.836088  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 207782us
W20260709 13:04:13.836930  9722 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:04:13.819438  9542 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.839231  9542 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.841923  9542 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:13.842644  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:13.843356  2296 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 27.
I20260709 13:04:13.844797  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:13.845130  9353 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.848112  2297 raft_consensus.cc:2393] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 20: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:04:13.849213  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 94318us
I20260709 13:04:13.793710  9564 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.863916  9460 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.812141  9679 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 24. Current term is 25.
I20260709 13:04:13.855927  9730 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:13.856271  1633 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:04:13.872044  9415 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.814618  9640 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:13.853272  1599 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:13.815546  9695 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:13.902315  1709 raft_consensus.cc:2468] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
W20260709 13:04:13.889211  9728 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:04:13.909482  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 56320.8us
I20260709 13:04:13.929620  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:13.904130  9695 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:13.911414  9633 consensus_meta.cc:287] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.690s	user 0.002s	sys 0.017s
I20260709 13:04:13.911955  9575 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.916570  9564 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:13.918069  9049 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:13.905880  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:13.919095  9460 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.924993  9572 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:13.925997  1703 raft_consensus.cc:2468] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
W20260709 13:04:13.958483  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.965541  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 115535us
I20260709 13:04:13.968930  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.958894  9695 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:13.970489  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:13.978986  1632 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:04:13.979439  1632 pending_rounds.cc:85] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Aborting all ops after (but not including) 13
I20260709 13:04:13.966775  9049 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:13.984722  9626 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:13.985436  9626 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:13.987133  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36806 (request call id 3358) took 1973 ms. Trace:
I20260709 13:04:13.987756  1635 rpcz_store.cc:276] 0709 13:04:12.014105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.220716 (+206611us) service_pool.cc:224] Handling call
0709 13:04:13.986746 (+1766030us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400280854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:13.986991 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490225024}
I20260709 13:04:13.979671  1632 pending_rounds.cc:107] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:04:13.995003  1632 raft_consensus.cc:2889] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.000011  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.000360  9210 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:14.001384  9210 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.002247  1523 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.003512  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.004019  9692 raft_consensus.cc:3060] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.005676  9675 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:13.999598  9305 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.007s
I20260709 13:04:14.007014  9305 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.003731  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.014372  1705 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:13.978773  9572 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:13.961360  9640 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.988514  9049 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.022768  9692 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:14.030462  9732 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.173s	user 0.000s	sys 0.003s
I20260709 13:04:14.030323  9701 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:13.987726  9626 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:14.036991  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.038591  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.039481  9646 raft_consensus.cc:2804] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.039965  9646 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.040369  9646 raft_consensus.cc:3060] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:14.044764  9210 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.045763  9210 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:14.051898  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 212124us
I20260709 13:04:14.057138  9701 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.061124  9589 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.064812  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36804 (request call id 3316) took 2644 ms (client timeout 1821 ms). Trace:
W20260709 13:04:14.065376  1628 rpcz_store.cc:269] 0709 13:04:11.420384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.421548 (+  1164us) service_pool.cc:224] Handling call
0709 13:04:14.064381 (+2642833us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001af9d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:14.064577 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4347640192}
I20260709 13:04:14.065984  9640 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:14.066712  9049 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:14.067505  9725 log.cc:927] Time spent T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:04:14.068328  2289 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
W20260709 13:04:14.073621  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 264757us
W20260709 13:04:14.071506  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 240805us
I20260709 13:04:14.076783  1598 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.015444  9305 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:14.077140  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 147611us
W20260709 13:04:14.122069  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45384.2us
I20260709 13:04:14.123399  1598 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.071097  9632 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.074395  9640 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.068307  9646 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:14.135085  9542 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.155949  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.140728  9469 raft_consensus.cc:2804] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:14.162624  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 124150us
I20260709 13:04:14.164331  9645 raft_consensus.cc:2804] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:14.164984  9645 raft_consensus.cc:697] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:14.165921  9645 consensus_queue.cc:237] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.174393  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:14.176403  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55072.5us
I20260709 13:04:14.182062  1631 raft_consensus.cc:1240] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:14.148835  9640 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.221575  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.204205  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.222379  2289 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:14.176991  9469 raft_consensus.cc:697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:14.227051  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:14.226955  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.200305  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:14.205329  9486 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:14.162284  9542 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.157547  9632 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.238878  1634 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 18: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
I20260709 13:04:14.206869  9305 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.240231  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.237010  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:14.244210  1631 raft_consensus.cc:3060] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:14.247537  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 73213.2us
I20260709 13:04:14.256628  9743 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:14.257568  9743 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.259455  9729 log.cc:927] Time spent T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.402s	user 0.001s	sys 0.001s
I20260709 13:04:14.252156  9353 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:04:14.261683  9695 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.264577  9738 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.265103  9745 log.cc:927] Time spent T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:14.265224  9738 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.264623  9744 log.cc:927] Time spent T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:14.268661  9738 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:14.269446  9736 log.cc:927] Time spent T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.527s	user 0.003s	sys 0.000s
I20260709 13:04:14.269961  9695 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:14.270674  9401 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.006s
I20260709 13:04:14.271061  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.271817  9305 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.203984  9406 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
W20260709 13:04:14.272789  9726 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.545s	user 0.005s	sys 0.011s
I20260709 13:04:14.273542  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.274192  9305 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:14.274303  1701 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 26.
I20260709 13:04:14.256729  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2924) took 1137 ms. Trace:
I20260709 13:04:14.279127  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:14.280052  1703 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.251108  9681 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.968s	user 0.012s	sys 0.014s
I20260709 13:04:14.281962  2292 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:14.285151  1523 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:14.286185  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.271128  9401 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.290830  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.294291  9746 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:14.276497  1702 rpcz_store.cc:276] 0709 13:04:13.119883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.126454 (+  6571us) service_pool.cc:224] Handling call
0709 13:04:14.010348 (+883894us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b580021ff98. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb8bfb71 00007f70cb8bfa22 00007f70d47d8c09 00007f70d3932ee0 00007f70d39fc0ab 00007f70d39fc131 00007f70d39bdfc0 00007f70d39a8929 00007f70d39a83d2 00007f70ccf1fbd4
0709 13:04:14.010585 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.010851 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:14.011198 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.256344 (+245146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.256412 (+    68us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:14.256464 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.256526 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.256646 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24934,"prepare.queue_time_us":680,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":30254,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":22060928,"thread_start_us":215451,"threads_started":1}
I20260709 13:04:14.302176  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.302934  2292 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:14.308559  1701 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:04:14.311823  9401 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.312016  1626 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:14.312306  9646 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:14.313719  9361 consensus_meta.cc:287] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.500s	user 0.006s	sys 0.003s
I20260709 13:04:14.313432  9469 consensus_queue.cc:237] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.314193  9361 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:14.320405  1706 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:14.322183  1703 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:04:14.336514  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.344915  1631 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
W20260709 13:04:14.345373  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 191733us
I20260709 13:04:14.351393  1631 raft_consensus.cc:1240] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:14.353034  2297 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:04:14.357430  2292 raft_consensus.cc:2468] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:14.315940  9632 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.309935  9743 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.362679  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:14.366616  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:14.366079  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 138591us
I20260709 13:04:14.292742  9681 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.354467  9664 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:14.372293  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.364319  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.375372  1627 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
W20260709 13:04:14.375855  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.371551  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.378134  1633 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 26.
I20260709 13:04:14.383320  1708 raft_consensus.cc:1240] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:14.385771  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.394030  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.401751  9049 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.404126  2260 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:14.406759  9691 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:14.407351  9691 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:14.408316  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 159569us
I20260709 13:04:14.408710  9534 consensus_queue.cc:1059] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:14.409375  1523 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.410538  9664 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.411161  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.412791  9743 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.424579  9747 log.cc:927] Time spent T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:04:14.425038  9700 log.cc:927] Time spent T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:04:14.432060  9406 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:14.433377  9723 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.691s	user 0.009s	sys 0.000s
W20260709 13:04:14.434273  9714 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.931s	user 0.003s	sys 0.000s
I20260709 13:04:14.441668  9681 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:14.441817  9542 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:14.445007  9564 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:14.449501  9640 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:14.451707  2290 raft_consensus.cc:1240] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:14.455885  9691 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:14.457175  9633 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.457250  2294 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:14.457815  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.458228  9664 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:04:14.459349  2260 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:14.466729  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48546.9us
I20260709 13:04:14.465848  1674 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.478243  2289 raft_consensus.cc:2468] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
W20260709 13:04:14.486683  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:14.487246  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 287025us
W20260709 13:04:14.490448  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.519460  1675 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.495906  2262 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.500566  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.504441  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.389523  9565 raft_consensus.cc:3055] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:14.506525  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:14.510958  2288 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
W20260709 13:04:14.523290  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64007us
I20260709 13:04:14.498075  9406 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:14.518929  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:14.528612  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62834.6us
I20260709 13:04:14.534983  9305 raft_consensus.cc:2804] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:14.547987  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.516911  9743 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.548892  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 137848us
I20260709 13:04:14.549175  9406 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.550011  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 54262.2us
I20260709 13:04:14.550249  9743 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.511415  9738 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:04:14.550781  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.513487  9739 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.551601  1708 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:14.528311  9632 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:14.524633  9633 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.552058  9739 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.552300  9632 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.553637  9564 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:14.554109  9664 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:14.554322  9632 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.554455  2292 raft_consensus.cc:1275] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 13:04:14.554780  9664 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.522279  9565 raft_consensus.cc:740] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:14.556617  9460 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.557071  9460 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.532873  1633 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
I20260709 13:04:14.556190  1673 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:14.550899  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.528623  9589 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:14.549803  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:14.530822  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.561710  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3330) took 1440 ms. Trace:
W20260709 13:04:14.566833  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 176314us
W20260709 13:04:14.571981  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 44750.3us
I20260709 13:04:14.574049  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.574203  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.575368  2289 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:14.579085  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 47049.7us
I20260709 13:04:14.563117  1630 rpcz_store.cc:276] 0709 13:04:13.121464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.121604 (+   140us) service_pool.cc:224] Handling call
0709 13:04:13.164079 (+ 42475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.212435 (+ 48356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.212474 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.248384 (+ 35910us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.746266 (+497882us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:13.746337 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.746431 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.374476 (+628045us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:14.533765 (+159289us) raft_consensus.cc:1685] finished
0709 13:04:14.533786 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.561643 (+ 27857us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1013,"raft.queue_time_us":5435,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":16163,"spinlock_wait_cycles":20583680,"thread_start_us":233997,"threads_started":1,"wal-append.queue_time_us":289777}
I20260709 13:04:14.557418  9460 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.576970  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:14.583060  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.557446  9542 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.583680  2293 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:14.577625  2263 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.586007  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.557291  9565 consensus_queue.cc:260] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.561633  9353 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:14.566325  9695 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.571310  9572 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:14.599413  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 109011us
I20260709 13:04:14.640112  9695 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.641141  9739 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:14.641523  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.642024  1675 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.643098  9695 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.601765  9750 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.644093  9750 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.602609  9277 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:14.646292  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.646750  9750 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.647039  2292 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:14.577831  9675 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:14.648192  9049 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.648727  9675 consensus_queue.cc:1059] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:14.649888  9675 raft_consensus.cc:3055] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:14.650241  9675 raft_consensus.cc:740] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:14.586441  9746 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:14.652552  1634 raft_consensus.cc:1240] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:14.653128  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.655143  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.574879  9575 raft_consensus.cc:2804] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:14.655917  1702 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 17.
I20260709 13:04:14.656164  9575 raft_consensus.cc:697] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:14.577454  9486 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:14.657145  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.657660  1709 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:14.657810  9572 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:14.657931  1706 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:14.589408  9646 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 13:04:14.604316  9534 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.608032  2295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1691) took 1500 ms. Trace:
I20260709 13:04:14.659133  9534 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:14.608186  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58474.9us
I20260709 13:04:14.659621  9534 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:14.661368  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.661897  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:14.580330  9305 raft_consensus.cc:697] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:14.664312  9406 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.657195  9575 consensus_queue.cc:237] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.611927  9401 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:14.613802  9753 consensus_queue.cc:1048] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:04:14.667996  9401 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:14.668510  9401 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:14.669106  9753 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:14.669188  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.669896  9753 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.670701  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.672223  9746 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:14.672748  9746 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.659318  2295 rpcz_store.cc:276] 0709 13:04:13.096222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.097389 (+  1167us) service_pool.cc:224] Handling call
0709 13:04:13.097823 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.098448 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.098483 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.099002 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.195961 (+ 96959us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:04:13.196006 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.196079 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.340986 (+1144907us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4400022b4c. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb81dc85 00007f70cb81e6a2 00007f70ce5901db 00007f70ce58baaf 00007f70ce57f86e 00007f70ce57ebb6 00007f70ce57e7d8 00007f70ce588221 00007f70ce5881d9 00007f70ce588169
0709 13:04:14.588672 (+247686us) raft_consensus.cc:1685] finished
0709 13:04:14.588710 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.596090 (+  7380us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":170,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":150,"spinlock_wait_cycles":11384064,"thread_start_us":96505,"threads_started":1,"wal-append.queue_time_us":133099}
I20260709 13:04:14.673076  9746 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.674067  9486 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:14.676321  9049 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:14.676909  9049 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.664335  9305 consensus_queue.cc:237] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.677317  9049 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.678252  9753 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:14.615482  1601 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.617307  9664 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.618541  9640 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:14.680413  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 64982.1us
W20260709 13:04:14.684659  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:14.685698  1601 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.625669  2289 raft_consensus.cc:2468] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 28.
I20260709 13:04:14.631264  9565 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:14.632227  9353 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.634585  9701 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:14.644344  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.706173  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.604965  9633 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:14.582002  9679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.013s	sys 0.034s
W20260709 13:04:14.643565  1675 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.676043  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:14.702919  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.703503  2293 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:14.706542  2288 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:14.708510  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.711651  2297 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:14.713586  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.704115  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:14.711128  1675 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:14.716010  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.683740  9640 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
W20260709 13:04:14.709699  9679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.013s	sys 0.034s
I20260709 13:04:14.724352  9640 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.714975  9664 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:04:14.730152  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.730878  1457 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:14.725459  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 49519.2us
W20260709 13:04:14.727340   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:14.715118  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.737700  1634 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:04:14.741505  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.742437  2296 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:14.651211  9675 consensus_queue.cc:260] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.746115  1599 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.751446  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.761631  1634 raft_consensus.cc:1275] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:14.714080  9542 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:14.761375  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.764861  2289 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:14.725771  9640 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.787411  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:14.708364  9701 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:14.727821  9626 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:14.793735  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 216135us
W20260709 13:04:14.801395  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 300) took 96147 ms (client timeout 15000 ms). Trace:
W20260709 13:04:14.804113  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 160562us
I20260709 13:04:14.806032  9353 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.806972  9675 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:14.709475  9210 consensus_queue.cc:1059] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
I20260709 13:04:14.812553  9210 raft_consensus.cc:3055] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:14.813126  9210 raft_consensus.cc:740] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:14.814317  9210 consensus_queue.cc:260] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.815851  9739 consensus_queue.cc:1175] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "00f3fbc60b884078a3c219909b44f64c" responder_term: 23 status { last_received { term: 20 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:14.764283  9647 raft_consensus.cc:2804] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:14.817543  9647 raft_consensus.cc:697] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 28 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:14.817422  9401 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.820215  9626 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:14.820709  9460 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.821549  9626 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.823310  9460 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.824488  9277 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:14.825942  9691 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.784618  9679 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.787354  9666 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.791008  9750 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.828321  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.872115  2289 raft_consensus.cc:2393] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:04:14.828523  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 142841us
W20260709 13:04:14.873234  1601 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.785563  9646 consensus_queue.cc:1059] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:04:14.828714  1457 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:14.834491  9401 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:14.838994  9701 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:14.840762  1709 raft_consensus.cc:1275] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:04:14.842070  2292 raft_consensus.cc:1240] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:14.843948  1706 raft_consensus.cc:2368] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18.
I20260709 13:04:14.850011  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.768582  1599 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.764714  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:14.858140  2296 raft_consensus.cc:2468] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 28.
I20260709 13:04:14.886549  2288 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28. Ops: [27.19-27.19]
W20260709 13:04:14.864622  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 287718us
I20260709 13:04:14.874059  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:14.891433  1626 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:14.761154  9695 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:14.893358  2260 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.895879  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 127388us
W20260709 13:04:14.897056  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 132425us
I20260709 13:04:14.819048  9647 consensus_queue.cc:237] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:14.905095  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.905733  2289 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.908478  1601 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.871526  9750 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.872864  9210 raft_consensus.cc:3060] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:14.917349  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:14.909126  1457 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:14.882375  9589 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:14.933805  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:14.934648  1626 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.907567  9353 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:14.932008  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.904563  9701 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:14.917013  9750 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:14.878787  9666 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:14.801888  1456 rpcz_store.cc:269] 0709 13:02:38.653910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.656208 (+  2298us) service_pool.cc:224] Handling call
0709 13:04:14.801164 (+96144956us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:09.885633 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:09.899554 (+ 13921us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:09.899593 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:04:09.899610 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:09.900044 (+   434us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:09.900107 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:09.900138 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:09.900466 (+   328us) tablet.cc:701] Row locks acquired
0709 13:04:09.900488 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:04:09.900638 (+   150us) write_op.cc:270] Start()
0709 13:04:09.900761 (+   123us) write_op.cc:276] Timestamp: P: 1783602249900587 usec, L: 0
0709 13:04:09.900783 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:09.901491 (+   708us) log.cc:844] Serialized 929 byte log entry
0709 13:04:10.526449 (+624958us) op_driver.cc:464] REPLICATION: finished
0709 13:04:11.118658 (+592209us) write_op.cc:301] APPLY: starting
0709 13:04:11.118763 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:11.129874 (+ 11111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:11.145169 (+ 15295us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:11.500090 (+354921us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:11.500289 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:04:11.500327 (+    38us) write_op.cc:312] APPLY: finished
0709 13:04:11.500542 (+   215us) log.cc:844] Serialized 33 byte log entry
0709 13:04:11.798721 (+298179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:11.807464 (+  8743us) write_op.cc:454] Released schema lock
0709 13:04:11.807522 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:11.808138 (+   616us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:11.808177 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":531,"reported_tablets":48,"spinlock_wait_cycles":1280,"child_traces":[["op",{"apply.queue_time_us":591831,"apply.run_cpu_time_us":20175,"apply.run_wall_time_us":689438,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"mutex_wait_us":12600,"num_ops":2,"prepare.queue_time_us":243742,"prepare.run_cpu_time_us":30705,"prepare.run_wall_time_us":115360,"raft.queue_time_us":305218,"raft.run_cpu_time_us":29594,"raft.run_wall_time_us":248229,"replication_time_us":625479,"thread_start_us":938982,"threads_started":5,"wal-append.queue_time_us":701681,"wal-append.run_cpu_time_us":26087,"wal-append.run_wall_time_us":2513733}]]}
I20260709 13:04:14.920683  9763 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:14.967482  1457 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:14.968592  1599 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:14.948583  9743 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.938331  9762 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:14.920109  9460 consensus_queue.cc:1048] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:04:14.914338  9632 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 13:04:14.980559  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 75784.4us
I20260709 13:04:14.983349  1706 raft_consensus.cc:1240] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: []
W20260709 13:04:14.976666  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.985677  2263 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:14.935879  9401 consensus_queue.cc:1048] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:04:14.963200  9764 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.993319  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.991359  2289 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:14.989892  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:14.997799  9691 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:14.998862  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:14.998744  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:15.000469  1701 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:14.972301  9763 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:14.939585  9534 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.026631  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:14.994304  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 10 to 13, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:14.979722  9743 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.977102  9469 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:15.028353  1673 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:15.032820  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:15.034375  2263 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:15.039903  9742 raft_consensus.cc:2804] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:15.040100  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:15.034624  1708 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:14.966320  9664 consensus_queue.cc:1059] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 13:04:15.035219  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:15.043113  9361 raft_consensus.cc:3055] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:15.047915  1709 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 13:04:15.064198  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 155726us
I20260709 13:04:15.067504  1626 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:14.991039  9701 raft_consensus.cc:697] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:15.046274  1457 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 16, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 16 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:15.043666  9361 raft_consensus.cc:740] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:15.071539  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:14.975746  9353 consensus_queue.cc:1059] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:15.082969  2294 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:04:15.086172  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 177693us
I20260709 13:04:15.030390  9763 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.090323  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:15.035032  9751 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:15.045068  9743 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.063649  9742 raft_consensus.cc:697] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:15.044771  9562 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:15.099884  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 219389us
I20260709 13:04:15.069799  9305 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:15.103394  1598 leader_election.cc:400] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18.
I20260709 13:04:15.110674  1598 leader_election.cc:403] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:15.109110  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:15.081152  9572 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.112207  1626 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:15.109964  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.113361  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 168870us
I20260709 13:04:15.026584  9764 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.070043  9589 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.117523  1708 raft_consensus.cc:1275] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 13:04:15.113862  9626 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.117465  9469 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 for earlier term 17. Current term is 18.
I20260709 13:04:15.118398  9305 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:15.220935  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:15.105154  9751 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.225356  1629 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:15.117600  9572 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:15.230041  9770 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:04:15.230608  9742 consensus_queue.cc:237] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.225329  9626 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.232502  9768 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:15.233489  9772 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:15.233943  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:15.234705  1630 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:15.198822  9776 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:15.276574  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:15.281529  1627 raft_consensus.cc:1240] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 28. Current term is 29. Ops: []
I20260709 13:04:15.114944  9562 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.303808  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:15.309152  1635 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:15.226845  9623 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.234014  9277 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.291075  9714 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:15.206140  9361 consensus_queue.cc:260] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.386205  9277 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:15.407318  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 602430us
W20260709 13:04:15.408414  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:15.408914  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:15.411795  9774 log.cc:927] Time spent T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:04:15.413722  2295 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:04:15.372192  9353 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:15.216634  9701 consensus_queue.cc:237] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.379581  9778 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.413007  9777 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:15.386435  9666 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:15.437793  1601 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:15.438437  9758 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.443113  9782 log.cc:927] Time spent T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:15.446404  9353 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.400034  9305 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.424497  9361 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:15.551304  9486 raft_consensus.cc:3055] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:15.378813  9623 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.586017  9486 raft_consensus.cc:740] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:15.687530  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:04:15.688841  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 68543.8us
I20260709 13:04:15.688385  9486 consensus_queue.cc:260] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.689939  9486 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:15.563974  9758 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.553057  9777 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:15.730496  9779 log.cc:927] Time spent T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 13:04:15.735085   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1585)
W20260709 13:04:15.723157  9787 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:04:15.544037  9778 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.758337  2290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1805) took 1135 ms. Trace:
I20260709 13:04:15.758060  9646 raft_consensus.cc:3055] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:15.764312  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:15.765967  1634 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:15.768217  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.765028  9469 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:15.782946  9469 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.784010  2293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1809) took 1070 ms. Trace:
I20260709 13:04:15.765627  9646 raft_consensus.cc:740] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:15.784561  2293 rpcz_store.cc:276] 0709 13:04:14.713968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.718797 (+  4829us) service_pool.cc:224] Handling call
0709 13:04:14.719233 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.931247 (+212014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.931284 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.948212 (+ 16928us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.191979 (+243767us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:04:15.192048 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.192159 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.761673 (+569514us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b24003132f8. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70cb77fdb1 00007f70cb77d9c4 00007f70cb77bacb 00007f70cb936dc9 00007f70cb936a55 00007f70cb935fb0 00007f70ce5ac488 00007f70ce5acbe8 00007f70ce584398 00007f70ce57f76a 00007f70ce57ebb6
0709 13:04:15.763011 (+  1338us) raft_consensus.cc:1685] finished
0709 13:04:15.763039 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.783910 (+ 20871us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":269474,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":39127,"spinlock_wait_cycles":13482240,"thread_start_us":400781,"threads_started":2,"wal-append.queue_time_us":258383}
I20260709 13:04:15.786808  9646 consensus_queue.cc:260] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.789855  9666 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.790601  9666 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.758937  2290 rpcz_store.cc:276] 0709 13:04:14.622854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.623090 (+   236us) service_pool.cc:224] Handling call
0709 13:04:14.623487 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.648767 (+ 25280us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:04:14.873833 (+225066us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.193878 (+320045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:15.193958 (+    80us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:15.194032 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.421477 (+227445us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b1800358ea0. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f7522 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:15.758011 (+336534us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54007701d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:15.758080 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.758247 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114583,"prepare.queue_time_us":318686,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":152685,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":16227,"spinlock_wait_cycles":108588160,"thread_start_us":512028,"threads_started":2}
I20260709 13:04:15.799636  9786 consensus_queue.cc:1048] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:15.838833  9784 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.853356  9784 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.230955  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:15.304193  1457 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:15.916574  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 478828us
W20260709 13:04:15.917816  1601 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:15.919610  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 486200us
W20260709 13:04:15.921756  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.934861  1626 consensus_meta.cc:287] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.738s	user 0.006s	sys 0.000s
I20260709 13:04:15.935483  1626 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
W20260709 13:04:15.938539  2289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.006s	sys 0.005s
W20260709 13:04:15.939168  2289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.006s	sys 0.005s
W20260709 13:04:15.939635  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.030s	sys 0.028s
W20260709 13:04:15.939855  9775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.004s	sys 0.006s
W20260709 13:04:15.940723  9775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.004s	sys 0.007s
W20260709 13:04:15.941618  1523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.011s	sys 0.004s
W20260709 13:04:15.942322  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.000s	sys 0.012s
W20260709 13:04:15.943176  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.000s	sys 0.012s
W20260709 13:04:15.942091  1523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.004s
W20260709 13:04:15.946509  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.940055  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.031s	sys 0.028s
W20260709 13:04:15.944025  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 975443us
I20260709 13:04:15.957187  9277 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:15.957875  9277 consensus_queue.cc:1059] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:04:15.959946  9565 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.254s	user 0.014s	sys 0.029s
W20260709 13:04:15.960163  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 551784us
W20260709 13:04:15.960861  1675 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:15.961283  1675 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:15.957945  9049 consensus_meta.cc:287] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.280s	user 0.029s	sys 0.068s
I20260709 13:04:15.962052  9049 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.962046  9623 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.965083  9049 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:15.967023  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1488) took 1284 ms. Trace:
I20260709 13:04:15.967443  2291 rpcz_store.cc:276] 0709 13:04:14.683039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.683178 (+   139us) service_pool.cc:224] Handling call
0709 13:04:14.708293 (+ 25115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.882642 (+174349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.882676 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.912191 (+ 29515us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.990893 (+ 78702us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:14.990932 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.991003 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.966800 (+975797us) raft_consensus.cc:1685] finished
0709 13:04:15.966828 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.966963 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":266961,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":16600,"spinlock_wait_cycles":34944,"thread_start_us":216275,"threads_started":2,"wal-append.queue_time_us":671621}
I20260709 13:04:15.968565  9796 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.969142  9796 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:15.973937  9764 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:15.974839  9469 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.976361  9626 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:15.976965  9626 raft_consensus.cc:3055] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:04:15.977293  9626 raft_consensus.cc:740] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:15.980055  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:15.980288  9679 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:15.980764  2292 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:15.980929  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:15.981550  1709 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:15.982235  9679 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:15.983635  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:15.984867  9785 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:15.985260  9785 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.985589  9785 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.988593  9751 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.989768  9769 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:15.990370  9769 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:15.991291  9353 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:15.973065  9534 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.994227  9769 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.995424  9784 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.998194  9666 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.000792  9777 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.980088  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.002064  1706 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:04:16.002588  9758 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:15.931443  9646 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.930728  9664 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:16.019531  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:16.009806  9640 consensus_meta.cc:287] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.217s	user 0.008s	sys 0.008s
I20260709 13:04:16.010251  9778 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.011755  1457 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 11 to 15, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:15.980430  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.024824  1703 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.024887  2262 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.015388  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.016443  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.026854  1706 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:16.019043  1634 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:16.020370  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57942 (request call id 1706) took 1286 ms. Trace:
W20260709 13:04:15.950965  1601 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.022450  9562 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:15.983002  9679 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.029158  1702 raft_consensus.cc:2393] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:16.030092  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.028123  2289 rpcz_store.cc:276] 0709 13:04:14.734323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.110949 (+376626us) service_pool.cc:224] Handling call
0709 13:04:15.111296 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.188448 (+ 77152us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:15.188486 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.210312 (+ 21826us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.939743 (+729431us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:15.949463 (+  9720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.949613 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.019897 (+ 70284us) raft_consensus.cc:1685] finished
0709 13:04:16.019925 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.020307 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76687,"raft.queue_time_us":88507,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4366,"thread_start_us":663592,"threads_started":1,"wal-append.queue_time_us":776664}
I20260709 13:04:16.032387  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.039407  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.038892  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.029819  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:16.041420  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.046252  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.020255  9740 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:16.025051  9645 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.058187  2262 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.059899  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.033375  9469 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:16.062485  1457 catalog_manager.cc:5697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } }
I20260709 13:04:16.066103  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.064266  2289 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:04:16.037433  9572 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:16.066298  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 105470us
W20260709 13:04:16.073467  1675 leader_election.cc:336] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.055172  9786 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.068427  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 117493us
I20260709 13:04:16.035904  9775 raft_consensus.cc:2804] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:16.072139  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.075589  2291 raft_consensus.cc:2368] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28.
I20260709 13:04:16.075628  1703 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:16.049434  9751 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:16.052711  9778 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:16.077709  2296 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:16.077664  1675 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:16.076843  1601 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.078474  1598 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.084950  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:16.028810  9664 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.064019  9633 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.086587  1457 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 11 to 16, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 16 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:16.065963  9469 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:16.090056  9632 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:16.091311  1523 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:16.095321  1601 leader_election.cc:336] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:16.097172  2263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.049s	sys 0.046s
W20260709 13:04:16.097529  2263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.049s	sys 0.046s
I20260709 13:04:16.074950  9769 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:16.096701  1601 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:16.101776  9534 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.081187  9751 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.084086  9778 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.090090  9469 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:16.088119  9775 raft_consensus.cc:697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:16.090826  9777 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:04:16.104122  9769 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:16.067395  9645 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:16.093669  9675 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.286s	user 0.017s	sys 0.004s
I20260709 13:04:16.091732  9740 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:16.106666  2263 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.102336  9534 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.089227  9758 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:16.109522  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 185770us
W20260709 13:04:16.111485  9743 consensus_meta.cc:287] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.999s	user 0.009s	sys 0.010s
W20260709 13:04:16.114321  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.015s	sys 0.024s
I20260709 13:04:16.081688  9786 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.110461  9751 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.112583  9778 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:16.129072  2263 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.094339  9633 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.129359  9623 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.112533  9753 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:16.131353  9791 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.113720  9784 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.148938  9791 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.149222  9534 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:16.133953  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.016s	sys 0.025s
I20260709 13:04:16.131390  9277 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:04:16.130012  1457 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 13 to 17, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:16.138300  9746 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.465s	user 0.016s	sys 0.004s
I20260709 13:04:16.154172  1457 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 6 to 7, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 7 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.124886  9666 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:04:16.163543  1457 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:16.150985  9210 consensus_meta.cc:287] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 1.231s	user 0.027s	sys 0.036s
I20260709 13:04:16.135190  9758 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.157752  9784 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.136395  9743 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.154277  9633 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.157868  9646 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.156144  9534 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:16.157315  9753 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:16.151362  9798 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.192123  9675 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.204597  1524 leader_election.cc:400] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28.
I20260709 13:04:16.225414  1524 leader_election.cc:403] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.203807  2291 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:16.259057  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.260052  1703 raft_consensus.cc:3060] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:16.179100  9763 consensus_meta.cc:287] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 1.074s	user 0.038s	sys 0.086s
W20260709 13:04:16.281034  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 88778.4us
I20260709 13:04:16.289238  1703 raft_consensus.cc:2468] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
I20260709 13:04:16.290931  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:16.192260  9775 consensus_queue.cc:237] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:16.285713  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 190430us
I20260709 13:04:16.275017  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 2960) took 1076 ms. Trace:
W20260709 13:04:16.294548  1601 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.297984  1630 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:04:16.294293  1701 rpcz_store.cc:276] 0709 13:04:15.198959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.201990 (+  3031us) service_pool.cc:224] Handling call
0709 13:04:15.202350 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.946119 (+743769us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:15.946153 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.968480 (+ 22327us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.274431 (+305951us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:16.274494 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.274679 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.274749 (+    70us) raft_consensus.cc:1685] finished
0709 13:04:16.274768 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.274932 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":722465,"raft.queue_time_us":756375,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":305477,"threads_started":1,"wal-append.queue_time_us":291739}
W20260709 13:04:16.304199  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.309540  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.324018  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.337390  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.335477  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.338997  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.339329  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.339000  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.340508  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.341188  1627 raft_consensus.cc:2468] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:16.343612  1701 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:16.344583  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.345440  2289 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
I20260709 13:04:16.223058  9740 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:16.372812  2291 raft_consensus.cc:1275] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:16.199671  9679 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:16.349646  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 204732us
I20260709 13:04:16.288204  9305 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.342370  1601 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:16.381537  1601 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:16.382822  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.289055  9640 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:04:16.383108  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.305491  9806 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.165503  9746 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.389696  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.301337  9575 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:16.333611  9777 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:16.323510  9797 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:16.394830  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:16.397989  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.331740  9565 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:16.328145  9808 raft_consensus.cc:3060] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:04:16.223438  9646 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.280668  9534 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.223997  9784 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:16.392072  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 293332us
I20260709 13:04:16.175525  9769 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.403522  9575 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:16.415261  1675 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.415726  1675 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.416429  9575 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.406121  9565 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.425155  9565 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:16.323549  9681 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
W20260709 13:04:16.390401  9794 log.cc:927] Time spent T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:16.375802  9664 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.434866  9679 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:16.435353  1675 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.394186  9806 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:16.402722  1601 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.436339  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:16.407825  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 285713us
W20260709 13:04:16.437582  1601 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.438005  9806 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.438727  9633 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 13:04:16.440371  1601 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.412408  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:16.436429  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.289261  9785 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.443878  1675 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.300820  9675 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.332922  9210 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 13:04:16.461560  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.401850  9542 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.409075  1628 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:16.462847  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.464906  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.409593  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.447556  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.449290  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.451526  1706 raft_consensus.cc:1240] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:04:16.466926  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.394236  9777 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:16.466176  1702 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:16.305907  9763 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.407375  9764 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.479753  1634 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:16.474349  1675 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.476032  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.466066  1709 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:04:16.431759  9361 consensus_meta.cc:287] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.838s	user 0.013s	sys 0.000s
I20260709 13:04:16.392227  9305 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.478013  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.406862  9795 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.463466  1708 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:16.504599  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.467362  1706 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:04:16.483093  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.392659  9640 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.502723  2291 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
W20260709 13:04:16.473402  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 68421.8us
I20260709 13:04:16.488720  1705 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:16.385056  9740 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.494336  1631 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:04:16.495380  1627 raft_consensus.cc:1275] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:04:16.496958  9675 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.497511  9784 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.499444  9305 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:16.475796  9750 consensus_meta.cc:287] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 1.502s	user 0.035s	sys 0.027s
W20260709 13:04:16.500435  9800 log.cc:927] Time spent T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.464s	user 0.001s	sys 0.005s
I20260709 13:04:16.450233  9469 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.501660  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36804 (request call id 3358) took 1208 ms. Trace:
I20260709 13:04:16.526753  1630 rpcz_store.cc:276] 0709 13:04:15.293736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.294564 (+   828us) service_pool.cc:224] Handling call
0709 13:04:15.305317 (+ 10753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.298568 (+993251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.501362 (+202794us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54000cd854. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e4e01 00007f70ce9e383e 00007f70d39a8419 00007f70ccf1fbd4 00007f70ccf1fb62 00007f70ccf1fa91 00007f70ccf1fa0c 00007f70ccf1ecc2 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:16.501585 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282163456}
I20260709 13:04:16.495555  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.451995  9645 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.513015  2293 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:16.479025  9777 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.521060  9589 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.557367  9803 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 13:04:16.557469  9589 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:16.515748  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.560065  9640 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.509022  9664 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.560547  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44829.8us
I20260709 13:04:16.560866  9664 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.404027  9797 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.563228  9664 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.563390  9746 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.565097  9797 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.395229  9753 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.502729  9791 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.526116  9750 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.551416  9758 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.553671  9469 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:16.561324  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.593859  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:16.484328  9542 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.502393  9575 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:04:16.595148  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.599287  1675 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.510007  9764 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.599634  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:16.526870  9786 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.560074  9305 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:16.601264  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:16.604807  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.612243  9534 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.613910  1633 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 22: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:16.596588  9806 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.599908  9743 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.512141  9795 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.600685  9589 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:16.614204  9460 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:16.655120  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 252178us
W20260709 13:04:16.656730  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 146802us
I20260709 13:04:16.661135  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:16.661849  2293 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.659543  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44862 (request call id 3334) took 1663 ms. Trace:
I20260709 13:04:16.663934  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.664103  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.663389  1707 rpcz_store.cc:276] 0709 13:04:14.995954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.996096 (+   142us) service_pool.cc:224] Handling call
0709 13:04:15.034376 (+ 38280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.063872 (+ 29496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.063901 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.105323 (+ 41422us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.949077 (+843754us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:16.350105 (+401028us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.658824 (+308719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.658930 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.658992 (+    62us) raft_consensus.cc:1685] finished
0709 13:04:16.659011 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.659463 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8338,"prepare.queue_time_us":314548,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":49359,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":8418,"spinlock_wait_cycles":15744,"thread_start_us":1352601,"threads_started":3,"wal-append.queue_time_us":801198}
I20260709 13:04:16.672242  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.672341  9763 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:16.673918  9785 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.675581  9764 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.675920  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:16.638355  9305 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.676563  2297 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:16.677013  9778 consensus_meta.cc:287] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.000s
I20260709 13:04:16.677443  9778 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.677927  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.678580  2296 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:16.678630  9305 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.658803  9796 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.679718  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f8c2a4b167d431aac3de9d8166bab1f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.649672  9827 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.683063  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:16.683355  2296 raft_consensus.cc:1275] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 13:04:16.684082  2297 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:16.693080  9763 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:16.693548  9763 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.693898  9763 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.690747  9764 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:16.697176  9764 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.699128  9830 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.699237  9804 consensus_queue.cc:1048] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:16.699858  9830 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.701143  9764 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:16.701244  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.702088  9830 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.702713  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.704059  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.704471  9750 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.703998  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.704945  1705 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:16.705236  1632 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:16.706209  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.706959  9740 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.707432  9763 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.707476  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.690529  2293 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 21.
I20260709 13:04:16.709255  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.708789  9827 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:16.711138  2289 raft_consensus.cc:1275] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 13:04:16.711954  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.714211  1703 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:16.679709  9646 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.689013  9838 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:16.724159  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:16.725032  1631 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 22: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:16.725334  9763 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.717828  9795 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:16.715323  9827 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:16.729362  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.718050  9626 consensus_queue.cc:260] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.720173  9777 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.720726  9542 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:16.726821  1632 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:16.728534  1707 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:16.730489  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.730013  1700 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
W20260709 13:04:16.738781   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:16.733372  9646 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.738837  1706 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.736438  1673 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.745927  9646 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.747219  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.748008  9740 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:16.748461  9740 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:16.748826  9740 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:16.749285  9839 consensus_queue.cc:1048] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:16.733991  9827 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:16.744097  1599 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:16.753625  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 419544us
I20260709 13:04:16.759231  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.757635  1706 raft_consensus.cc:2468] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 21.
W20260709 13:04:16.761797  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45399.3us
W20260709 13:04:16.762324  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.765208  9777 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.748895  9646 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.768867  9804 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:16.769356  9804 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:16.769748  9804 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.772181  9778 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.772351  9632 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.773869  9751 consensus_meta.cc:287] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.002s
I20260709 13:04:16.774717  9798 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.776561  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.777271  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.783862  9572 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:16.783923  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.784567  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.784271  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.792977  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.794699  1707 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:16.796550  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:16.779630  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.785574  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.798883  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.800175  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.801430  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.802641  1706 raft_consensus.cc:2410] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:04:16.793838  1700 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:16.804840  9778 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:16.791333  9632 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.806969  9681 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:16.808699  9632 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:16.785024  9626 raft_consensus.cc:3060] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:16.787528  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:16.790201  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:16.802721  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.789485  9830 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:16.812764  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:16.828769  9562 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:16.829245  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.824968  9743 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:16.830097  9753 consensus_queue.cc:1048] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:04:16.831269  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.846074  2289 raft_consensus.cc:2468] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:16.832353  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:16.846545  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 59113.9us
I20260709 13:04:16.832548  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:16.828801  2295 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:04:16.834082  2296 raft_consensus.cc:1240] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:04:16.837175  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:16.837203  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:16.848721  2297 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:16.839277  9821 log.cc:927] Time spent T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.238s	user 0.000s	sys 0.003s
I20260709 13:04:16.839929  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.849224  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:16.853888  2260 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:16.853556  2292 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:16.855971  9542 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:16.858968  9838 raft_consensus.cc:2804] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:16.859336  9542 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.859833  9542 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:16.861497  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48821.1us
I20260709 13:04:16.862712  1523 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:16.864576  1598 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:16.792932  9751 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:16.863492  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:16.867588  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.881384  1525 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.868351  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:16.893137  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.897584  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.845506  9562 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.900115  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:16.900792  1703 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.902462  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.910198  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:16.916430  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:16.855988  9830 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.795049  9798 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.860948  9777 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.888379  9277 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.920046  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:16.968981  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.962891  2297 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:16.866545  9763 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:16.967741  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.908883  9786 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:04:16.976449  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53724.9us
I20260709 13:04:16.908697  9837 raft_consensus.cc:2749] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.868848  9796 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.860697  9778 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.113s
I20260709 13:04:16.978296  1703 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:16.867861  9565 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:04:16.980572  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.980350  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:16.868296  9838 raft_consensus.cc:697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:16.888018  9828 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:16.975746  9277 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.913511  9646 consensus_queue.cc:1059] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.080s
I20260709 13:04:16.860471  9623 consensus_queue.cc:1048] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.110s
I20260709 13:04:16.988381  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:16.995862  9361 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.910225  9562 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:17.017514  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.018414  1700 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:16.918262  9210 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.994549  9763 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.916198  9797 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:16.919420  9830 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.982252  9777 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:16.973377  9049 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:16.920499  9785 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:16.984045  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.044446  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.011545  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:16.988600  9803 log.cc:927] Time spent T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:16.985922  9786 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.016044  1630 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:16.988736  1709 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:04:16.888679  9791 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:16.968169  9784 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:04:17.005654  9701 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.027256  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 135411us
W20260709 13:04:16.920472  9833 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:17.013988  9277 raft_consensus.cc:3060] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.988858  9353 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:04:17.018728  9827 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.048544  1633 raft_consensus.cc:2393] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 29: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:17.011119  9589 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:04:17.029337  9763 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:17.031744  9830 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:17.063009  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.065698  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.049667  9786 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:17.014932  9828 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.003027  9796 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.111152  1707 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:17.066817  9701 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.047593  9049 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.161017  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.170011  2262 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:17.172907  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 110078us
I20260709 13:04:17.174357  1525 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.175850  1633 raft_consensus.cc:1275] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:04:17.176659  2262 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:17.159431  9847 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:17.069557  9785 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:17.167909  9845 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:04:17.181627  9769 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:17.187083  9764 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.230218  2291 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
I20260709 13:04:17.080334  9838 consensus_queue.cc:237] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.221632  1598 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:17.219990  9852 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.258898  9777 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:17.259969  9853 log.cc:927] Time spent T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:04:17.261937  9827 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:17.228648  9751 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:17.242714  9844 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.224s	user 0.000s	sys 0.004s
I20260709 13:04:17.240666  9798 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:17.275939  2295 raft_consensus.cc:1275] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:04:17.277515  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 357133us
I20260709 13:04:17.280715  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
W20260709 13:04:17.283747  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 164749us
I20260709 13:04:17.331570  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:17.333339  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.256549  9647 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:17.339262  1707 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.17-20.17]
W20260709 13:04:17.342238  9846 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 13:04:17.343096  9848 log.cc:927] Time spent T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:04:17.342890  9632 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.346401  9796 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.283260  9777 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:17.268177  9852 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.334431  9746 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.352571  9804 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.000s
I20260709 13:04:17.352936  9854 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:17.346906  9562 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:17.355872  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 70702.1us
W20260709 13:04:17.381438  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 207096us
W20260709 13:04:17.277160  9808 consensus_meta.cc:287] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.002s
I20260709 13:04:17.358574  2295 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:04:17.369725  9844 log.cc:927] Time spent T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:04:17.277228  9353 consensus_queue.cc:1059] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.097s
W20260709 13:04:17.374969  9542 consensus_meta.cc:287] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:04:17.356537  9647 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:17.361908  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.390363  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:17.387398  1525 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:17.385316  9740 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 13:04:17.399305  1631 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:17.362159  9469 consensus_queue.cc:1048] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 13:04:17.370724  9632 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.385581  9854 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.479758  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 89518.3us
I20260709 13:04:17.392779  9808 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 27. Current term is 28.
W20260709 13:04:17.449812  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 198774us
I20260709 13:04:17.383826  9746 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.491230  1598 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:17.491820  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:17.413806  9843 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
I20260709 13:04:17.396354  9049 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.395704  9647 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.479206  9353 raft_consensus.cc:3055] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:17.481443  9852 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.481776  9828 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:17.501710  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 114603us
I20260709 13:04:17.493263  9854 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:17.400580  9861 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.387032  9804 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.505008  9562 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.503136  9777 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:17.507576  9746 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.508587  9863 raft_consensus.cc:2804] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:17.516983  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.499969  1524 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:17.395538  9542 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.523145  9575 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:17.456871  9633 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 13:04:17.487823  9740 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.542683  1701 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
W20260709 13:04:17.525828  9856 log.cc:927] Time spent T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:04:17.543913  9575 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.546666  9575 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:17.526007  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:17.521139  9798 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 13:04:17.527192  9861 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.527884  9866 raft_consensus.cc:2804] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:17.528602  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:17.533919  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.509577  9851 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:17.497983  9859 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:17.503379  9353 raft_consensus.cc:740] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:17.541109  1708 raft_consensus.cc:1240] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:17.539835  1524 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:17.520305  9795 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:17.559303  9646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.017s
I20260709 13:04:17.553553  1525 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.551900  2294 raft_consensus.cc:2393] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:17.563117  2295 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 18.
I20260709 13:04:17.550076  9777 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:17.551160  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.593182  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:17.562639  9866 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.583964  9646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.006s	sys 0.017s
I20260709 13:04:17.562167  9863 raft_consensus.cc:697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:17.591710  9647 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.572728  9828 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:17.602897  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:17.593688  9852 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:17.610836  1628 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:17.610473  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.597764  9859 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:17.616791  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.617898  9860 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.596572  9633 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:17.602432  9740 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.623710  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.623942  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:17.603945  9866 raft_consensus.cc:3060] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:17.621124  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:04:17.652117  1629 raft_consensus.cc:2368] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22.
I20260709 13:04:17.593096  9795 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.627930  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.634799  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.627543  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.585842  9763 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.619872  9808 consensus_queue.cc:1059] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:04:17.650816  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:17.666172  1701 raft_consensus.cc:3060] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:17.624114  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.660641  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:17.606268  9777 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:17.635352  9542 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.627400  9646 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.627724  9873 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.647346  9626 consensus_meta.cc:287] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.002s
I20260709 13:04:17.669282  1633 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:17.625362  9852 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:17.671479  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:17.687916  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.685096  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.692153  9361 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.692624  9873 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.692795  9361 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.695056  9873 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.695225  9361 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.696663  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.698297  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:17.698905  1707 raft_consensus.cc:3060] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:17.699870  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:17.700937  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:17.692014  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:17.701522  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.704476  1706 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:04:17.702172  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:17.706914  1633 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:17.708915  1704 raft_consensus.cc:2393] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 23: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:04:17.701025  1675 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
I20260709 13:04:17.626418  9828 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.672236  9769 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:17.692644  9867 raft_consensus.cc:2804] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:17.652482  9633 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:17.722527  9277 consensus_meta.cc:287] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.002s
I20260709 13:04:17.704521  9740 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.736421  1634 raft_consensus.cc:2468] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 27.
I20260709 13:04:17.737360  1707 raft_consensus.cc:2468] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:17.739404  9633 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.742211  9828 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:17.742923   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:17.744987  9804 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.750995  1704 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:17.750445  9777 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.753654  9763 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:17.754577  9805 log.cc:927] Time spent T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.532s	user 0.003s	sys 0.000s
I20260709 13:04:17.754359  9852 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.754881  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:17.755841  9854 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.686576  9646 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:17.724820  9863 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.728919  9769 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:17.766300  9830 consensus_meta.cc:287] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.004s
I20260709 13:04:17.776890  1701 raft_consensus.cc:2468] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:17.653825  9353 consensus_queue.cc:260] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.728842  9867 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:17.786141  1700 raft_consensus.cc:3060] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:17.734380  9277 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.801185  9785 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:17.801576  9867 raft_consensus.cc:3060] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:17.805877  9353 raft_consensus.cc:3060] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:17.816972  9786 consensus_meta.cc:287] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.000s
I20260709 13:04:17.800915  9830 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:17.825454  9778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.012s	sys 0.000s
W20260709 13:04:17.829061  9778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.013s	sys 0.000s
I20260709 13:04:17.843917  9623 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:17.844780  9623 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.850445  9769 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:17.873493  9880 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:04:17.828094  9786 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.878131  1635 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:17.902148  9786 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.847522  9623 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:17.941634  9866 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:17.975091  1700 raft_consensus.cc:2468] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 25.
W20260709 13:04:17.988078  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 230102us
I20260709 13:04:17.974952  9804 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:17.997766  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.000564  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:17.980777  9796 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
W20260709 13:04:18.002265  2263 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.012286  2263 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.007359  9769 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:18.014490  9664 raft_consensus.cc:3055] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 LEADER]: Stepping down as leader of term 28
W20260709 13:04:18.022886  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 321865us
I20260709 13:04:17.997766  9797 consensus_queue.cc:1059] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:04:18.014149  9830 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:18.024417  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 312698us
I20260709 13:04:18.002211  9804 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.015347  9866 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.023275  9664 raft_consensus.cc:740] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:18.056341  9664 consensus_queue.cc:260] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.057740  9664 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:04:18.035135  9786 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.059784  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.060161  9786 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.061206  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.062471  9786 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.064097  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.064875  1707 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:18.066994  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.068207  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:18.069145  1703 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:18.069085  9861 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.072728  9859 raft_consensus.cc:3055] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:18.073113  9795 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.073298  9859 raft_consensus.cc:740] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:18.030885  9277 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.074410  9859 consensus_queue.cc:260] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.063657  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.077607  9854 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.078116  1704 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:18.035948  9632 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.009459  9796 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.080359  9791 raft_consensus.cc:3055] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:18.080835  9791 raft_consensus.cc:740] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:18.081063  9796 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.043411  9804 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:18.082669  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:18.033052  9750 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:04:18.040634  9701 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.060696  1700 raft_consensus.cc:2410] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
W20260709 13:04:18.034207  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.083653  1706 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:18.085014  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.086062  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 51876.9us
W20260709 13:04:18.086846  1675 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.087307  1675 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.088680  1675 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.089085  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.090704  1675 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.092051  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.047086  9867 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.100257  9839 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.101155  9804 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.075533  9859 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:18.102442  9743 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.103484  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.104147  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.105010  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:18.106671  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.129760  1634 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:18.130916  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.107250  9763 consensus_queue.cc:1048] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:18.131596  2260 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.129088  1629 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:18.132913  1599 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.108379  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.109257  9646 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.081871  9791 consensus_queue.cc:260] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.111749  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:18.116595  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.054854  1598 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:18.117005  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:18.138139  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 83358.5us
W20260709 13:04:18.118943  1675 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.133894  2260 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:18.140496  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:18.134985  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.135063  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.144193  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.144560  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.146237  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:18.147454  9753 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:18.147984  9753 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.148361  9753 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.149183  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.151386  1635 raft_consensus.cc:1240] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:18.152734  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42818 (request call id 1373) took 1232 ms (client timeout 1509 ms). Trace:
I20260709 13:04:18.153319  1627 raft_consensus.cc:2393] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:18.137619  1523 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:18.156101  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.156189  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:18.153143  1631 rpcz_store.cc:269] 0709 13:04:16.920204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.920396 (+   192us) service_pool.cc:224] Handling call
0709 13:04:18.152544 (+1232148us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540014f0d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:18.152681 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911175040}
I20260709 13:04:18.159118  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.160544  2290 raft_consensus.cc:1240] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:04:18.114354  9866 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.163703  1635 raft_consensus.cc:1240] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:04:18.163915  1634 raft_consensus.cc:1240] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:18.148276  9791 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:18.173455  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 94302.9us
I20260709 13:04:18.171025  1523 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:18.169198  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.175403  1634 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:04:18.175841  1633 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 25: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:18.170770  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:18.175930  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.166191  2295 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:04:18.173873  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55105.8us
I20260709 13:04:18.167380  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:18.167668  1525 leader_election.cc:400] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22.
I20260709 13:04:18.177678  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:18.177970  1675 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.152176  9534 raft_consensus.cc:2804] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:18.152333  9854 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.178483  1675 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.178395  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.157502  9542 raft_consensus.cc:2804] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:18.182367  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:18.156795  9786 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.184593  2292 raft_consensus.cc:2393] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 19: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
W20260709 13:04:18.181385  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:18.136521  9828 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
W20260709 13:04:18.187566  1601 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.190707  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.188814  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:18.191826  1601 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.192139  1703 raft_consensus.cc:1240] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Rejecting Update request from peer 8b2c3934407d422495728f84374c6b53 for earlier term 26. Current term is 27. Ops: []
I20260709 13:04:18.193110  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.136186  9861 consensus_queue.cc:1059] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:04:18.194731  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:18.171360  9646 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:18.197458  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.213546  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.197894  9534 raft_consensus.cc:697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:18.216511  9534 consensus_queue.cc:237] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.195912  9804 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.220245  2295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.188259  9864 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.221369  9305 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.197996  9681 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:18.224331  9890 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:04:18.224754  9769 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.165302  9889 raft_consensus.cc:2804] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:18.198894  9542 raft_consensus.cc:697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:18.225745  9889 raft_consensus.cc:697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:18.226452  9542 consensus_queue.cc:237] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.226646  9889 consensus_queue.cc:237] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.189044  9854 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.173125  9777 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.207342  9796 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:18.175185  9778 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:18.213052  9647 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.229864  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:18.196893  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 53984.1us
W20260709 13:04:18.244910  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66649.5us
I20260709 13:04:18.246330  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.235105  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.249493  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.252102  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.252818  1703 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 22.
I20260709 13:04:18.253913  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.193150  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.256603  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:18.256978  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 63853.7us
W20260709 13:04:18.258351  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:18.248867  2263 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.229717  9854 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.232182  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:18.283464  1708 raft_consensus.cc:2393] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 25: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:04:18.269480  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.284281  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:18.285372  1705 raft_consensus.cc:1275] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:04:18.286219  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:18.290601  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55543.5us
I20260709 13:04:18.292023  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.292846  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.291386  2260 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.297845  1601 leader_election.cc:336] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.244624  9778 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.298653  1601 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:18.305510  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.305634  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.299750  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.247189  9796 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:18.311048  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:18.256140  9797 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:18.315476  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 69238us
I20260709 13:04:18.280978  9808 raft_consensus.cc:3055] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:18.316371  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.291451  2263 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.330596  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.331970  2263 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.331895  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.332677  2263 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:18.293159  9743 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:18.363513  9632 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.330755  9812 consensus_queue.cc:1059] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:04:18.365389  9812 raft_consensus.cc:3055] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:04:18.365772  9812 raft_consensus.cc:740] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:18.334699  9746 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.366578  9812 consensus_queue.cc:260] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.367184  9746 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.369369  9746 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.297122  9589 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.306635  9763 raft_consensus.cc:2749] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:18.344166  9575 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:18.330184  9808 raft_consensus.cc:740] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:18.372831  9812 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:18.306479  9647 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.373674  9647 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:18.374161  9647 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:18.350744  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 61187.9us
I20260709 13:04:18.307299  9778 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:18.298516  9798 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:04:18.333822  9861 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:18.289000  9769 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:04:18.357932  1525 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.358018  9646 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:18.358843  9828 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.331003  9786 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.336793  9565 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:18.372469  9808 consensus_queue.cc:260] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.420017  9808 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:18.220286  9804 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.345791  9867 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:18.400522  1675 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.310922  9796 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:18.407938  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 73512.1us
I20260709 13:04:18.417706  9855 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:18.424790  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 66938.3us
I20260709 13:04:18.443264  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.444252  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.450734  9804 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:18.454653  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 54193us
W20260709 13:04:18.455624  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:18.459870  1675 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.460522  1675 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.462486  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.463807  1675 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.466691  1525 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:18.473528  9740 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:18.516173  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.516964  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.521052  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.485610  9864 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.486625  9791 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:18.549127  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 82534.3us
I20260709 13:04:18.549772  1598 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.524448  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.554842  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.560722  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.562592  1525 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.565645  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 292) took 57722 ms (client timeout 15000 ms). Trace:
I20260709 13:04:18.569240  2288 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:18.573333  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.588613  2293 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:04:18.592571  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.593367  1675 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.503738  9859 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:04:18.559136  9645 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:18.559412  9784 raft_consensus.cc:2804] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:18.611989  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:18.573812  9632 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:18.570334  9623 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.590147  9854 consensus_queue.cc:1059] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:04:18.595371  9851 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.636194  9753 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.668076  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:18.639091  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:18.580459  9855 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.651677  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.573509  9884 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:18.657014  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.626055  9647 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.600518  9791 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.664443  1704 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:18.600090  9864 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:18.649355  9777 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
W20260709 13:04:18.665653  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 383082us
I20260709 13:04:18.711372  1673 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.677470  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.617733  9645 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.713605  9645 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:18.690662  2288 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:18.618121  9784 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:18.703383  1627 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:18.716917  9784 raft_consensus.cc:3060] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:18.706761  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 55185.3us
I20260709 13:04:18.718843  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:18.702015  9791 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.722636  9791 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.723234  9791 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.672803  1705 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:04:18.566463  1457 rpcz_store.cc:269] 0709 13:03:20.849454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.869597 (+ 20143us) service_pool.cc:224] Handling call
0709 13:03:21.000297 (+130700us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94e090 00007f70ce9ecf3a 00007f70d47a31cc 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc 00007f70d2d61392 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:18.565508 (+57565211us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:16.887615 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:16.888420 (+   805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:16.888489 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:04:16.888507 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:16.889809 (+  1302us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:16.889918 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:16.889943 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:04:17.120741 (+230798us) tablet.cc:701] Row locks acquired
0709 13:04:17.120818 (+    77us) write_op.cc:260] PREPARE: finished
0709 13:04:17.121018 (+   200us) write_op.cc:270] Start()
0709 13:04:17.121203 (+   185us) write_op.cc:276] Timestamp: P: 1783602257120939 usec, L: 0
0709 13:04:17.121251 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:04:17.171469 (+ 50218us) log.cc:844] Serialized 5194 byte log entry
0709 13:04:17.619290 (+447821us) op_driver.cc:464] REPLICATION: finished
0709 13:04:17.678943 (+ 59653us) write_op.cc:301] APPLY: starting
0709 13:04:17.679071 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:17.805174 (+126103us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:17.807599 (+  2425us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:17.926481 (+118882us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:17.926951 (+   470us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:04:17.927038 (+    87us) write_op.cc:312] APPLY: finished
0709 13:04:17.927598 (+   560us) log.cc:844] Serialized 121 byte log entry
0709 13:04:17.928568 (+   970us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:17.928762 (+   194us) write_op.cc:454] Released schema lock
0709 13:04:17.928851 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:17.997553 (+ 68702us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:17.999786 (+  2233us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39790,"reported_tablets":48,"spinlock_wait_cycles":23192960,"child_traces":[["op",{"apply.queue_time_us":59343,"apply.run_cpu_time_us":28143,"apply.run_wall_time_us":287442,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":207652,"prepare.run_cpu_time_us":20909,"prepare.run_wall_time_us":504002,"raft.queue_time_us":87572,"raft.run_cpu_time_us":18199,"raft.run_wall_time_us":59641,"replication_time_us":497847,"thread_start_us":430951,"threads_started":4,"wal-append.queue_time_us":310580}]]}
I20260709 13:04:18.696513  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:18.726055  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 394083us
W20260709 13:04:18.726910  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 69933.1us
I20260709 13:04:18.730330  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:18.731295  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.732028  2263 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.643096  9866 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.061s
I20260709 13:04:18.692682  9828 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.724134  1451 catalog_manager.cc:5697] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:18.662067  9769 consensus_queue.cc:1048] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 13:04:18.711838  9632 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:18.766052  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 54789.1us
I20260709 13:04:18.738339  9864 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:18.763058  1451 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 11 to 24, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:18.778671   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 1585)
W20260709 13:04:18.747642  2263 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.781795  2263 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.785830  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.778752  1451 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 18, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:18.755409  9534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.093s	sys 0.100s
I20260709 13:04:18.761008  9542 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.725597  9777 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.815145  9565 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:04:18.799151  9534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.094s	sys 0.100s
I20260709 13:04:18.801440  1451 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 21 to 27, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:18.821254  9895 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:04:18.821421  1451 catalog_manager.cc:5697] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:18.824865  9534 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:04:18.826313  9893 log.cc:927] Time spent T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:04:18.837970  9861 raft_consensus.cc:3055] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:18.838595  9861 raft_consensus.cc:740] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:18.839744  9861 consensus_queue.cc:260] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.842835  9861 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:18.844353  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 96763.4us
W20260709 13:04:18.845041  2263 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.746095  9854 consensus_queue.cc:1059] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:04:18.822932  1451 catalog_manager.cc:5697] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 19, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:18.845453  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:18.864524  2263 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.865830  2263 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.850483  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.869089  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.761991  9828 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.888159  9778 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.892360  9906 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.892688  9778 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:18.887455  2263 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.901611  9777 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.903095  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:18.904011  2294 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 17.
I20260709 13:04:18.905761  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:18.870848  9565 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:18.840327  9753 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:18.799903  9859 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.085s
W20260709 13:04:18.910156  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44360.4us
I20260709 13:04:18.910574  9859 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:18.914152  9812 consensus_meta.cc:287] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 13:04:18.914211  9859 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:18.915021  9859 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:18.917045  2263 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.921449  2263 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.927590  1601 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.928406  1601 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.929554  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.931155  1601 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.947978  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:18.940076  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:18.968179  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.933517  9812 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:18.935472  9565 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:18.948205  1601 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.978559  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:18.938503  1631 raft_consensus.cc:1240] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:18.949653  9892 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.216s	user 0.004s	sys 0.000s
I20260709 13:04:18.911595  9784 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:18.934104  1451 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 21 to 22, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:18.979285  1601 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.982195  1451 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 25, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:18.973498  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
W20260709 13:04:18.983649  1601 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.971174  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:18.987655  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 400619us
I20260709 13:04:18.937151  9645 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:18.993197  1630 raft_consensus.cc:1240] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:04:18.993532  9645 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:18.988677  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.995004  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:18.995884  2297 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:18.995996  1601 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:18.996546  1601 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:18.997295  9645 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:18.998590  1601 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:18.999755  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:19.003957  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:19.000473  1700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.006019  1451 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.008113  1451 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 23 to 28, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 28 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.008967  2297 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:19.014729  1709 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:18.976720  9589 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:19.018996  1451 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 26. New cstate: current_term: 26 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.016474  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.027855  1451 catalog_manager.cc:5697] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 20 to 23, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.031911  1599 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:19.030498  1601 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.017352  9795 raft_consensus.cc:3055] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:19.040560  1601 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.005867  9798 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:19.043382  1451 catalog_manager.cc:5697] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.046083  9623 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:19.041795  9903 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.047127  9909 log.cc:927] Time spent T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:19.047261  9903 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.030099  9855 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:04:19.050957  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.051671  1601 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.060286  1599 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:19.055490  9910 log.cc:927] Time spent T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.002s
I20260709 13:04:19.047375  9784 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.045435  1451 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 21 to 26. New cstate: current_term: 26 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.038846  9645 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:04:19.042235  9795 raft_consensus.cc:740] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:19.055508  9864 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:19.056085  9623 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:04:19.063210  9796 consensus_meta.cc:287] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.006s
I20260709 13:04:19.069644  1451 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.063113  9863 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.078830  9888 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:19.079586  9888 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:19.080638  9888 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.078312  1451 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 21 to 23, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:19.083678  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:19.085096  1601 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.083590  1451 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:19.086794  1601 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.088790  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.089720  9863 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:19.090225  1601 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.090636  9534 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:19.090783  1601 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.091579  1631 raft_consensus.cc:1240] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Rejecting Update request from peer 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:04:19.091872  9786 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.097374  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.100764  9786 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:19.102536  1601 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.110355  1601 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.112085  9647 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:19.113224  9859 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.108577  1451 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 18, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.116343  1451 catalog_manager.cc:5697] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 9 to 20, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.120688  9828 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:19.113706  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.121800  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.122903  9645 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.124285  9903 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:19.124719  1601 leader_election.cc:336] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.148321  1601 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.149492  9647 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:19.151543  1601 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.134130  9795 consensus_queue.cc:260] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.137822  9791 raft_consensus.cc:3055] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:19.152819  1601 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.153368  9791 raft_consensus.cc:740] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:19.154330  9791 consensus_queue.cc:260] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.155428  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:19.156042  9645 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:19.141956  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:19.143596  1525 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.160486  9664 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:19.163975  1601 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.125349  9796 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.120648  1451 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 22, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.165526  1451 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 22, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:19.155519  9791 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.174917  9867 raft_consensus.cc:3055] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:19.175487  9867 raft_consensus.cc:740] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:19.164672  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.212563  1630 raft_consensus.cc:1240] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Rejecting Update request from peer 444b494bb53842b59408a7230884398a for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:19.212721  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 48787.4us
I20260709 13:04:19.213723  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:19.214380  1629 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:19.177441  9796 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:19.230685  1601 leader_election.cc:336] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.231212  1601 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.229063  1598 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:19.224650  9750 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:04:19.274879  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57934 (request call id 1845) took 1153 ms. Trace:
I20260709 13:04:19.218578  9786 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:04:19.275462  2291 rpcz_store.cc:276] 0709 13:04:18.121542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.121715 (+   173us) service_pool.cc:224] Handling call
0709 13:04:18.152464 (+ 30749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.152654 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:04:18.605252 (+452598us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.274436 (+669184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.274497 (+    61us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:04:19.274555 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.274653 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.274806 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67013,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":31935,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":1971,"thread_start_us":1120039,"threads_started":3,"wal-append.queue_time_us":668896}
W20260709 13:04:19.275862  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:04:19.280344  2289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57952 (request call id 1514) took 1133 ms. Trace:
I20260709 13:04:19.268646  9645 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:19.274634  9746 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:19.276042  9542 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:19.299249  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 450764us
I20260709 13:04:19.234028  9867 consensus_queue.cc:260] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.282492  9859 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Requested vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:19.295751  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 279391us
I20260709 13:04:19.298447  2289 rpcz_store.cc:276] 0709 13:04:18.146454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.154373 (+  7919us) service_pool.cc:224] Handling call
0709 13:04:18.154655 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.154866 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:04:18.657961 (+503095us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.279970 (+622009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.280043 (+    73us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:19.280102 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.280174 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.280287 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133895,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":6377,"raft.queue_time_us":28380,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":3181,"thread_start_us":1108797,"threads_started":3,"wal-append.queue_time_us":577876}
I20260709 13:04:19.316375  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:19.315478  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:19.308676  9796 consensus_queue.cc:1059] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:04:19.303210  9664 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.332208  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:19.332931  2297 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.337498  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.341361  1701 raft_consensus.cc:2468] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:19.335670  9664 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.348628  9664 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:19.359881  2297 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 18.
I20260709 13:04:19.353050  9861 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.349749  9812 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:19.339902  9924 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:19.397341  2291 raft_consensus.cc:1240] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:04:19.399183  2290 raft_consensus.cc:1240] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Rejecting Update request from peer 00f3fbc60b884078a3c219909b44f64c for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:19.432214  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.437963  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.440899  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.441865  1705 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:19.441030  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.442747  1706 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:19.443516  1629 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:19.447207  1525 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:19.448066  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 120774us
I20260709 13:04:19.451999  1627 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:19.458455  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.471644  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.456688  9852 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:19.485634  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44866 (request call id 3019) took 1191 ms. Trace:
I20260709 13:04:19.484928  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.428817  9861 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.469092  9864 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:19.471319  9854 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:19.491361  1708 rpcz_store.cc:276] 0709 13:04:18.293941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.294938 (+   997us) service_pool.cc:224] Handling call
0709 13:04:18.295384 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:18.772816 (+477432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.772866 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:18.773487 (+   621us) log.cc:844] Serialized 52 byte log entry
0709 13:04:19.059998 (+286511us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:19.483110 (+423112us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.485182 (+  2072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.485290 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.485353 (+    63us) raft_consensus.cc:1685] finished
0709 13:04:19.485374 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.485553 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35296,"prepare.queue_time_us":320863,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":76730,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"thread_start_us":1104382,"threads_started":3,"wal-append.queue_time_us":489598}
I20260709 13:04:19.467338  9812 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:19.500836  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:19.513201  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.449788  9924 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.476552  9305 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:04:19.494861  9907 consensus_queue.cc:1059] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:04:19.540683  9763 consensus_queue.cc:1059] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:04:19.510793  9854 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.558063  1627 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:19.559275  9854 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:19.560708  9922 log.cc:927] Time spent T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:04:19.563236  9861 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:19.564358  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.546746  9839 consensus_queue.cc:1059] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:19.518465  9812 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:19.524513  9919 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:04:19.505050  9791 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.584005  9791 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:19.586328  9791 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:19.523979  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:19.590579  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 67308us
I20260709 13:04:19.593283  9867 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.564605  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.495886  9852 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
W20260709 13:04:19.602749  9917 log.cc:927] Time spent T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:04:19.602564  9852 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.610102  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.611085  1708 raft_consensus.cc:2393] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 24: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:19.613006  1708 raft_consensus.cc:1275] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:19.546180  9305 raft_consensus.cc:3055] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:19.618886  9305 raft_consensus.cc:740] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:19.619521  9565 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:19.619915  9305 consensus_queue.cc:260] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:19.570191  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46932.9us
I20260709 13:04:19.525409  1524 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:19.581831  9927 raft_consensus.cc:3055] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:04:19.622128  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 96803.4us
I20260709 13:04:19.622397  9927 raft_consensus.cc:740] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:19.624729  9854 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:19.625303  9854 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:19.626188  9854 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.580883  1635 raft_consensus.cc:1275] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:19.623413  9927 consensus_queue.cc:260] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.581831  9812 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.620119  9565 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.628598  9927 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.630370  9791 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.630934  9791 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.633375  9791 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.621424  9305 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.621526  9795 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.572790  1629 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 29.
W20260709 13:04:19.643365  2263 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:19.643821  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:19.644022  1523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.644093  2263 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.645998  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.647238  9798 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:19.648702  9905 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.651628  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.652346  1626 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
W20260709 13:04:19.653527  9913 log.cc:927] Time spent T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:04:19.654289  1525 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:19.656999  9861 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:19.657496  9861 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:19.657998  9924 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:19.658357  9861 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.659410  9565 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.659750  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:19.661710  1675 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:19.671160  9664 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:04:19.671604  9664 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.671972  9664 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:04:19.673612  9798 consensus_queue.cc:1048] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:19.675089  1702 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:04:19.675465  1702 pending_rounds.cc:85] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Aborting all ops after (but not including) 14
I20260709 13:04:19.675679  1702 pending_rounds.cc:107] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:04:19.675930  1702 raft_consensus.cc:2889] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.679075  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.679842  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.680507  1627 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
I20260709 13:04:19.681248  9905 consensus_queue.cc:1059] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:19.682391  9921 raft_consensus.cc:3055] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:19.682772  9921 raft_consensus.cc:740] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:04:19.686446  2263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.664772  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.665548  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.662711  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:19.692948  1706 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 26.
I20260709 13:04:19.694911  1705 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:19.705225  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.711438  1524 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.714710  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:19.718225  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.702158  9791 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.727295  9808 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:19.722007  9884 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:19.743104  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 233764us
I20260709 13:04:19.744450  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:19.740700  1601 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:19.748332  1601 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:19.683614  9921 consensus_queue.cc:260] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.734695  9808 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:19.779507   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:19.767367  9746 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:19.730957  9785 consensus_queue.cc:1048] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:04:19.753300  9861 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:04:19.824962  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.825650  1703 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
W20260709 13:04:19.825706  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58536.1us
I20260709 13:04:19.766762  9808 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:19.824329  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 55444.9us
W20260709 13:04:19.819852  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:19.828541  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.830312  1705 raft_consensus.cc:2393] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 27: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:19.832151  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.835371  9777 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.838011  9777 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.841984  9921 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:19.835402  9936 log.cc:927] Time spent T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
I20260709 13:04:19.848964  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58846 (request call id 1450) took 1002 ms. Trace:
I20260709 13:04:19.849673  1704 rpcz_store.cc:276] 0709 13:04:18.846925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.847505 (+   580us) service_pool.cc:224] Handling call
0709 13:04:18.847931 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:18.848421 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:19.166081 (+317660us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b480024ce58. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d390daa1 00007f70d3904b37 00007f70d39049ca 00007f70d39fc26f 00007f70d39fbf37 00007f70d3a1055c 00007f70d3a10508 00007f70d3a10488 00007f70d3a10443 00007f70d3a0f6ec 00007f70d2939a96
0709 13:04:19.300167 (+134086us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.722988 (+422821us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.723320 (+   332us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:19.730927 (+  7607us) log.cc:844] Serialized 52 byte log entry
0709 13:04:19.731202 (+   275us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:19.735075 (+  3873us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.735332 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.746471 (+ 11139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.773096 (+ 26625us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b18002b1380. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70d47bfde1 00007f70d47cc5b1 00007f70cb8f73f3 00007f70cb8f97c1 00007f70cb8f9779 00007f70cb8f9709 00007f70cb8f96d1 00007f70cb8f89cd 000000000038f354 000000000038f199 00007f70cb8da83a
0709 13:04:19.848674 (+ 75578us) raft_consensus.cc:1685] finished
0709 13:04:19.848701 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.848891 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23358,"prepare.queue_time_us":518686,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":4255,"raft.queue_time_us":44144,"raft.run_cpu_time_us":3268,"raft.run_wall_time_us":295515,"spinlock_wait_cycles":15211008,"thread_start_us":762461,"threads_started":2,"wal-append.queue_time_us":423139}
I20260709 13:04:19.853130  9777 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.811748  9664 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.830581  9884 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.860493  9884 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.863144  9884 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:19.866287  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:19.867128  1627 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:04:19.871459  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:19.874423  9664 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 30 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:19.875488  1627 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 30.
I20260709 13:04:19.877562  1525 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:19.877596  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 133288us
I20260709 13:04:19.897424  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.899935  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:19.892818  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 66264.9us
I20260709 13:04:19.900780  2297 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:04:19.905090  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.906198  1705 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:19.906332  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:19.913327  1523 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:19.917608  2297 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 30.
I20260709 13:04:19.892359  9884 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:04:19.920404  2291 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:19.919891  9867 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:19.932904  9867 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.884274  9808 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:19.938535  9808 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:19.938792  9861 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:19.922250  1523 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a; no voters: 
I20260709 13:04:19.903864  9859 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.942651  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:19.935534  9867 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:19.962026  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:19.965488  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:19.962498  9791 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:19.969233  9469 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:04:19.977589  9915 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:19.992209  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:19.993074  1635 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.000710  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21235338303a443e8f10b8773c68e2db" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.001339  9808 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.001987  9808 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.004251  9808 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.005597  9854 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.006685  9854 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:19.964533  9884 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 30 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:20.011833  1635 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:20.006027  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.013134  1525 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.014935  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.022982  2262 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:20.027359  9565 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:20.028208  9565 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:20.028635  9565 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:20.028990  9565 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:19.961422  9859 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.036218  9746 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:19.972548  9867 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:20.037096  1629 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:20.037716  9884 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:20.038448  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:19.973857  9664 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.973517  9791 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.089318  9854 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.093492  9864 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.094215  9864 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.097967  9864 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.102648  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.103448  1629 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:20.061847  9852 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.105268  1525 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.058220  9808 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:20.106299  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:20.124742  1707 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.111999  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2139fc31dcac45ddbcc3f5046bd79ade" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.116324  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.067680  9791 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:20.139693  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.141644  1629 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 27.
I20260709 13:04:20.084662  9951 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.159477  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 107715us
W20260709 13:04:20.156399  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 87255.1us
I20260709 13:04:20.185452  1707 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:20.068796  9664 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.159260  9830 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.215844  9854 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:20.215708  9830 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.216341  9854 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.216732  9854 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.218106  9830 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:20.191219  9937 log.cc:927] Time spent T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:20.190224  9859 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.173904  9951 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.224288  9951 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.140792  9852 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.226434  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.227110  2297 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:20.227718  9852 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.228771  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.194975  9864 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:20.241792  9664 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.246444  9701 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:04:20.307896  9960 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:04:20.307426  9701 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.248930  9864 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.307695  9763 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.320371  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.321205  1627 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 21.
W20260709 13:04:20.321846  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 132874us
I20260709 13:04:20.323952  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:20.325270  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 240877us
I20260709 13:04:20.325814  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.326876  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.332190  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.328311  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.332912  2289 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:20.318687  9701 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.334687  2290 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:04:20.339232  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 143043us
I20260709 13:04:20.344754  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.340054  9784 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:20.356576  1523 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.322887  9763 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.343226  9907 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.369596  1524 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.354259  9784 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:20.422593  9961 log.cc:927] Time spent T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:04:20.420202  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63733.5us
I20260709 13:04:20.361045  9918 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:04:20.427121  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 57612.9us
I20260709 13:04:20.349608  9565 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:20.421267  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 58413.3us
W20260709 13:04:20.420881  9945 log.cc:927] Time spent T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:20.412446  9884 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.432482  1627 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 13:04:20.420543  9796 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:20.438026  9796 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:20.433764  1629 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:20.447865  9796 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.448477  9796 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.449301  1706 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:20.449818  9864 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:20.426433  9918 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.453135  9918 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:20.451740  9852 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:04:20.453896  9763 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.370183  9907 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.458330  9907 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:20.458441  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.423230  9784 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.460373  1626 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 13:04:20.462354  9951 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 13:04:20.432590  9974 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:20.465260  9565 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.465945  2291 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 27.
I20260709 13:04:20.466035  9884 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.452651  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.450558  9796 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.469161  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.469936  1630 raft_consensus.cc:2393] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 21: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:20.470685  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.471485  1706 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:04:20.472604  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.473371  1707 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 20.
I20260709 13:04:20.467545  1629 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:20.476500  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.479005  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.496622  9861 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:20.501610  9861 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.502015  9861 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:20.509357  9791 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.518743  9791 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.511613  9854 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.519698  9701 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.559293  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:20.525502  9854 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.567029  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:20.569890  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 141916us
I20260709 13:04:20.567724  1630 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:20.524880  9784 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.523708  9918 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.571444  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.576529  1629 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:20.529157  9861 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.573472  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:20.575477  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.583364  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:20.583832  9791 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:04:20.593182  9918 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.596112  9973 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.596776  9973 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.599412  9973 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.621326  9861 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.623327  1709 raft_consensus.cc:2368] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 27. Current term is 28.
W20260709 13:04:20.638773  9960 log.cc:927] Time spent T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:20.639948  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52261.2us
I20260709 13:04:20.641263  9981 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:20.641865  9981 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.642284  9981 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.647629  9884 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:20.672053  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 88760.8us
I20260709 13:04:20.676013  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.633257  1628 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 27.
I20260709 13:04:20.692543  9565 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:20.693025  9565 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.693406  9565 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:20.693753  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.694814  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:20.695446  2290 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.694955  9962 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.695942  9701 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.696044  9962 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.697230  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.698256  9962 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:20.699271  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.699909  1627 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:20.701999  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.702754  1635 raft_consensus.cc:2393] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 24: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:20.699936  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.704879  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:20.705164  2260 leader_election.cc:400] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 27. Current term is 28.
I20260709 13:04:20.705529  2260 leader_election.cc:403] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 13:04:20.706416  9769 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:20.702625  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.709995  1630 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:20.712342  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.713058  1635 raft_consensus.cc:3060] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:20.714423  9565 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.715700  2292 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:20.716687  9784 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.718701  9565 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:20.759202  2289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.719141  1627 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:20.720841  1635 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 28.
I20260709 13:04:20.721587  2290 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 24.
I20260709 13:04:20.722571  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:20.725272  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.767971  1630 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 23.
I20260709 13:04:20.735010  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:20.769116  1706 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:20.736318  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:20.770481  2292 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 444b494bb53842b59408a7230884398a in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
W20260709 13:04:20.772464  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:20.743724  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:20.748826  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.752506  1523 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:20.757599  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:20.773874  1628 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.738904  1709 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
W20260709 13:04:20.780292   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:20.786108  1628 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:20.794255  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:20.794926  1709 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.727013  9981 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:20.794325  1451 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 215) took 4348 ms. Trace:
I20260709 13:04:20.795979  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:20.797505  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45096.8us
I20260709 13:04:20.801930  1523 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:20.806881  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 138170us
I20260709 13:04:20.782047  9664 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.815917  9981 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:20.816911  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.800278  1451 rpcz_store.cc:276] 0709 13:04:16.428292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.428617 (+   325us) service_pool.cc:224] Handling call
0709 13:04:20.776634 (+4348017us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:19.567578 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:19.568192 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:19.568236 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:04:19.568274 (+    38us) tablet.cc:662] Decoding operations
0709 13:04:19.578124 (+  9850us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:19.578187 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:19.578215 (+    28us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:04:19.579752 (+  1537us) tablet.cc:701] Row locks acquired
0709 13:04:19.701923 (+122171us) write_op.cc:260] PREPARE: finished
0709 13:04:19.721053 (+ 19130us) write_op.cc:270] Start()
0709 13:04:19.728500 (+  7447us) write_op.cc:276] Timestamp: P: 1783602259715874 usec, L: 0
0709 13:04:19.730452 (+  1952us) op_driver.cc:348] REPLICATION: starting
0709 13:04:19.839905 (+109453us) log.cc:844] Serialized 8260 byte log entry
0709 13:04:20.337151 (+497246us) op_driver.cc:464] REPLICATION: finished
0709 13:04:20.434922 (+ 97771us) write_op.cc:301] APPLY: starting
0709 13:04:20.435003 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:20.447256 (+ 12253us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:20.447391 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:20.663436 (+216045us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:20.663789 (+   353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:04:20.663840 (+    51us) write_op.cc:312] APPLY: finished
0709 13:04:20.664281 (+   441us) log.cc:844] Serialized 188 byte log entry
0709 13:04:20.665129 (+   848us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:20.665336 (+   207us) write_op.cc:454] Released schema lock
0709 13:04:20.665401 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:20.670873 (+  5472us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:20.670918 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":24466,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":94672,"apply.run_cpu_time_us":28851,"apply.run_wall_time_us":235899,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91991,"num_ops":21,"prepare.queue_time_us":391499,"prepare.run_cpu_time_us":34087,"prepare.run_wall_time_us":396961,"raft.queue_time_us":190639,"raft.run_cpu_time_us":9380,"raft.run_wall_time_us":26779,"replication_time_us":606501,"spinlock_wait_cycles":90880,"thread_start_us":607431,"threads_started":4,"wal-append.queue_time_us":303445}]]}
I20260709 13:04:20.825664  9854 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:20.825587  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.827215  1525 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:20.845947  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 46509.4us
I20260709 13:04:20.860131  1524 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:20.851684  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 265915us
I20260709 13:04:20.781451  9984 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:20.858477  1627 raft_consensus.cc:2393] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 23: Already voted for candidate 444b494bb53842b59408a7230884398a in this term.
W20260709 13:04:20.861690  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 274860us
I20260709 13:04:20.866948  1706 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 25.
I20260709 13:04:20.861855  1674 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:20.882239  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 55117.3us
I20260709 13:04:20.883353  1525 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.883417  1709 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 22.
I20260709 13:04:20.883504  1524 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.884421  1525 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:20.809167  9777 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:20.882097  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:20.886543  1627 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.884876  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:20.781684  9784 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
W20260709 13:04:20.889873  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 64372.5us
I20260709 13:04:20.889221  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
W20260709 13:04:20.881338  9808 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.027s	sys 0.053s
I20260709 13:04:20.888548  9974 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:20.868917  9985 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:04:20.888787  9968 raft_consensus.cc:2804] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:20.860464  9664 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:20.894414  1454 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 13 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:20.949857  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 52046.6us
W20260709 13:04:20.952514  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 67702.3us
I20260709 13:04:20.931461  1627 raft_consensus.cc:2468] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:04:20.952740  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 68403us
I20260709 13:04:20.897411  9796 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:20.895196  9791 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:20.954149  9777 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:20.899793  9944 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:21.020056  9769 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021) with higher term. Message: Invalid argument: T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb for earlier term 27. Current term is 28.
W20260709 13:04:21.022176  9987 log.cc:927] Time spent T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:20.895203  9971 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:20.953779  9992 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:20.952337  9305 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.901932  9808 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.028s	sys 0.053s
I20260709 13:04:21.036142  9777 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 25 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:21.028831  9968 raft_consensus.cc:697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:21.038481  9859 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.036963  9864 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:20.891957  9854 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:21.052663  9808 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:21.001886  9861 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:04:21.121114  9664 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.054270  9305 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:21.124634  9993 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:04:21.128175  1454 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 10 to 15, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 15 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:21.095743  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 53753.4us
W20260709 13:04:21.051947  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:20.885416  9984 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:21.173062  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 122951us
I20260709 13:04:21.157406  2291 raft_consensus.cc:2468] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 28.
I20260709 13:04:21.051939  9971 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:21.040103  9944 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:21.183189  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 202466us
I20260709 13:04:21.185271  1454 catalog_manager.cc:5697] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 9 to 12, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 12 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:21.186048  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:21.184123  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:21.195287  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.158242  9974 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:21.183028  9852 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.204114  9974 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.202926  9968 consensus_queue.cc:237] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.201820  9808 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.204202  9984 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.220755  9784 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.172497  9859 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.220726  9664 raft_consensus.cc:493] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.172922  9864 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.191046  9854 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.232136  9589 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.232782  9589 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.235100  9589 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.239619  9664 raft_consensus.cc:515] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.204875  9971 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.246126  9305 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.261536  9784 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.199970  9777 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.285423  9944 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.245220  1454 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:21.227969  9984 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:21.222419  9852 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.248240  9664 leader_election.cc:290] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.292433  9791 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:21.297647  1454 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 11 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 14 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.299108  9784 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.295964  9984 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:21.338272  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 165686us
I20260709 13:04:21.345428  2262 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c
I20260709 13:04:21.359602  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:21.356297  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:21.366174  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.367192  2293 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
I20260709 13:04:21.373281  9305 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:21.373961  9305 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.376143  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.376870  1629 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
W20260709 13:04:21.349279  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:21.383258  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:21.386794  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:21.391297  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.391835  1675 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.397687  2293 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 13:04:21.397753  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.371495  9589 raft_consensus.cc:2804] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:21.399412  1702 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
W20260709 13:04:21.403615  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 375805us
I20260709 13:04:21.405607  1454 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:21.416605  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:21.426162  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.426680  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.433465  1454 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.435781  9968 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:21.436728  9305 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.475050  1673 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:21.490198  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 98981.6us
W20260709 13:04:21.492367  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:21.480700  1454 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 10 to 18, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 18 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:21.495415  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:04:21.480690  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 79623.5us
I20260709 13:04:21.404811  9791 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.499873  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:21.471935  9786 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.507676  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 90702.9us
I20260709 13:04:21.473726  9980 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.444108  9974 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.061s
I20260709 13:04:21.478832  9852 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.481019  9565 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.486796  9864 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:21.481019  9769 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:21.530872  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.526996  9664 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.526906  9980 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.536584  9664 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.538419  9980 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:21.538951  9664 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.524487  9305 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:21.542598  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.523916  9786 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.543737  9565 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:21.559381  9565 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.560497  9565 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:21.546383  9589 raft_consensus.cc:697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:21.561877  9589 consensus_queue.cc:237] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.564774 10013 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.565402 10013 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.568252 10013 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.541935  9852 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:21.543138  9305 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:21.536365  9769 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.650200  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.653802  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.655205  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.570757  9984 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.573611  9305 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:21.658742  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 159531us
I20260709 13:04:21.573863  9852 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.689375  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.690800  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.693326  1628 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 15.
I20260709 13:04:21.693616  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.695832  9786 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.710542  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.686687 10010 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:21.722632  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.727742  1525 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.729995  1629 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:21.729530  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:21.738826  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.739586  1627 raft_consensus.cc:2368] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:21.734614 10013 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:21.740998  9565 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.741570  9664 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:21.768518  9664 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.772614  9664 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.779898  9859 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.744849  9984 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.780625  9852 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.780951  9984 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.783186  9984 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:21.783612   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:21.731681 10010 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.794487  9974 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.795094  9974 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.797219  9974 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:21.804147  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.805861  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.808975  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:21.815603 10010 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:21.816840 10022 log.cc:927] Time spent T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.122s	user 0.003s	sys 0.000s
I20260709 13:04:21.817978  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.822386  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.824896  9769 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:21.825486  2294 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:21.830910  2291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57952 (request call id 1535) took 1138 ms. Trace:
W20260709 13:04:21.831266  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 351263us
I20260709 13:04:21.831382  2291 rpcz_store.cc:276] 0709 13:04:20.692999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.722351 (+ 29352us) service_pool.cc:224] Handling call
0709 13:04:21.830730 (+1108379us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540040a954. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e82fa 00007f70d39a9b17 00007f70ccf246d4 00007f70ccf24662 00007f70ccf24591 00007f70ccf2450c 00007f70ccf237c2 00007f70ccd1f48c 00007f70ccd1e8c6 00007f70ccd1e26f
0709 13:04:21.830852 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639110528}
I20260709 13:04:21.833839  2290 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:04:21.836763  2293 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:04:21.841508  9305 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:21.845221  9305 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:21.847919  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.852845  1601 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:21.849887  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.860282  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:21.861001  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:21.861328  2294 raft_consensus.cc:2368] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
W20260709 13:04:21.873900  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:21.875557  2263 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.863646 10013 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.881971  9565 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:21.884266 10002 log.cc:927] Time spent T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.382s	user 0.000s	sys 0.002s
I20260709 13:04:21.888248  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.888465  1630 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:04:21.889863  1631 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:21.859474  9914 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:21.892822  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:21.894647 10017 log.cc:927] Time spent T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 13:04:21.896682  1675 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.897154  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:21.900745  1628 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:21.905424  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.898016  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:21.911249  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:21.912585  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.923086  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:21.920584  9305 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:21.931371  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.932155  1702 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 14.
I20260709 13:04:21.947350  9918 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.948312  9777 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:21.948969  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec944cb8c5da4037a28a6917759780b4" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.969038  1673 leader_election.cc:400] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:04:21.969408  1673 leader_election.cc:403] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.949602  9962 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:21.944624  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.971123  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.957261  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.972007  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:21.971901  1628 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 15: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:21.957615  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:21.957938  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:21.964308  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.964716  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.970695  1626 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:21.971858  1707 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 14.
I20260709 13:04:21.927627  9769 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.927325  9982 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:21.975283  1629 raft_consensus.cc:2393] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:21.976241  9305 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:21.978222  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:21.984699  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 130697us
I20260709 13:04:21.924821  9999 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:04:21.926126  9884 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:04:21.990345  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77308d0ddc334cbfa9170c6422895ca9" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.994967  9305 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509) with higher term. Message: Invalid argument: T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 23. Current term is 24.
I20260709 13:04:21.997701  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:21.975059  9664 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:22.002209  9664 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:22.004659  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:22.005317  1629 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.012434  9859 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 13:04:22.013080  1629 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:04:22.014443  1525 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:22.015311  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.024657  9884 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:22.025242  9884 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Becoming Leader. State: Replica: 444b494bb53842b59408a7230884398a, State: Running, Role: LEADER
I20260709 13:04:22.025915  9995 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.026079  9884 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:22.026429  9995 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:21.973656  9918 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:22.028885  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.030846  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:22.036064  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:22.036600  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.013821  9996 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:04:22.041564  9995 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:22.023176  9867 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:04:22.042640  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:22.043207  9830 raft_consensus.cc:2764] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:21.983038  9944 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
W20260709 13:04:22.032174  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.056990  1709 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:04:22.061507  1709 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:22.088927  9944 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:22.089610  9918 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:22.098742  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:22.099618  1704 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:04:22.092244  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.113549  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.121384  1628 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:22.089689  9944 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:22.164139  9884 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.164975  9884 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:22.172408  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.176201  9884 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:22.182309  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 74357us
I20260709 13:04:22.195401  9944 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:22.196615  1674 leader_election.cc:400] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:22.197013  1674 leader_election.cc:403] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:22.198809  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.199376  9864 raft_consensus.cc:3060] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.209316  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:22.210052  1704 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 444b494bb53842b59408a7230884398a in term 16.
I20260709 13:04:22.211187  9305 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.135967  9982 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 13:04:22.221308  1601 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.222389  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:22.222954  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.227746  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:22.229285  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.230865  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:22.250656  1675 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.257278  1701 raft_consensus.cc:2468] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 24.
I20260709 13:04:22.264202  1523 leader_election.cc:304] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.273818  9867 raft_consensus.cc:2749] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:22.274966  1523 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.276855  9859 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.278321  1523 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:22.282418  2260 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.293768  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.296773  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:22.296967  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:22.298547  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.304862  1630 raft_consensus.cc:1275] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:04:22.316883  9305 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:22.323534  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 45279.8us
I20260709 13:04:22.326290  1523 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
W20260709 13:04:22.313431 10009 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:04:22.320286  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:22.374797  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 131298us
I20260709 13:04:22.311305  9982 raft_consensus.cc:2804] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:22.381675  9982 raft_consensus.cc:697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:22.336009  9974 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:22.387066  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.392836  9968 consensus_queue.cc:1048] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:04:22.399633  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 73430.7us
I20260709 13:04:22.382791  9982 consensus_queue.cc:237] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:22.459659  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.460959  1523 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.400146  9918 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:22.425690  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 156584us
I20260709 13:04:22.388355 10007 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:22.470067  1523 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.475534  1523 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.482383  9995 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:22.488389 10037 log.cc:927] Time spent T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:22.496970  9884 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:22.498466  1523 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:22.500900  9884 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:22.513649 10035 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:04:22.514313 10004 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:22.475406  9859 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:22.523679  9864 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515) with higher term. Message: Invalid argument: T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c for earlier term 19. Current term is 20.
I20260709 13:04:22.537545  9995 raft_consensus.cc:493] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.540055  1706 raft_consensus.cc:1275] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:22.540313  9995 raft_consensus.cc:515] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:22.542644  9995 leader_election.cc:290] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:22.405794  9867 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:22.566089  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.575565  1626 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:04:22.590147  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 91773.8us
I20260709 13:04:22.591600  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:22.595969 10010 consensus_queue.cc:1048] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:04:22.615639  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.623733  1523 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.629967  9985 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:22.628059  9995 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:04:22.629257  1523 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:22.613967  9867 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
W20260709 13:04:22.642874  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.657521  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a10c27e2d4480a7a262c77167ed4b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.658396  1701 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:22.658978  1707 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:22.655565  9962 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:22.643603  9777 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:04:22.671888  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 72522.7us
I20260709 13:04:22.666882  9995 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:22.664139  9884 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.690408  1451 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:22.666649  9999 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.704541  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:22.706544  9995 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:22.705889  1451 catalog_manager.cc:5697] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 20 to 24, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 24 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.713403  9995 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:22.719955  9962 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:04:22.720361  9999 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:22.756716  9999 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:22.745167  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 43844.2us
I20260709 13:04:22.761700  1626 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Refusing update from remote peer 444b494bb53842b59408a7230884398a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:22.762876  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:22.715405  1451 catalog_manager.cc:5697] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 12 to 27, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 27 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:22.716292  1601 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 225 similar messages]
W20260709 13:04:22.769817  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 53561.3us
W20260709 13:04:22.772665  1599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 293662us
I20260709 13:04:22.770380  1451 catalog_manager.cc:5697] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.714440  9884 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:22.784089   431 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 1585)
W20260709 13:04:22.782095  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:22.799276  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 97277.9us
I20260709 13:04:22.800102  1523 leader_election.cc:304] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:22.804207  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 99108us
I20260709 13:04:22.806185  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:22.813993  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:04:22.819113  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bcf131126594cae9b8c0f94d39caaf8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.823652  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:22.828615 10066 raft_consensus.cc:2749] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:22.830767 10057 log.cc:927] Time spent T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:04:22.845348 10055 log.cc:927] Time spent T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:04:22.846808  1523 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:22.873445  1523 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:22.789973  9974 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:04:22.899940 10066 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:22.883945  9995 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:22.891754 10063 log.cc:927] Time spent T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: Append to log took a long time: real 0.112s	user 0.004s	sys 0.000s
I20260709 13:04:22.886801 10067 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:22.940142  9884 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:22.995674  1451 catalog_manager.cc:5697] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:23.002127  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 127143us
W20260709 13:04:23.005386  1524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 130215us
I20260709 13:04:23.006317  1451 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 21, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.013298  1451 catalog_manager.cc:5697] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:23.026654  1454 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 301) took 75685 ms (client timeout 15000 ms). Trace:
I20260709 13:04:23.036677  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.051393  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:23.031080  1451 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 22, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.130155  1709 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:04:23.134486  9995 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.135411  9995 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:23.074258  1454 rpcz_store.cc:269] 0709 13:03:07.341051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.359459 (+ 18408us) service_pool.cc:224] Handling call
0709 13:04:23.026313 (+75666854us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:21.791053 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:21.816435 (+ 25382us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:21.816493 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:21.816512 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:21.817514 (+  1002us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:21.817571 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:21.817596 (+    25us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:04:21.818279 (+   683us) tablet.cc:701] Row locks acquired
0709 13:04:21.818294 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:04:21.818440 (+   146us) write_op.cc:270] Start()
0709 13:04:21.818558 (+   118us) write_op.cc:276] Timestamp: P: 1783602261818390 usec, L: 0
0709 13:04:21.818577 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:21.819223 (+   646us) log.cc:844] Serialized 3258 byte log entry
0709 13:04:22.075868 (+256645us) op_driver.cc:464] REPLICATION: finished
0709 13:04:22.303098 (+227230us) write_op.cc:301] APPLY: starting
0709 13:04:22.309630 (+  6532us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:22.363277 (+ 53647us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:22.363396 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.478140 (+114744us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.478436 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:04:22.478509 (+    73us) write_op.cc:312] APPLY: finished
0709 13:04:22.478796 (+   287us) log.cc:844] Serialized 81 byte log entry
0709 13:04:22.485011 (+  6215us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.485172 (+   161us) write_op.cc:454] Released schema lock
0709 13:04:22.485222 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.485643 (+   421us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.485676 (+    33us) write_op.cc:454] Released schema lock
0709 13:04:22.532105 (+ 46429us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94d571 00007f70ce9e132a 00007f70ce966f14 00007f70ce966ec3 00007f70ce966e38 00007f70ce966df3 00007f70ce9660dc 00007f70ce98fe16 00007f70ce977cbb 00007f70ce96239d 00007f70ce965d75
Metrics: {"mutex_wait_us":2338926,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":175680,"apply.run_cpu_time_us":15474,"apply.run_wall_time_us":212644,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":211539,"prepare.run_cpu_time_us":8220,"prepare.run_wall_time_us":83088,"raft.queue_time_us":85895,"raft.run_cpu_time_us":52845,"raft.run_wall_time_us":458839,"replication_time_us":256100,"spinlock_wait_cycles":2747136,"thread_start_us":283638,"threads_started":4,"wal-append.queue_time_us":107353}]]}
I20260709 13:04:23.137920  9995 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:23.140007  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 71736.5us
I20260709 13:04:23.145255  1451 catalog_manager.cc:5697] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 26 to 28, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 28 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.170437  9985 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.171598  9985 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:23.179903  9985 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:23.147289  1451 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 19, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.181692  1451 catalog_manager.cc:5697] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 23 to 25, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 25 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.184763  1524 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:23.188083  9791 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:23.212457  9565 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:23.212517  1626 raft_consensus.cc:3060] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:23.223202  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.225073  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.183698  1451 catalog_manager.cc:5697] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.231741  1451 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 23 to 26, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 26 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.234284  1451 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 15 to 26. New cstate: current_term: 26 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.236296  1451 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 17 to 18, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 18 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.262241  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e0479a2b6a6473eac54a01fde6fc25c" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:23.264228  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:23.274796  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:23.297901  1626 raft_consensus.cc:1275] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:04:23.308565  9565 consensus_queue.cc:1048] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:23.309043  9974 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.309849  9974 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:23.312460  9974 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:23.238072  1451 catalog_manager.cc:5697] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 25, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:23.322144  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:23.337527  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:23.385848  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:23.391937  1451 catalog_manager.cc:5697] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 14 to 24, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 24 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.406984  9995 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.407670  9995 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:23.414592  9985 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.415262  9985 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:23.417403  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.420136  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.434149  1525 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:23.440343  9995 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:23.440565  9867 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:23.441816  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 68540.2us
I20260709 13:04:23.443238  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:23.420253  9985 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:23.451207  1524 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:23.452664  9867 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:23.455864  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:23.456765  9974 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:23.457371  1451 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.459035  1451 catalog_manager.cc:5697] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 20, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 20 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:23.484215  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2292aebe10c64bc99c27e3976075c309" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:23.494279  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a9cb8cee6d447e9042f38fe30cb6e0" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:23.501248  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:23.501999  9985 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.502884  9985 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:23.505676  9985 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:23.507325  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:23.555042  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:23.555795  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:23.556412  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:23.556988  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:23.583819  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:23.600322  1525 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:23.602100  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:23.612277  9867 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:23.619102  1525 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:23.623476  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:23.625310  9985 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:23.628868  9796 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:23.707206  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:23.746798  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:23.748602  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:23.763139  9973 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.763881  9973 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } attrs { replace: true } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:23.779088  9973 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:23.779027  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:23.780176  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:23.781728  1524 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:23.782488  9973 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:23.784894   431 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:23.802058  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b765b4fb8a24da18c3dc1ec41dd5e80" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:23.959335  1451 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 216) took 2730 ms. Trace:
I20260709 13:04:23.965039  1451 rpcz_store.cc:276] 0709 13:04:21.228547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.228896 (+   349us) service_pool.cc:224] Handling call
0709 13:04:23.959240 (+2730344us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:23.514578 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:23.515446 (+   868us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:23.515502 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:04:23.515523 (+    21us) tablet.cc:662] Decoding operations
0709 13:04:23.523074 (+  7551us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:23.523294 (+   220us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:23.523318 (+    24us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:04:23.529233 (+  5915us) tablet.cc:701] Row locks acquired
0709 13:04:23.529285 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:04:23.533042 (+  3757us) write_op.cc:270] Start()
0709 13:04:23.533255 (+   213us) write_op.cc:276] Timestamp: P: 1783602263532895 usec, L: 0
0709 13:04:23.533293 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:04:23.535004 (+  1711us) log.cc:844] Serialized 7490 byte log entry
0709 13:04:23.704723 (+169719us) op_driver.cc:464] REPLICATION: finished
0709 13:04:23.766876 (+ 62153us) write_op.cc:301] APPLY: starting
0709 13:04:23.766982 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:23.811602 (+ 44620us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:23.811736 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:23.866162 (+ 54426us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:23.866587 (+   425us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:04:23.866652 (+    65us) write_op.cc:312] APPLY: finished
0709 13:04:23.867115 (+   463us) log.cc:844] Serialized 172 byte log entry
0709 13:04:23.867303 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:23.867488 (+   185us) write_op.cc:454] Released schema lock
0709 13:04:23.867562 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:23.919360 (+ 51798us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:23.919393 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":312936,"reported_tablets":28,"spinlock_wait_cycles":12032,"child_traces":[["op",{"apply.queue_time_us":61745,"apply.run_cpu_time_us":25180,"apply.run_wall_time_us":152436,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83623,"num_ops":19,"prepare.queue_time_us":5394,"prepare.run_cpu_time_us":15015,"prepare.run_wall_time_us":69459,"raft.queue_time_us":82771,"raft.run_cpu_time_us":12633,"raft.run_wall_time_us":68638,"replication_time_us":167937,"thread_start_us":135874,"threads_started":4,"wal-append.queue_time_us":23647}]]}
W20260709 13:04:23.969149  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:23.980567  1453 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 11 to 13, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:24.042145  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.047271  9796 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.049285  9796 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:24.053335  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:24.053517  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:24.056303  1523 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:24.057256  9973 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:24.057967  1451 catalog_manager.cc:5697] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.051733  9796 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:24.059784  1451 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:24.062597  1451 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 20 to 23, leader changed from 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 23 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:24.071209  1451 catalog_manager.cc:5697] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.074219  1451 catalog_manager.cc:5697] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 26 to 28, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 28 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:24.076061  1451 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:24.152832  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:24.165756  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:24.192726  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:24.197018  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.225312  9973 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:24.234920  9973 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:24.238421  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:24.241194  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6beee9f16eb4c7db2d293d74dfec496" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:24.242867  1525 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:24.243696  9985 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:24.243091  1457 catalog_manager.cc:5697] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.244831  9973 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:24.259213  9973 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.259809  9973 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:24.257074  1453 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 330) took 75417 ms (client timeout 15000 ms). Trace:
I20260709 13:04:24.263476  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:24.267277  9973 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:24.273851  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334958c5aee349629d63e898f13ba95b" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:24.279340  1524 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:24.280764  9985 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:24.261165  1457 catalog_manager.cc:5697] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 20 to 27, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 27 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.282435  1457 catalog_manager.cc:5697] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.284735  1457 catalog_manager.cc:5697] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.293102  1457 catalog_manager.cc:5697] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 14 to 22, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.298017  9985 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.264489  1453 rpcz_store.cc:269] 0709 13:03:08.840012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.842507 (+  2495us) service_pool.cc:224] Handling call
0709 13:03:09.612597 (+770090us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94e090 00007f70ce9ecf3a 00007f70d47a31cc 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc 00007f70d2d61392 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:24.042727 (+74430130us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.042763 (+    36us) write_op.cc:454] Released schema lock
0709 13:04:24.256976 (+214213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:24.024894 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:24.025486 (+   592us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:24.025526 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:04:24.025552 (+    26us) tablet.cc:662] Decoding operations
0709 13:04:24.025920 (+   368us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:24.025980 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:24.026013 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:24.026202 (+   189us) tablet.cc:701] Row locks acquired
0709 13:04:24.026217 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:04:24.026369 (+   152us) write_op.cc:270] Start()
0709 13:04:24.026520 (+   151us) write_op.cc:276] Timestamp: P: 1783602264026319 usec, L: 0
0709 13:04:24.026550 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:04:24.027194 (+   644us) log.cc:844] Serialized 541 byte log entry
0709 13:04:24.029331 (+  2137us) op_driver.cc:464] REPLICATION: finished
0709 13:04:24.030304 (+   973us) write_op.cc:301] APPLY: starting
0709 13:04:24.030397 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:24.031298 (+   901us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:24.031400 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:24.035384 (+  3984us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:24.035552 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:04:24.035601 (+    49us) write_op.cc:312] APPLY: finished
0709 13:04:24.035831 (+   230us) log.cc:844] Serialized 25 byte log entry
0709 13:04:24.036011 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.036102 (+    91us) write_op.cc:454] Released schema lock
0709 13:04:24.036144 (+    42us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":166713,"reported_tablets":48,"spinlock_wait_cycles":46331776,"child_traces":[["op",{"apply.queue_time_us":726,"apply.run_cpu_time_us":3023,"apply.run_wall_time_us":6158,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":7031,"prepare.run_cpu_time_us":2578,"prepare.run_wall_time_us":2578,"replication_time_us":2635}]]}
I20260709 13:04:24.380470  1457 catalog_manager.cc:5697] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 13 to 20, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 20 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.483117  9796 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.491086  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.493263  9985 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:24.496397  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:24.496002  1457 catalog_manager.cc:5697] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.497531  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 13 to 20, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 20 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.499411  1457 catalog_manager.cc:5697] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 18 to 26, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.499995  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce0393bceaa489d896dd65113130131" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:24.503623  1457 catalog_manager.cc:5697] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 13 to 23, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 23 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.505250  1524 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:24.495712  9985 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:24.519362  9995 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:24.536864  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.525758  9796 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:24.554957  9796 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:24.555922  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:24.574712  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49005.5us
I20260709 13:04:24.531375  1457 catalog_manager.cc:5697] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 12 to 25. New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:24.576972  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 67507.8us
I20260709 13:04:24.578685  1457 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:24.582540  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.588745  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e43ddec31b504a3b9c8f06a96cd7013d" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:24.596710  1457 catalog_manager.cc:5697] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 19, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.598838  1457 catalog_manager.cc:5697] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
W20260709 13:04:24.617817  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.617105  1523 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:24.627578  9796 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:24.624174  1457 catalog_manager.cc:5697] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } }
I20260709 13:04:24.638368  1457 catalog_manager.cc:5697] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 12 to 22. New cstate: current_term: 22 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.647169  1457 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 7 to 14, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 14 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.648878  1457 catalog_manager.cc:5697] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 15 to 23. New cstate: current_term: 23 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:24.755542 10098 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:24.756371 10098 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:24.759945  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:24.761144 10098 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:24.775076  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0ffe513d79430292b928b5fcde37fe" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:24.780133  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:24.782156  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:24.784005  1525 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:24.785367   431 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 1585)
I20260709 13:04:24.785096 10098 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:24.822885  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.004731  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.019522  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.024009  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 44538.7us
W20260709 13:04:25.038743  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:04:25.040601  1457 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 294) took 6149 ms. Trace:
I20260709 13:04:25.041056  1457 rpcz_store.cc:276] 0709 13:04:18.890940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.933629 (+ 42689us) service_pool.cc:224] Handling call
0709 13:04:25.040513 (+6106884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:24.672808 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:24.673421 (+   613us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:24.673456 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:04:24.673476 (+    20us) tablet.cc:662] Decoding operations
0709 13:04:24.675113 (+  1637us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:24.675201 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:24.675234 (+    33us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:04:24.676701 (+  1467us) tablet.cc:701] Row locks acquired
0709 13:04:24.676724 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:04:24.676896 (+   172us) write_op.cc:270] Start()
0709 13:04:24.677023 (+   127us) write_op.cc:276] Timestamp: P: 1783602264676838 usec, L: 0
0709 13:04:24.677048 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:04:24.677886 (+   838us) log.cc:844] Serialized 7134 byte log entry
0709 13:04:24.866583 (+188697us) op_driver.cc:464] REPLICATION: finished
0709 13:04:24.881369 (+ 14786us) write_op.cc:301] APPLY: starting
0709 13:04:24.881475 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:24.919753 (+ 38278us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:24.919928 (+   175us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:24.948562 (+ 28634us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:24.948885 (+   323us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:04:24.948938 (+    53us) write_op.cc:312] APPLY: finished
0709 13:04:24.949355 (+   417us) log.cc:844] Serialized 164 byte log entry
0709 13:04:24.949548 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.949743 (+   195us) write_op.cc:454] Released schema lock
0709 13:04:24.949802 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:24.999934 (+ 50132us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.999975 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27957,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":14494,"apply.run_cpu_time_us":25540,"apply.run_wall_time_us":96306,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79439,"num_ops":18,"prepare.queue_time_us":15424,"prepare.run_cpu_time_us":5513,"prepare.run_wall_time_us":12189,"replication_time_us":189370,"spinlock_wait_cycles":62208,"thread_start_us":30719,"threads_started":2}]]}
W20260709 13:04:25.064107  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.064927 10097 raft_consensus.cc:493] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.065513 10097 raft_consensus.cc:515] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:25.070384  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:25.079847  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb76969419149b19b9ea9d40ebd2c24" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:25.081372  1523 leader_election.cc:304] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:25.082219  9985 raft_consensus.cc:2749] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:25.084296 10097 leader_election.cc:290] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:25.100059  1449 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 16 to 20, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 20 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.118645  1449 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.177943 10097 raft_consensus.cc:493] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.178630 10097 raft_consensus.cc:515] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:25.242746  1449 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45542 (request call id 293) took 29724 ms (client timeout 15000 ms). Trace:
I20260709 13:04:25.258200  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:25.271804  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.255245  1449 rpcz_store.cc:269] 0709 13:03:55.517885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.527987 (+ 10102us) service_pool.cc:224] Handling call
0709 13:04:25.242645 (+29714658us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:25.160483 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:25.161186 (+   703us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:25.161229 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:04:25.161257 (+    28us) tablet.cc:662] Decoding operations
0709 13:04:25.161782 (+   525us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:25.161854 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:25.161887 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:25.162271 (+   384us) tablet.cc:701] Row locks acquired
0709 13:04:25.162294 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:04:25.162473 (+   179us) write_op.cc:270] Start()
0709 13:04:25.162613 (+   140us) write_op.cc:276] Timestamp: P: 1783602265162408 usec, L: 0
0709 13:04:25.162639 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:04:25.163361 (+   722us) log.cc:844] Serialized 929 byte log entry
0709 13:04:25.165230 (+  1869us) op_driver.cc:464] REPLICATION: finished
0709 13:04:25.165783 (+   553us) write_op.cc:301] APPLY: starting
0709 13:04:25.165859 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:25.167163 (+  1304us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:25.167253 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:25.168446 (+  1193us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:25.168591 (+   145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:04:25.168632 (+    41us) write_op.cc:312] APPLY: finished
0709 13:04:25.168817 (+   185us) log.cc:844] Serialized 33 byte log entry
0709 13:04:25.168977 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.169058 (+    81us) write_op.cc:454] Released schema lock
0709 13:04:25.169097 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:25.169452 (+   355us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.169477 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1149,"reported_tablets":48,"spinlock_wait_cycles":16640,"child_traces":[["op",{"apply.queue_time_us":330,"apply.run_cpu_time_us":3601,"apply.run_wall_time_us":3601,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8368,"num_ops":2,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":3215,"prepare.run_wall_time_us":3215,"replication_time_us":2476}]]}
I20260709 13:04:25.287992 10097 leader_election.cc:290] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:25.289737  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e008100b0d36427c939df1036b5e0b67" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:25.291425  1524 leader_election.cc:304] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:25.292707 10098 raft_consensus.cc:2749] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:25.311920  1450 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.314229  1450 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 20 to 23, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 23 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.322420  1450 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 16 to 24, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.324765 10098 raft_consensus.cc:493] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.325412 10098 raft_consensus.cc:515] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:25.330153  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:25.333640  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5f3bc0667d44e69539243d156dac36" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:25.336984 10098 leader_election.cc:290] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:25.345193  1525 leader_election.cc:304] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:25.346218  9796 raft_consensus.cc:2749] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:25.356546 10097 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.357093 10097 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:25.359157 10097 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:25.362973  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:25.368649  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd4e3d43c7d4c97a3f229082334515f" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:25.371837  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.374130  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.373342  1525 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:25.377310 10097 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:25.325682  1450 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a reported cstate change: term changed from 19 to 22, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 444b494bb53842b59408a7230884398a (127.0.107.193). New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.410588  9796 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:25.411332  9796 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:25.414376  9796 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:25.418066  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:25.450105  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165cfdf51b34a5ea1bb6ece57f06bbc" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:25.481886  1525 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:25.488881 10097 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:25.500144  1452 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 14 to 17, leader changed from 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.524995  1450 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 304) took 10988 ms. Trace:
I20260709 13:04:25.544275 10098 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.545027 10098 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:25.547758 10098 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:25.550230  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:25.529225  1450 rpcz_store.cc:276] 0709 13:04:14.536763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.540032 (+  3269us) service_pool.cc:224] Handling call
0709 13:04:14.643310 (+103278us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b540002b7d4. stack: 00007f70cb8c5343 00007f70cb8c5240 00007f70cb2541d4 00007f70d4f92304 00007f70d4f92299 00007f70ce94e090 00007f70ce9ecf3a 00007f70d47a31cc 00007f70d48a137e 00007f70d2d622a4 00007f70d2d62232 00007f70d2d62161 00007f70d2d620dc 00007f70d2d61392 00007f70ccd1f48c 00007f70ccd1e8c6
0709 13:04:25.524899 (+10881589us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:25.392600 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:25.393116 (+   516us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:25.393158 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:04:25.393170 (+    12us) tablet.cc:662] Decoding operations
0709 13:04:25.393752 (+   582us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:25.393802 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:25.393838 (+    36us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:04:25.394225 (+   387us) tablet.cc:701] Row locks acquired
0709 13:04:25.394240 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:04:25.394376 (+   136us) write_op.cc:270] Start()
0709 13:04:25.394561 (+   185us) write_op.cc:276] Timestamp: P: 1783602265394335 usec, L: 0
0709 13:04:25.394580 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:25.395329 (+   749us) log.cc:844] Serialized 1706 byte log entry
0709 13:04:25.397227 (+  1898us) op_driver.cc:464] REPLICATION: finished
0709 13:04:25.397659 (+   432us) write_op.cc:301] APPLY: starting
0709 13:04:25.397735 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:25.399887 (+  2152us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:25.399978 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:25.402293 (+  2315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:25.402470 (+   177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:04:25.402516 (+    46us) write_op.cc:312] APPLY: finished
0709 13:04:25.402757 (+   241us) log.cc:844] Serialized 49 byte log entry
0709 13:04:25.403541 (+   784us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.403640 (+    99us) write_op.cc:454] Released schema lock
0709 13:04:25.403681 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:25.404020 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.404050 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":48,"spinlock_wait_cycles":52993920,"child_traces":[["op",{"apply.queue_time_us":238,"apply.run_cpu_time_us":5733,"apply.run_wall_time_us":6317,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16736,"num_ops":4,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":3024,"prepare.run_wall_time_us":3036,"replication_time_us":2532}]]}
I20260709 13:04:25.572700  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:25.577255   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:25.579977   431 tablet_replica.cc:333] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.582047   431 raft_consensus.cc:2243] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.582576  1524 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:25.583024   431 raft_consensus.cc:2272] T 9ce0393bceaa489d896dd65113130131 P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.594811   431 tablet_replica.cc:333] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.597110   431 raft_consensus.cc:2243] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.598685   431 raft_consensus.cc:2272] T de80dd9c151b41e88e1469985d074b81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.602389   431 tablet_replica.cc:333] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.616719  9796 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:25.644789  9796 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:25.617327 10097 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:25.648154  9796 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:25.647563  1457 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.648694   431 raft_consensus.cc:2243] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.649437  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:25.655310   431 pending_rounds.cc:70] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:25.655786   431 pending_rounds.cc:77] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 20 index: 17 } timestamp: 7305634835809849344 op_type: NO_OP noop_request { }
I20260709 13:04:25.656262   431 raft_consensus.cc:2889] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:25.658150  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:25.658855  1457 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.661485  1457 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.663414  1457 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.681466  1457 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.672165  1524 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:25.671933   431 raft_consensus.cc:2272] T 78699c8b654944aa9788917e2a177d14 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.683055  9985 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:25.686358   431 tablet_replica.cc:333] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:25.690737  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.693179   431 raft_consensus.cc:2243] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:25.693929  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.691682  1457 catalog_manager.cc:5697] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.700587  1457 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "444b494bb53842b59408a7230884398a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } }
I20260709 13:04:25.701695   431 raft_consensus.cc:2272] T e43ddec31b504a3b9c8f06a96cd7013d P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.706161   431 tablet_replica.cc:333] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.719515   431 raft_consensus.cc:2243] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:04:25.707306  1452 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 331) took 42112 ms (client timeout 15000 ms). Trace:
W20260709 13:04:25.731544  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:25.759364  1452 rpcz_store.cc:269] 0709 13:03:43.594594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.596002 (+  1408us) service_pool.cc:224] Handling call
0709 13:04:25.707219 (+42111217us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:25.521268 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:25.521799 (+   531us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:25.521836 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:04:25.521853 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:25.522202 (+   349us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:25.522258 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:25.522288 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:25.522469 (+   181us) tablet.cc:701] Row locks acquired
0709 13:04:25.522486 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:04:25.522629 (+   143us) write_op.cc:270] Start()
0709 13:04:25.522740 (+   111us) write_op.cc:276] Timestamp: P: 1783602265522590 usec, L: 0
0709 13:04:25.522762 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:25.523347 (+   585us) log.cc:844] Serialized 541 byte log entry
0709 13:04:25.525358 (+  2011us) op_driver.cc:464] REPLICATION: finished
0709 13:04:25.525852 (+   494us) write_op.cc:301] APPLY: starting
0709 13:04:25.525935 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:25.526747 (+   812us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:25.526840 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:25.527549 (+   709us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:25.527710 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:04:25.527800 (+    90us) write_op.cc:312] APPLY: finished
0709 13:04:25.527992 (+   192us) log.cc:844] Serialized 25 byte log entry
0709 13:04:25.528180 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.528267 (+    87us) write_op.cc:454] Released schema lock
0709 13:04:25.528317 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:25.528643 (+   326us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.528672 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":182551,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":305,"apply.run_cpu_time_us":2757,"apply.run_wall_time_us":2757,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":105,"prepare.run_cpu_time_us":2355,"prepare.run_wall_time_us":2355,"replication_time_us":2484}]]}
I20260709 13:04:25.778043   431 raft_consensus.cc:2272] T bcf9e0da0f934b968a7e9da19f0e0749 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.797716   431 tablet_replica.cc:333] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.820101   431 raft_consensus.cc:2243] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.822454   431 raft_consensus.cc:2272] T bcd4e3d43c7d4c97a3f229082334515f P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.840866   431 tablet_replica.cc:333] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.861171   431 raft_consensus.cc:2243] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.862563   431 raft_consensus.cc:2272] T 0d703c11931d4a2688046a3a48b71290 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.866354   431 tablet_replica.cc:333] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:25.866966  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.885116   431 raft_consensus.cc:2243] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.892942   431 pending_rounds.cc:70] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:25.898942   431 pending_rounds.cc:77] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 16 index: 11 } timestamp: 7305634793172705280 op_type: NO_OP noop_request { }
I20260709 13:04:25.899510   431 raft_consensus.cc:2889] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:25.899938   431 raft_consensus.cc:2272] T 3f0ffe513d79430292b928b5fcde37fe P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.903216   431 tablet_replica.cc:333] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.916764   431 raft_consensus.cc:2243] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:25.917485  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:25.917728   431 raft_consensus.cc:2272] T 4bcf131126594cae9b8c0f94d39caaf8 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.943287   431 tablet_replica.cc:333] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.956876   431 raft_consensus.cc:2243] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:04:25.962937  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:25.944648  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:25.973894   431 raft_consensus.cc:2272] T c4b89801fe624f179701cbd4b98f1490 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.993784   431 tablet_replica.cc:333] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:25.995177   431 raft_consensus.cc:2243] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.997084   431 raft_consensus.cc:2272] T f9d92cdb4b624c3bbc074afe1340b761 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.003520   431 tablet_replica.cc:333] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.024734   431 raft_consensus.cc:2243] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.030082   431 raft_consensus.cc:2272] T 2292aebe10c64bc99c27e3976075c309 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.065971  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.070775  9985 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.071424  9985 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:26.073815  9985 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:26.075887  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:26.076022  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:26.080539  1525 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:26.084205  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 302) took 52650 ms (client timeout 15000 ms). Trace:
W20260709 13:04:26.084759  1455 rpcz_store.cc:269] 0709 13:03:33.433735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.434088 (+   353us) service_pool.cc:224] Handling call
0709 13:04:26.084004 (+52649916us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2354461,"reported_tablets":48,"spinlock_wait_cycles":1792}
I20260709 13:04:26.045058   431 tablet_replica.cc:333] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.090232   431 raft_consensus.cc:2243] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.099046  9796 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:26.101874 10097 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.109328 10097 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:26.112208 10097 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:26.115144  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:26.119653   431 raft_consensus.cc:2272] T 1d5f3bc0667d44e69539243d156dac36 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.119498  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:26.125730   431 tablet_replica.cc:333] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.137012  1524 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:26.152773 10098 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.153460   431 raft_consensus.cc:2243] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.158869   431 raft_consensus.cc:2272] T 1fb76969419149b19b9ea9d40ebd2c24 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.167092   431 tablet_replica.cc:333] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:26.167888  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.181053   431 raft_consensus.cc:2243] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.192504   431 raft_consensus.cc:2272] T 9c25c5b342d344f3ac325ab5073d1747 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.200829   431 tablet_replica.cc:333] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.214388   431 raft_consensus.cc:2243] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.220523   431 pending_rounds.cc:70] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:26.230934   431 pending_rounds.cc:77] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 27 index: 19 } timestamp: 7305634824845651968 op_type: NO_OP noop_request { }
I20260709 13:04:26.248513 10098 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:26.249190 10098 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:26.235195   431 raft_consensus.cc:2889] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:04:26.213225  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.252086 10098 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:26.253058  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:26.253652  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea1928bdd634efaa34550f21ff564f3" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:26.260915  1525 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:26.261878   431 raft_consensus.cc:2272] T 334958c5aee349629d63e898f13ba95b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.263948  9985 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:26.205955  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45518 (request call id 303) took 32617 ms (client timeout 15000 ms). Trace:
W20260709 13:04:26.264748  1458 rpcz_store.cc:269] 0709 13:03:53.588603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.588870 (+   267us) service_pool.cc:224] Handling call
0709 13:04:26.205751 (+32616881us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36267,"reported_tablets":48}
I20260709 13:04:26.294015 10098 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.294656 10098 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:26.295565   431 tablet_replica.cc:333] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.298419  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:26.299290 10098 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:26.298774  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f032b031edf4e5eb9736c18f8ba0c03" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:26.302404  1525 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:26.303289 10098 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:26.303956   431 raft_consensus.cc:2243] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.304793   431 raft_consensus.cc:2272] T e008100b0d36427c939df1036b5e0b67 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.315213   431 tablet_replica.cc:333] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.329136   431 raft_consensus.cc:2243] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.330324   431 raft_consensus.cc:2272] T 7346e6ca66624e10898433387e0bbe68 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.339251   431 tablet_replica.cc:333] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.350471   431 raft_consensus.cc:2243] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:26.351646  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.367295   431 raft_consensus.cc:2272] T 542a10c27e2d4480a7a262c77167ed4b P 444b494bb53842b59408a7230884398a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.377980   431 tablet_replica.cc:333] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:26.387605  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.371157  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45532 (request call id 332) took 10752 ms. Trace:
I20260709 13:04:26.388336  1456 rpcz_store.cc:276] 0709 13:04:15.618779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.709326 (+ 90547us) service_pool.cc:224] Handling call
0709 13:04:26.370957 (+10661631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44226,"reported_tablets":48}
I20260709 13:04:26.390514   431 raft_consensus.cc:2243] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.420727   431 raft_consensus.cc:2272] T b1f19e2b30024564847382b872fc80e1 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.443874   431 tablet_replica.cc:333] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.444916   431 raft_consensus.cc:2243] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.450537   431 raft_consensus.cc:2272] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.472867   431 tablet_replica.cc:333] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.480446   431 raft_consensus.cc:2243] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.481993   431 raft_consensus.cc:2272] T 012e7f60d41140d496901ad4a16e6d4d P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.491261   431 tablet_replica.cc:333] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:26.496899  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.497954   431 raft_consensus.cc:2243] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.504357   431 raft_consensus.cc:2272] T a165cfdf51b34a5ea1bb6ece57f06bbc P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.522985   431 tablet_replica.cc:333] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.529618   431 raft_consensus.cc:2243] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.532104   431 raft_consensus.cc:2272] T 9df64a488003414aac04146e56487c81 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.554790   431 tablet_replica.cc:333] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.559721   431 raft_consensus.cc:2243] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.580452   431 raft_consensus.cc:2272] T 21235338303a443e8f10b8773c68e2db P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.583962   431 tablet_replica.cc:333] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.592624   431 raft_consensus.cc:2243] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.594692   431 pending_rounds.cc:70] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Trying to abort 2 pending ops.
I20260709 13:04:26.599901   431 pending_rounds.cc:77] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 24 index: 12 } timestamp: 7305634827878064128 op_type: NO_OP noop_request { }
I20260709 13:04:26.600481   431 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:26.600901   431 pending_rounds.cc:77] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 23 index: 11 } timestamp: 7305634783097540608 op_type: NO_OP noop_request { }
I20260709 13:04:26.604450   431 raft_consensus.cc:2889] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:26.604909   431 raft_consensus.cc:2272] T 9f032b031edf4e5eb9736c18f8ba0c03 P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.610671   431 tablet_replica.cc:333] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.620042   431 raft_consensus.cc:2243] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.628217   431 pending_rounds.cc:70] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:26.628640   431 pending_rounds.cc:77] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305634799656095744 op_type: NO_OP noop_request { }
W20260709 13:04:26.634243  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.630012   431 raft_consensus.cc:2889] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:26.641427   431 raft_consensus.cc:2272] T 88dfd27965e145dcb810423d7396b403 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.649324   431 tablet_replica.cc:333] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.650307   431 raft_consensus.cc:2243] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:04:26.651728  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.664238   431 raft_consensus.cc:2272] T c2dc331078c54a6aa71b5eab7cb93250 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.673200   431 tablet_replica.cc:333] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.686991   431 raft_consensus.cc:2243] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.689260   431 pending_rounds.cc:70] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:26.691918   431 pending_rounds.cc:77] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305634785767227392 op_type: NO_OP noop_request { }
I20260709 13:04:26.692431   431 raft_consensus.cc:2889] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:26.693545   431 raft_consensus.cc:2272] T ec944cb8c5da4037a28a6917759780b4 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.697535   431 tablet_replica.cc:333] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:26.725030  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.729887   431 raft_consensus.cc:2243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.740165   431 raft_consensus.cc:2272] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.761534   431 tablet_replica.cc:333] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.762633   431 raft_consensus.cc:2243] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.768168   431 raft_consensus.cc:2272] T ce44b855d6d442778079778cd0e68c34 P 444b494bb53842b59408a7230884398a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.771322   431 tablet_replica.cc:333] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.781914   431 raft_consensus.cc:2243] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.784241   431 pending_rounds.cc:70] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Trying to abort 1 pending ops.
I20260709 13:04:26.784579   431 pending_rounds.cc:77] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a: Aborting op as it isn't in flight: id { term: 22 index: 16 } timestamp: 7305634826620088320 op_type: NO_OP noop_request { }
I20260709 13:04:26.785013   431 raft_consensus.cc:2889] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:26.791986   431 raft_consensus.cc:2272] T 35904a4f4e3747dd85997b0dae5ae7bc P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.797155  9985 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.798949  9985 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:26.801285  9985 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:26.803349  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:26.803369  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d254c50f08d431298fe57ef1dde64be" candidate_uuid: "444b494bb53842b59408a7230884398a" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:26.805873  1525 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 444b494bb53842b59408a7230884398a; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:26.809106  9985 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:26.828358   431 tablet_replica.cc:333] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.839471   431 raft_consensus.cc:2243] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.849334   431 raft_consensus.cc:2272] T bd868018972045628b3604762f904e05 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.852957   431 tablet_replica.cc:333] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a: stopping tablet replica
W20260709 13:04:26.867448  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.880357   431 raft_consensus.cc:2243] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.881736   431 raft_consensus.cc:2272] T 62c0efbdda23450cb269a39d62c15422 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.886734   431 tablet_replica.cc:333] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.895423   431 raft_consensus.cc:2243] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.896795   431 raft_consensus.cc:2272] T c9744e1e31604277953315481c56b171 P 444b494bb53842b59408a7230884398a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.902889   431 tablet_replica.cc:333] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.909931   431 raft_consensus.cc:2243] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.911012   431 raft_consensus.cc:2272] T dea1928bdd634efaa34550f21ff564f3 P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.918625   431 tablet_replica.cc:333] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.926990   431 raft_consensus.cc:2243] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.928534   431 raft_consensus.cc:2272] T 26aae1e2780d42cda794411c9c1c0f32 P 444b494bb53842b59408a7230884398a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.940552   431 tablet_replica.cc:333] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.941464   431 raft_consensus.cc:2243] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.942317   431 raft_consensus.cc:2272] T d6beee9f16eb4c7db2d293d74dfec496 P 444b494bb53842b59408a7230884398a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.946363   431 tablet_replica.cc:333] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.947240   431 raft_consensus.cc:2243] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.949409   431 raft_consensus.cc:2272] T 8fb72ffc01144a2880010ee54d6e6c44 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.952347   431 tablet_replica.cc:333] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.956385   431 raft_consensus.cc:2243] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.960425   431 raft_consensus.cc:2272] T 9438873b8e1a4f0780fff8b93a9d89b8 P 444b494bb53842b59408a7230884398a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.963620   431 tablet_replica.cc:333] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.967571   431 raft_consensus.cc:2243] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:04:26.973193  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:26.977494  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:26.982672   431 raft_consensus.cc:2272] T e629c77df1d84117a78341cd5a55b213 P 444b494bb53842b59408a7230884398a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.986020   431 tablet_replica.cc:333] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.989315   431 raft_consensus.cc:2243] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.990675   431 raft_consensus.cc:2272] T 77308d0ddc334cbfa9170c6422895ca9 P 444b494bb53842b59408a7230884398a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.995887   431 tablet_replica.cc:333] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:26.999559 10090 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.002285 10090 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.005218 10090 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.006369  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:27.010540  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.011055  2263 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:27.012013  9565 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:27.012816 10105 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.013401 10105 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.017827 10105 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.019407  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:27.019852  1675 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.020151  2294 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
I20260709 13:04:27.022224   431 raft_consensus.cc:2243] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.025478 10105 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:27.029609  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.032840  9565 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.033420  9565 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.035492  9565 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.026284 10105 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.037503  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:27.038642  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.059705  1707 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:27.061229 10007 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:27.061764 10007 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.062211 10007 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:27.073594 10105 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:27.046856  2263 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.079190  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.080129  9565 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:27.080631  9565 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.081044  9565 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:27.076210  1675 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.083851   431 raft_consensus.cc:2272] T af498956d398433a8c293e398bd1255f P 444b494bb53842b59408a7230884398a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.087211   431 tablet_replica.cc:333] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:27.088187   431 raft_consensus.cc:2243] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.088968   431 raft_consensus.cc:2272] T 5e0479a2b6a6473eac54a01fde6fc25c P 444b494bb53842b59408a7230884398a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.100302   431 tablet_replica.cc:333] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:27.101115   431 raft_consensus.cc:2243] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.101722   431 raft_consensus.cc:2272] T 9d254c50f08d431298fe57ef1dde64be P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.105104   431 tablet_replica.cc:333] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:27.105895   431 raft_consensus.cc:2243] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.107218   431 raft_consensus.cc:2272] T 309f24e44cc94815a1ad3911262a7872 P 444b494bb53842b59408a7230884398a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.110350   431 tablet_replica.cc:333] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a: stopping tablet replica
I20260709 13:04:27.111303   431 raft_consensus.cc:2243] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.112243   431 raft_consensus.cc:2272] T 2139fc31dcac45ddbcc3f5046bd79ade P 444b494bb53842b59408a7230884398a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.115993 10104 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.116844 10104 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.121201  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:27.136613 10095 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.137281 10095 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.139688 10095 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.141701  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:27.142467  1707 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:27.144815  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.151218 10095 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:27.151755 10095 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.152230 10095 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.189131 10095 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.191803 10095 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.194357  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf9e0da0f934b968a7e9da19f0e0749" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:27.232082  1707 raft_consensus.cc:3060] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.216341 10007 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.202997  9565 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:27.238708  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 207237us
W20260709 13:04:27.244148  1675 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.244859 10007 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:27.252182  2263 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.254681  9565 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.255048  1707 raft_consensus.cc:2468] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:27.255465 10104 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.256310  1598 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:27.257002  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.258255  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:27.258937  1709 raft_consensus.cc:3060] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:27.262441  9976 raft_consensus.cc:2804] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:27.263046  9976 raft_consensus.cc:697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:27.263978  9976 consensus_queue.cc:237] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.270023  1709 raft_consensus.cc:2468] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
W20260709 13:04:27.272361  1601 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.282398  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.283561  2294 raft_consensus.cc:2393] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 23: Already voted for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in this term.
I20260709 13:04:27.284286 10007 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.285007 10007 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.291602 10108 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.292243 10108 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.292182  1458 catalog_manager.cc:5697] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 24 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:27.293071  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.291229 10007 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:27.274461  1601 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:27.300648  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.301288  1626 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:27.301589  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.302284  2260 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.303882 10112 raft_consensus.cc:2804] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:27.304430 10112 raft_consensus.cc:697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:27.305405 10112 consensus_queue.cc:237] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.314561  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:27.315436  2294 raft_consensus.cc:3060] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:27.295107 10108 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:27.311807  1675 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.326033  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:27.312736  9976 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:27.334741  2294 raft_consensus.cc:2468] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 27.
I20260709 13:04:27.336218  9976 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.338418  9976 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.342203  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:27.350057  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.355647  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
W20260709 13:04:27.356499  1601 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.357527  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:27.358656  1601 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.344250 10007 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:27.364707  1709 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:27.367022  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.343078  9976 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.367954 10095 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:27.368402 10095 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.368805 10095 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.375002  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.375705  2297 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
W20260709 13:04:27.394224  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.394018  1456 catalog_manager.cc:5697] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 23, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:27.400233  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:27.407112  1601 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.401147 10095 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.404253  1673 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:27.420382 10108 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.421110  1673 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.421476 10095 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.421993 10108 raft_consensus.cc:2804] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:27.422845 10108 raft_consensus.cc:697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 27 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:27.423609  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d703c11931d4a2688046a3a48b71290" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:27.424249  1709 raft_consensus.cc:3060] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.424314 10108 consensus_queue.cc:237] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.434221  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.435600 10007 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:27.436039 10007 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.436621 10007 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:27.445672  1601 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.446581  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.470386  1455 catalog_manager.cc:5697] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 26 to 27, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 27 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.472108  1709 raft_consensus.cc:2468] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:27.473280  1598 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.474381  9976 raft_consensus.cc:2804] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:27.474906  9976 raft_consensus.cc:697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:27.475878  9976 consensus_queue.cc:237] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.516948  1447 maintenance_manager.cc:419] P 8c0fa710867745f189aafd0e7d84fa64: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033383
W20260709 13:04:27.528263  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.531934 10112 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.532608 10112 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.538576 10112 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:27.540278  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.545050  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.547415  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:27.548203 10110 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:27.544620 10112 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.550014 10112 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:27.557332  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.559958  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.561915  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:27.562817 10110 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:27.562091 10112 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:27.597406  1458 catalog_manager.cc:5697] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 24 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.572571 10007 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.619758  9976 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.621663  9976 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.625945  9976 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.626475  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.627175  2297 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:27.628556  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.632488 10095 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:27.632917 10095 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.633292 10095 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:27.644802 10095 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.648214  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df64a488003414aac04146e56487c81" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:27.648887  2297 raft_consensus.cc:3060] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:27.657426  1446 maintenance_manager.cc:643] P 8c0fa710867745f189aafd0e7d84fa64: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.139s	user 0.031s	sys 0.007s Metrics: {"bytes_written":6538,"delete_count":0,"lbm_write_time_us":1101,"lbm_writes_lt_1ms":6,"reinsert_count":0,"update_count":276}
W20260709 13:04:27.659096  1675 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.663035 10095 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.673045 10095 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.667841  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:27.673743  1630 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.673619 10095 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:27.671314  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.673205  1447 maintenance_manager.cc:419] P 8c0fa710867745f189aafd0e7d84fa64: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.317349
W20260709 13:04:27.675666  1601 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:27.676440  1601 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.677563  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.664992  2297 raft_consensus.cc:2468] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 27.
I20260709 13:04:27.685774  1630 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:04:27.687285  1599 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:27.688822  1601 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.689338  1601 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:27.689615 10094 raft_consensus.cc:2804] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:27.686233  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:27.690567 10094 raft_consensus.cc:697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:27.690215  9976 raft_consensus.cc:2749] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.690716  2294 raft_consensus.cc:3060] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.691598 10094 consensus_queue.cc:237] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.701169  2294 raft_consensus.cc:2468] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:04:27.702898  1673 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.703919 10105 raft_consensus.cc:2804] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:27.687530 10007 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.712011 10095 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.718287 10105 raft_consensus.cc:697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:27.720593 10105 consensus_queue.cc:237] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:27.725585  1601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 310 similar messages]
W20260709 13:04:27.737846  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.749449  1709 raft_consensus.cc:1275] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:04:27.751061  9976 consensus_queue.cc:1048] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:27.760093  1709 raft_consensus.cc:1275] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:04:27.762884 10110 consensus_queue.cc:1048] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:04:27.769183  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.789544 10007 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.790210 10007 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.792497 10007 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:27.771719  1458 catalog_manager.cc:5697] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 26 to 27, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.802062  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:27.803283  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.804649  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:27.805805 10105 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:27.781236  1455 catalog_manager.cc:5697] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 24 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 25 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.818750 10113 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.822417 10113 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.822940 10111 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.823514 10111 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.824651 10113 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:27.825623 10111 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:27.828699  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.828946  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:27.829499  1630 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 22.
I20260709 13:04:27.839020  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.846238  9976 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.846820  9976 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.849045  9976 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.849603 10113 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:27.852244 10113 raft_consensus.cc:493] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.852746 10113 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:27.851137  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.853770  2297 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:27.855003  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
W20260709 13:04:27.856962  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.860175  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.862257  1709 raft_consensus.cc:1275] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 13:04:27.859681 10092 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:27.863324 10092 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.863699 10092 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.863802 10094 consensus_queue.cc:1048] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:27.865195 10094 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.865633 10094 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:27.873977 10092 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.876181 10092 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.877705  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7346e6ca66624e10898433387e0bbe68" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:27.878494  2297 raft_consensus.cc:3060] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:27.889142 10094 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:27.890604  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:27.891273  1703 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:27.894435  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.896744 10094 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:27.897169 10094 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
W20260709 13:04:27.897354  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:27.899264  1601 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.899425 10094 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:27.905661  1601 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:27.906921  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:27.907972 10113 raft_consensus.cc:515] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.911515  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f19e2b30024564847382b872fc80e1" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:27.912173  1630 raft_consensus.cc:3060] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:27.916596  2297 raft_consensus.cc:2468] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 25.
I20260709 13:04:27.920260  1630 raft_consensus.cc:2468] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 23.
I20260709 13:04:27.921823  2262 leader_election.cc:304] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:27.922794  1599 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:27.923584  9976 raft_consensus.cc:2804] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:27.924109  9976 raft_consensus.cc:697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:27.924965  9976 consensus_queue.cc:237] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.926709 10126 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:27.927292 10126 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.928972  2297 raft_consensus.cc:1275] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:04:27.935202 10105 consensus_queue.cc:1048] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:27.938202 10126 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:27.939157  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:27.932333 10112 raft_consensus.cc:2804] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:27.940001 10112 raft_consensus.cc:697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:27.940873 10112 consensus_queue.cc:237] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:27.930150 10113 leader_election.cc:290] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:27.985301  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:27.985718  1675 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:27.986104  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:27.986646  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:27.986243  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
W20260709 13:04:27.987835  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.988214 10007 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:27.989645 10113 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:27.996943  2263 leader_election.cc:336] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.007786 10104 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.008598 10104 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.011168 10104 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.013259  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.036530 10112 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.037194 10112 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.039384 10111 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.042232 10111 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.044430 10111 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:28.046375  2263 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.047827  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.050925  1601 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.051373  2263 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.051368  1601 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:28.053014 10092 raft_consensus.cc:2749] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:28.055642  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.059880  1458 catalog_manager.cc:5697] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 24 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:28.061262  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.061918  1709 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 18.
I20260709 13:04:28.063082  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:28.063980 10110 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:28.064373 10110 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.064745 10110 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:28.042171 10112 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.074358 10141 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.074965 10141 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.075572  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.058550  1455 catalog_manager.cc:5697] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 22 to 23, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 23 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.085251  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:28.087944 10112 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:28.101687 10141 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:28.105119  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:28.105890  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 23.
I20260709 13:04:28.105579 10110 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.108561  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.109691 10137 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:28.110163 10137 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.110478 10137 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:28.110528 10110 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.112851  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:28.113466  1703 raft_consensus.cc:3060] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:28.115994 10126 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.116559 10126 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:28.117163  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.119956  2263 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.122681 10105 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.123283 10105 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.124532  1675 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.125377 10105 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.129127  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:28.129786  2294 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 24.
I20260709 13:04:28.092736 10094 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.132596  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:28.132789  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.133575 10105 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:28.133986 10105 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.136943 10094 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.128072 10126 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:28.130780  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:28.141201  2297 raft_consensus.cc:1275] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:04:28.134363 10105 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.146730  1703 raft_consensus.cc:2468] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:28.147621  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:28.149158  1709 raft_consensus.cc:3060] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:28.141510  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:28.148964 10142 consensus_queue.cc:1048] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:28.151536  2260 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:28.152727 10112 raft_consensus.cc:2804] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:28.155225  1601 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.156147  9976 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.155987 10112 raft_consensus.cc:697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:28.157917  9976 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.160585 10112 consensus_queue.cc:237] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.162135  9976 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:28.168354  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:28.152729 10108 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:28.164937 10137 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.175390  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2dc331078c54a6aa71b5eab7cb93250" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:28.176048  2297 raft_consensus.cc:3060] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:28.177371  1709 raft_consensus.cc:2468] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:28.184445  1598 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:28.185444 10138 raft_consensus.cc:2804] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:28.185945 10138 raft_consensus.cc:697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:28.186774  2297 raft_consensus.cc:2468] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 24.
I20260709 13:04:28.186796 10138 consensus_queue.cc:237] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.199152  1599 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.174275 10137 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:28.191859 10126 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.204878 10126 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.184584  1601 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.201874  9976 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.203591 10142 raft_consensus.cc:2804] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:28.204135 10094 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.207016  1601 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:28.206498  9976 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.216228  1601 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.215750 10142 raft_consensus.cc:697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:28.216892 10137 raft_consensus.cc:2749] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:04:28.213750 10094 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.217535 10142 consensus_queue.cc:237] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.219842 10094 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.221673  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.222198  2297 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:04:28.222287  1709 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 26.
I20260709 13:04:28.223459  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.230770 10143 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.231560  9976 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.230047 10108 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.233827 10108 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.234637  1455 catalog_manager.cc:5697] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 18 to 19, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.231370 10143 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.237711 10126 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.237903 10143 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.228454 10138 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:28.239480 10138 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.239869 10138 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:28.240358  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:28.242363  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.243018  1709 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 16.
I20260709 13:04:28.244102  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:28.244233  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.244714  1626 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 16.
I20260709 13:04:28.245180  9976 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:28.245553  9976 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.245934  9976 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.249562  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.249936  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:28.250699  1626 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
W20260709 13:04:28.251360  1601 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.233213  1458 catalog_manager.cc:5697] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 25 to 26, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.252146  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:28.238313 10126 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:04:28.254849 10105 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.245556 10108 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.258611 10105 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:28.259548  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:28.260465  2297 raft_consensus.cc:3060] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.252139  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.268829 10105 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:28.267285 10143 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:28.286830 10105 raft_consensus.cc:493] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.306986 10138 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.290722 10143 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.298653 10105 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:28.336010  2297 raft_consensus.cc:2468] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:04:28.322105 10143 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.344928  1673 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:28.364583 10148 raft_consensus.cc:2804] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:28.365347 10148 raft_consensus.cc:697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:28.368142  2297 raft_consensus.cc:1275] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:04:28.377015  1601 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.378876 10094 consensus_queue.cc:1048] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 13:04:28.387171 10105 raft_consensus.cc:515] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.366510 10148 consensus_queue.cc:237] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:28.394289  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9744e1e31604277953315481c56b171" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:28.401355  1703 raft_consensus.cc:3060] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:28.379941 10138 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.419879  1626 raft_consensus.cc:1275] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:04:28.423256  1703 raft_consensus.cc:2468] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 27.
I20260709 13:04:28.424366  9976 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.424607  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47623.7us
I20260709 13:04:28.429137  1598 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.429557 10111 consensus_queue.cc:1048] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:04:28.440742  9976 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.452612 10138 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.461474  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:28.460285 10138 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.465986 10138 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:28.431368 10112 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.473301 10105 leader_election.cc:290] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.468703 10142 raft_consensus.cc:2804] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:28.478664  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:28.479535  1709 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 19.
I20260709 13:04:28.483290  1626 raft_consensus.cc:1275] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:04:28.492342  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309f24e44cc94815a1ad3911262a7872" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:28.493017  1626 raft_consensus.cc:3060] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:28.492228  1458 catalog_manager.cc:5697] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 23 to 24, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 24 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.500718  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 155860us
I20260709 13:04:28.503577  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:28.513594 10151 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:28.514114 10151 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.514480 10151 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:28.491328 10142 raft_consensus.cc:697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
W20260709 13:04:28.525704  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.534896  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.536253  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.536891  1675 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:28.547850 10126 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:28.549346 10152 log.cc:927] Time spent T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:04:28.550654  1675 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.485837 10112 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:28.581924  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.582814  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:28.509052 10143 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.590421  1601 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.595338  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.608793  1703 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b2c3934407d422495728f84374c6b53 in current term 17: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:04:28.604329  1601 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.610011  1601 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.615195  1626 raft_consensus.cc:2468] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 23.
W20260709 13:04:28.616657  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 126214us
W20260709 13:04:28.617220   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1581)
I20260709 13:04:28.622519 10155 consensus_queue.cc:1048] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 13:04:28.624591  1675 leader_election.cc:336] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.628372 10112 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.619413 10142 consensus_queue.cc:237] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:28.638936  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:28.639635  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:28.640583  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.652889 10143 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:28.653770 10151 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.661448  1455 catalog_manager.cc:5697] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 24 to 25, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 25 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:28.663985 10151 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:28.678364  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:28.688472  1674 leader_election.cc:304] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:28.692330 10148 raft_consensus.cc:2804] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:28.692904 10148 raft_consensus.cc:697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:28.693780 10148 consensus_queue.cc:237] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.699862  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:28.700728  1630 raft_consensus.cc:2393] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f3fbc60b884078a3c219909b44f64c in current term 17: Already voted for candidate 8b2c3934407d422495728f84374c6b53 in this term.
I20260709 13:04:28.701632  1703 raft_consensus.cc:1275] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:04:28.700538  2263 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.713788  1601 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:28.715826  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:28.716372  1675 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:28.720645 10158 consensus_queue.cc:1048] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
W20260709 13:04:28.718684  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.730139  1456 catalog_manager.cc:5697] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 26 to 27, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.735069  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.735577  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:28.758900  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.745282 10155 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:28.772306  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c0efbdda23450cb269a39d62c15422" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:28.773028  1705 raft_consensus.cc:3060] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:28.776746  1701 raft_consensus.cc:1275] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:28.806392  1598 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:28.810570  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:28.832463  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:28.834107 10137 consensus_queue.cc:1048] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:28.839584  1626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:28.840399  1626 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 30.
I20260709 13:04:28.842832  1705 raft_consensus.cc:2468] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2c3934407d422495728f84374c6b53 in term 20.
I20260709 13:04:28.825044 10142 raft_consensus.cc:2749] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:28.852494  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
W20260709 13:04:28.862118  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.860497  1598 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:28.874205  9976 raft_consensus.cc:2804] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:28.874730  9976 raft_consensus.cc:697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Becoming Leader. State: Replica: 8b2c3934407d422495728f84374c6b53, State: Running, Role: LEADER
I20260709 13:04:28.875535  9976 consensus_queue.cc:237] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:28.858371 10113 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 13:04:28.883056  1455 catalog_manager.cc:5697] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 22 to 23, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 23 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.884173 10113 raft_consensus.cc:493] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 444b494bb53842b59408a7230884398a)
I20260709 13:04:28.899711  2290 raft_consensus.cc:1275] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:04:28.925496 10113 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:04:28.932502 10142 consensus_queue.cc:1048] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:28.895203  1446 maintenance_manager.cc:643] P 8c0fa710867745f189aafd0e7d84fa64: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 1.207s	user 0.111s	sys 0.008s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":27615,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":84717,"dirs.run_cpu_time_us":559,"dirs.run_wall_time_us":6334,"lbm_read_time_us":10841,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":19,"lbm_write_time_us":2257,"lbm_writes_lt_1ms":11,"peak_mem_usage":301851,"reinsert_count":0,"thread_start_us":88887,"threads_started":2,"update_count":286}
I20260709 13:04:28.965801  1447 maintenance_manager.cc:419] P 8c0fa710867745f189aafd0e7d84fa64: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:04:28.974227  1446 maintenance_manager.cc:643] P 8c0fa710867745f189aafd0e7d84fa64: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:04:28.985118  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.988300  2290 raft_consensus.cc:1275] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 13:04:28.990415 10144 consensus_queue.cc:1048] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 13:04:28.994154  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:28.997954  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:28.998665  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.010152 10167 log.cc:927] Time spent T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:29.063117  1703 raft_consensus.cc:1275] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:04:29.065514 10151 consensus_queue.cc:1048] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:29.070252 10113 raft_consensus.cc:515] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:29.096745  2263 leader_election.cc:336] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:29.113075 10113 leader_election.cc:290] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:29.115636  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af498956d398433a8c293e398bd1255f" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:29.116437  1633 raft_consensus.cc:3060] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 30 FOLLOWER]: Advancing to term 31
W20260709 13:04:29.118808  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.137737  1633 raft_consensus.cc:2468] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 31.
I20260709 13:04:29.145359  2262 leader_election.cc:304] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53; no voters: 444b494bb53842b59408a7230884398a
I20260709 13:04:29.146551 10111 raft_consensus.cc:2804] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 13:04:29.147701 10111 raft_consensus.cc:697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 31 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:29.148741 10111 consensus_queue.cc:237] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:29.150662  1456 catalog_manager.cc:5697] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 reported cstate change: term changed from 19 to 20, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 8b2c3934407d422495728f84374c6b53 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:29.230684  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.255904  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.250797  1455 catalog_manager.cc:5697] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 30 to 31, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 31 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:29.279428  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.280045  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.280669  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 82198.5us
W20260709 13:04:29.313400  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.310730  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:29.326946  1633 raft_consensus.cc:1275] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:29.328477 10108 consensus_queue.cc:1048] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:04:29.401204  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.408505  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.419371  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.448720  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:29.475600  1703 raft_consensus.cc:1275] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:29.479807 10137 consensus_queue.cc:1048] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:29.486742  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:29.556962  1627 raft_consensus.cc:1275] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 31 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 13:04:29.558593 10111 consensus_queue.cc:1048] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 13:04:29.603005  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.617651   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:29.760036  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.829859  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.844338  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:29.888666  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.167033  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.205227  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.231807  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.235165  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.309067  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:30.374600 10143 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.375263 10143 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:30.377512 10143 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:30.378980  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
I20260709 13:04:30.379724  1627 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 17.
I20260709 13:04:30.381021  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:30.382544 10108 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:30.382975 10108 raft_consensus.cc:493] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.383328 10108 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.425593 10108 raft_consensus.cc:515] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:30.429100 10108 leader_election.cc:290] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Requested vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:30.455310  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.458137  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e629c77df1d84117a78341cd5a55b213" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53"
I20260709 13:04:30.458810  1633 raft_consensus.cc:3060] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.466383  1633 raft_consensus.cc:2468] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 18.
I20260709 13:04:30.467893  1674 leader_election.cc:304] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53; no voters: 
I20260709 13:04:30.473135 10143 raft_consensus.cc:2804] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:30.485993  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.498209  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.505863  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:30.516822 10143 raft_consensus.cc:697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:30.518527 10143 consensus_queue.cc:237] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:30.600687  1456 catalog_manager.cc:5697] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 16 to 18, leader changed from 444b494bb53842b59408a7230884398a (127.0.107.193) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 18 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:30.618160   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:30.772678  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.824270  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:30.830389  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.953616  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.979916  1626 raft_consensus.cc:1275] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:30.985495 10143 consensus_queue.cc:1048] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:31.124864  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 48564us
W20260709 13:04:31.189790  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 63193.6us
W20260709 13:04:31.107587  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:31.194308  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 86780.2us
W20260709 13:04:31.211575  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:31.251859  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 46974.4us
W20260709 13:04:31.270336  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 74243.9us
W20260709 13:04:31.277064  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:31.278414  1675 leader_election.cc:336] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:31.292066  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 59257.9us
W20260709 13:04:31.306979  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.336779  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.361153  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.374583  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.430961  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.523350  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.526608  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:31.618661   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:31.628472  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 50524us
W20260709 13:04:31.652727  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.688534  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.692057  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.732323  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.771469  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.824623  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.835186  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.930814  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.934206  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.944941  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:31.980552  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.025319  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.039803  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.153925  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.190815  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.194144  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.277549  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.328073  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.432428  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.476243  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.490197  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.516237  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.619635   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:32.642326  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.719352  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:32.726639  2263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 443 similar messages]
W20260709 13:04:32.804695  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 65273.2us
W20260709 13:04:33.484798  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 158491us
W20260709 13:04:33.620352   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1581)
I20260709 13:04:33.802093 10216 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:33.804584 10216 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:33.829950  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:33.847134 10216 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
W20260709 13:04:33.852583  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:33.858675 10219 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:33.859520  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:33.859555 10219 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:33.866566 10219 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:33.877307 10219 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:33.877986 10219 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:33.880411 10219 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:33.882899 10219 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:33.883538 10219 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:33.885851 10219 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:33.890904  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "8b2c3934407d422495728f84374c6b53" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:33.903466  2290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:33.909017  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 95044.7us
I20260709 13:04:33.914166  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:33.916985  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:33.918708  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:33.927027  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:33.927675  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:33.928555 10219 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:33.930371  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:33.930883  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:33.931674 10219 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:33.940631  1598 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b2c3934407d422495728f84374c6b53; no voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb
I20260709 13:04:33.948007 10142 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:33.954262  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:33.961351  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 100566us
W20260709 13:04:33.964152  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:33.972962  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:33.984295  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:33.996240  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.997195  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:33.999006 10108 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:34.005595  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.007968  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.008496  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.033140  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08001754c0 after lost signal to thread 434
W20260709 13:04:34.034664  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd400 after lost signal to thread 1492
W20260709 13:04:34.036077  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08002e33e0 after lost signal to thread 1659
W20260709 13:04:34.037366  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800370b80 after lost signal to thread 1733
W20260709 13:04:34.038697  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080016a560 after lost signal to thread 2321
W20260709 13:04:34.039201  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080016a5a0 after lost signal to thread 10195
W20260709 13:04:34.039469  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800306a20 after lost signal to thread 10198
W20260709 13:04:34.109890  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.200590  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.211305  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.228363  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.229024  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.394871  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.428529  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.448340  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.461627  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.555714  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.621461  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:34.623847   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:34.656056  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:34.719964  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.729467  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.809777  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.924551  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.036561  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.079705  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.116590  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.135327  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.182164  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.222859  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.223309  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.241134  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.245838  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.291880  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.300647  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.352016  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.502646  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.515604  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.624348   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:35.640043  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.725175  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:35.823693  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.832108  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.039927  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.047830  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:36.058498  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.140853  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.206914  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.282651  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.351960  2260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 213303us
W20260709 13:04:36.441255  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 52105.6us
W20260709 13:04:36.463295  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.556936  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.557508  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.567849  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.610476  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.624840   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:36.629786  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 49915.4us
W20260709 13:04:36.675846  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 61287.6us
W20260709 13:04:36.676981  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:36.715242  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.719728  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.872419  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.886931  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.887684  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.918717  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.955997  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:36.961294  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.980201  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.018328  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:37.032590  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:37.073540  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.133039  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.164158  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.408588  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.427006  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:37.448007  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.488947  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.569906  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.579373  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:37.625380   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:37.666903  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.710079  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.730839  1601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 396 similar messages]
W20260709 13:04:37.745469  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.778270  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.828717  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.848099  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.868856  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.920177  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:37.935866  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.031859  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.047622  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.162767  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.208307  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.224702  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.294055  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.369278  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 47063.9us
W20260709 13:04:38.434373  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.455674  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.479805  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.492259  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.512007  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.625895   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:38.645960  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.709388  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.820792  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.843825  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.876048  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:38.897051  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.917347  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.020021  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.050637  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.047976  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.109279  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.258530  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.310612  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.311290  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.427174  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.445585  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:39.532846  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.549795  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.550459  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.573074  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:39.626446   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:39.799546  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.871096  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:39.893234  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.009135  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:40.071394  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.125977  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.131994  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 99142.7us
W20260709 13:04:40.140306  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.171418  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.215255  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 62030.4us
W20260709 13:04:40.289937  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 135759us
W20260709 13:04:40.297101  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.298602  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.299117  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.303627  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.307633  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.356600  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.459535  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.517351  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.577133  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.627009   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:40.627154  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.734545  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.802672  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.816159  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.906951  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.944113  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:40.949486  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.036221  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.139777  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.147410  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.177520  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080021ce80 after lost signal to thread 434
W20260709 13:04:41.179075  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003120e0 after lost signal to thread 1492
W20260709 13:04:41.181142  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08002ff6c0 after lost signal to thread 1659
W20260709 13:04:41.182464  1730 debug-util.cc:398] Leaking SignalData structure 0x7b0800357540 after lost signal to thread 1733
W20260709 13:04:41.183902  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080031ca20 after lost signal to thread 2321
W20260709 13:04:41.299948  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.446818  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.478547  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.484925  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.510299  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.516144  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.593048  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.593641  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.627575   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:41.664788  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.693455  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.739362  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.745934  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.762254  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.813344  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.947486  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.959118  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:41.992215  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.045030  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:42.082494  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:42.179601  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.199571  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.221738  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.294210  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:42.434298  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:42.593123  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:42.628110   431 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:42.647554  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:42.651741  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.089454  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 437731us
W20260709 13:04:43.051033  2263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 455 similar messages]
W20260709 13:04:43.096287  2263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45300.2us
W20260709 13:04:43.125524  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.225252  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.226006  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:43.356345  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.370018  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.418567  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.428015  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.432713  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.433303  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.505297  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.516717  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:43.528586  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.629370   431 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:43.648301  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 90541.4us
W20260709 13:04:43.846964  1673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:44509, user_credentials={real_user=slave}} blocked reactor thread for 192109us
I20260709 13:04:43.884203 10258 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:43.886128 10258 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:43.890954 10242 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:43.893214 10242 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:43.898408 10242 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:43.891841 10258 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:43.970988  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 56085.6us
I20260709 13:04:43.983086  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb72ffc01144a2880010ee54d6e6c44" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:43.988054  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:43.988348  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012e7f60d41140d496901ad4a16e6d4d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8b2c3934407d422495728f84374c6b53" is_pre_election: true
W20260709 13:04:44.019326  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.052338  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:44.053251  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:44.063306  1598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45021, user_credentials={real_user=slave}} blocked reactor thread for 89758.9us
I20260709 13:04:44.077378  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:44.089989 10261 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:44.079298  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.091547  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:44.090658  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:44.084142  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:44.105226 10261 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:44.142608  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.173887  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.199579  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc880 after lost signal to thread 434
W20260709 13:04:44.201455  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800094ba0 after lost signal to thread 1492
W20260709 13:04:44.203575  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2ee0 after lost signal to thread 1659
W20260709 13:04:44.205505  1656 debug-util.cc:398] Leaking SignalData structure 0x7b0800352f40 after lost signal to thread 1733
W20260709 13:04:44.207585  1656 debug-util.cc:398] Leaking SignalData structure 0x7b08002f72e0 after lost signal to thread 2321
W20260709 13:04:44.230695  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.334275  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.376520  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.380847  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.446086  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.529570  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.530248  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.610461  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.622077  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.629817   431 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:44.646690  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.648829  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.668516  1489 debug-util.cc:398] Leaking SignalData structure 0x7b08003823a0 after lost signal to thread 434
W20260709 13:04:44.669783  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080017d320 after lost signal to thread 1492
W20260709 13:04:44.671249  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800388880 after lost signal to thread 1659
W20260709 13:04:44.671536  1489 debug-util.cc:398] Leaking SignalData structure 0x7b080032ae40 after lost signal to thread 1660
W20260709 13:04:44.672768  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800400b80 after lost signal to thread 1733
W20260709 13:04:44.674115  1489 debug-util.cc:398] Leaking SignalData structure 0x7b0800418220 after lost signal to thread 2321
W20260709 13:04:44.679456  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.823293  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.910321  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:44.944460  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:45.026304  1633 consensus_queue.cc:237] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:45.094630  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38031, user_credentials={real_user=slave}} blocked reactor thread for 45786.4us
W20260709 13:04:45.114390  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.129760  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.172645  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:45.207127  2293 raft_consensus.cc:1275] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Refusing update from remote peer 8b2c3934407d422495728f84374c6b53: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:45.208984 10251 consensus_queue.cc:1048] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:45.224699 10250 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 4 to 15, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
I20260709 13:04:45.226059  2296 raft_consensus.cc:2955] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 4 to 15, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:45.250084  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:45.252300  1455 catalog_manager.cc:5697] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: config changed from index 4 to 15, VOTER 00f3fbc60b884078a3c219909b44f64c (127.0.107.195) evicted. New cstate: current_term: 19 leader_uuid: "8b2c3934407d422495728f84374c6b53" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } } }
W20260709 13:04:45.261137  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.332917  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
W20260709 13:04:45.356246  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.362658  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.404655  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.454716  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.504487  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:45.535758  1696 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 with delete_type TABLET_DATA_TOMBSTONED (TS 00f3fbc60b884078a3c219909b44f64c not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:36406
W20260709 13:04:45.537957  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:45.557020 10275 tablet_replica.cc:333] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:45.566857 10275 raft_consensus.cc:2243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.567992 10275 raft_consensus.cc:2272] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:45.569077  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:45.573144 10275 ts_tablet_manager.cc:1916] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:45.601331  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.617086  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.630288   431 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 1581)
I20260709 13:04:45.632148 10275 ts_tablet_manager.cc:1929] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.14
I20260709 13:04:45.632849 10275 log.cc:1199] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 00f3fbc60b884078a3c219909b44f64c: Deleting WAL directory at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0/minicluster-data/ts-2-root/wals/9b765b4fb8a24da18c3dc1ec41dd5e80
I20260709 13:04:45.639511  1442 catalog_manager.cc:5028] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): tablet 9b765b4fb8a24da18c3dc1ec41dd5e80 (table test-workload [id=5089d902ed414408826ea2cd4e7a981f]) successfully deleted
W20260709 13:04:45.673624  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.689815  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.692584  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.692950  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.737215  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:45.828318  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.888058  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.889377  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:45.947186  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.062520  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.075452  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.087860  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.292404  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.390976  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.521067  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.590731  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.617897  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.630825   431 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:46.644659  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.668185  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.685840  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.853832  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:46.913533  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.026450  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.071616  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.073331  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.104909  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.153003  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.174830  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.196209  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.223707  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.244659  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.282557  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.338932  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.368177  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.441149  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.490164  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:47.514458  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.635057   431 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:47.697415  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:47.760058  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.793479  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.812537  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:47.864984  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:47.980661  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:47.983740  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.005477  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.054447  1601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 461 similar messages]
W20260709 13:04:48.086293  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.131134  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.177208  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.182811  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.195308  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.252480  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.255158  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:48.316716  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.374019  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.376308  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.449750  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.490231  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.564694  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.622016  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.635550   431 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 1581)
W20260709 13:04:48.648691  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.919436  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:48.995405  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.047794  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.181449  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.258898  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.310424  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.347378  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.413036  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.423110  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.479075  1601 consensus_peers.cc:597] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.482581  1601 consensus_peers.cc:597] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.532697  1601 consensus_peers.cc:597] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.562057  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.570284  1601 consensus_peers.cc:597] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.589145  1601 consensus_peers.cc:597] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.699785  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.716894  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.778604  1601 consensus_peers.cc:597] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.793347  1601 consensus_peers.cc:597] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:49.793946  1601 consensus_peers.cc:597] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.796448  1601 consensus_peers.cc:597] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.897819  1601 consensus_peers.cc:597] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.005546  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:50.060034  1601 consensus_peers.cc:597] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:50.165589  1601 consensus_peers.cc:597] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.171419  1601 consensus_peers.cc:597] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.228988  1601 consensus_peers.cc:597] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.235320  1601 consensus_peers.cc:597] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:50.426280  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.466922  1601 consensus_peers.cc:597] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.522181  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.565478  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.639919  1601 consensus_peers.cc:597] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:50.650238  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.678478  1601 consensus_peers.cc:597] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.710880  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.721009  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.736351  1601 consensus_peers.cc:597] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.739399  1601 consensus_peers.cc:597] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.767026  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.858183  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.879439  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.926294  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:50.973336  1601 consensus_peers.cc:597] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.010317  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.147089  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.280825  1601 consensus_peers.cc:597] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.335305  1601 consensus_peers.cc:597] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:51.462108   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:51.539268  1601 consensus_peers.cc:597] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.616720  1601 consensus_peers.cc:597] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:51.621719  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:51.655886   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:51.657356  1601 consensus_peers.cc:597] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.666188  2262 consensus_peers.cc:597] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:51.679230  1674 consensus_peers.cc:597] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:51.688448  1674 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:51.706377  1674 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:51.719831  2262 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:51.758723  2262 consensus_peers.cc:597] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.762853   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:04:51.766458  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:51.770128   431 tablet_replica.cc:333] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.771270   431 raft_consensus.cc:2243] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.772639   431 raft_consensus.cc:2272] T 9ce0393bceaa489d896dd65113130131 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.776737   431 tablet_replica.cc:333] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.777617   431 raft_consensus.cc:2243] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.778288   431 raft_consensus.cc:2272] T 309f24e44cc94815a1ad3911262a7872 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.785264   431 tablet_replica.cc:333] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.786345   431 raft_consensus.cc:2243] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.787599   431 raft_consensus.cc:2272] T e43ddec31b504a3b9c8f06a96cd7013d P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.791014   431 tablet_replica.cc:333] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.792057   431 raft_consensus.cc:2243] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.793401   431 raft_consensus.cc:2272] T bcf9e0da0f934b968a7e9da19f0e0749 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.796725   431 tablet_replica.cc:333] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.797655   431 raft_consensus.cc:2243] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.799422   431 raft_consensus.cc:2272] T bcd4e3d43c7d4c97a3f229082334515f P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.802847   431 tablet_replica.cc:333] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.803864   431 raft_consensus.cc:2243] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.804538   431 raft_consensus.cc:2272] T 7f8c2a4b167d431aac3de9d8166bab1f P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.807519   431 tablet_replica.cc:333] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.808741   431 raft_consensus.cc:2243] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.810221   431 raft_consensus.cc:2272] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:51.822896  1674 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.824002   431 tablet_replica.cc:333] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.825173   431 raft_consensus.cc:2243] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.825888   431 raft_consensus.cc:2272] T 3d1d1686090d4b3abcf51f94f1a798d4 P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.829994   431 tablet_replica.cc:333] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.830960   431 raft_consensus.cc:2243] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.832353   431 raft_consensus.cc:2272] T 0d703c11931d4a2688046a3a48b71290 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.838219   431 tablet_replica.cc:333] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.839994   431 raft_consensus.cc:2243] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:04:51.840540  1674 consensus_peers.cc:597] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.841266   431 raft_consensus.cc:2272] T 4bcf131126594cae9b8c0f94d39caaf8 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.846272   431 tablet_replica.cc:333] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.847410   431 raft_consensus.cc:2243] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:04:51.857017  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:51.862557   431 raft_consensus.cc:2272] T 3f0ffe513d79430292b928b5fcde37fe P 8b2c3934407d422495728f84374c6b53 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.870909   431 tablet_replica.cc:333] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
W20260709 13:04:51.871322  1601 consensus_peers.cc:597] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:51.871948   431 raft_consensus.cc:2243] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.873140   431 raft_consensus.cc:2272] T f9d92cdb4b624c3bbc074afe1340b761 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.876112   431 tablet_replica.cc:333] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.876996   431 raft_consensus.cc:2243] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.878203   431 raft_consensus.cc:2272] T 2292aebe10c64bc99c27e3976075c309 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.881223   431 tablet_replica.cc:333] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.882961   431 raft_consensus.cc:2243] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.883828   431 raft_consensus.cc:2272] T 1d5f3bc0667d44e69539243d156dac36 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.886427   431 tablet_replica.cc:333] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
W20260709 13:04:51.886487  2262 consensus_peers.cc:597] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.887430   431 raft_consensus.cc:2243] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.888657   431 raft_consensus.cc:2272] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:51.897068  1674 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.897596   431 tablet_replica.cc:333] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.898522   431 raft_consensus.cc:2243] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.899238   431 raft_consensus.cc:2272] T 1fb76969419149b19b9ea9d40ebd2c24 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.902042   431 tablet_replica.cc:333] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.902931   431 raft_consensus.cc:2243] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.904114   431 raft_consensus.cc:2272] T 7346e6ca66624e10898433387e0bbe68 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.906872   431 tablet_replica.cc:333] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.907689   431 raft_consensus.cc:2243] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.908874   431 raft_consensus.cc:2272] T 334958c5aee349629d63e898f13ba95b P 8b2c3934407d422495728f84374c6b53 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.911669   431 tablet_replica.cc:333] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.912518   431 raft_consensus.cc:2243] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.913198   431 raft_consensus.cc:2272] T 542a10c27e2d4480a7a262c77167ed4b P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.915830   431 tablet_replica.cc:333] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.916621   431 raft_consensus.cc:2243] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.917409   431 raft_consensus.cc:2272] T b1f19e2b30024564847382b872fc80e1 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.920220   431 tablet_replica.cc:333] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.921032   431 raft_consensus.cc:2243] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.922194   431 raft_consensus.cc:2272] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.924876   431 tablet_replica.cc:333] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.925725   431 raft_consensus.cc:2243] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Raft consensus shutting down.
W20260709 13:04:51.926395  2262 consensus_peers.cc:597] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.928956   431 raft_consensus.cc:2272] T 012e7f60d41140d496901ad4a16e6d4d P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.935309   431 tablet_replica.cc:333] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.936340   431 raft_consensus.cc:2243] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.937512   431 raft_consensus.cc:2272] T a165cfdf51b34a5ea1bb6ece57f06bbc P 8b2c3934407d422495728f84374c6b53 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.940512   431 tablet_replica.cc:333] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.941391   431 raft_consensus.cc:2243] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.942554   431 raft_consensus.cc:2272] T a986b2c96ad0416f8fc5927d45ebadc4 P 8b2c3934407d422495728f84374c6b53 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.945346   431 tablet_replica.cc:333] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.946242   431 raft_consensus.cc:2243] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.947600   431 raft_consensus.cc:2272] T c2dc331078c54a6aa71b5eab7cb93250 P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.951650   431 tablet_replica.cc:333] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.952503   431 raft_consensus.cc:2243] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.953264   431 raft_consensus.cc:2272] T ec944cb8c5da4037a28a6917759780b4 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.956218   431 tablet_replica.cc:333] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.956980   431 raft_consensus.cc:2243] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.957952   431 raft_consensus.cc:2272] T 9df64a488003414aac04146e56487c81 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.960410   431 tablet_replica.cc:333] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.961226   431 raft_consensus.cc:2243] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.962486   431 raft_consensus.cc:2272] T 9f032b031edf4e5eb9736c18f8ba0c03 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.965402   431 tablet_replica.cc:333] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.966154   431 raft_consensus.cc:2243] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.967087   431 raft_consensus.cc:2272] T 21235338303a443e8f10b8773c68e2db P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.969986   431 tablet_replica.cc:333] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.970579   431 raft_consensus.cc:2243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.971606   431 raft_consensus.cc:2272] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.974231   431 tablet_replica.cc:333] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.975054   431 raft_consensus.cc:2243] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.976305   431 raft_consensus.cc:2272] T 62c0efbdda23450cb269a39d62c15422 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.978688   431 tablet_replica.cc:333] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.979468   431 raft_consensus.cc:2243] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.980089   431 raft_consensus.cc:2272] T 288b67d6ad124d2aa18d641ed4e1f3df P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.982635   431 tablet_replica.cc:333] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.983450   431 raft_consensus.cc:2243] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 27 LEADER]: Raft consensus shutting down.
W20260709 13:04:51.984184  1674 consensus_peers.cc:597] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.984737   431 raft_consensus.cc:2272] T c9744e1e31604277953315481c56b171 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.986985   431 tablet_replica.cc:333] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.987576   431 raft_consensus.cc:2243] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:51.988181   431 raft_consensus.cc:2272] T 41b83284fac147339ac4124c2eedee11 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.990298   431 tablet_replica.cc:333] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.990844   431 raft_consensus.cc:2243] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.991883   431 raft_consensus.cc:2272] T 4481ad55e16e48b7a3442111d9636f55 P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.993971   431 tablet_replica.cc:333] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.994599   431 raft_consensus.cc:2243] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.995550   431 raft_consensus.cc:2272] T dea1928bdd634efaa34550f21ff564f3 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:51.998042   431 tablet_replica.cc:333] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:51.998807   431 raft_consensus.cc:2243] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:51.999972   431 raft_consensus.cc:2272] T d6beee9f16eb4c7db2d293d74dfec496 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.004473   431 tablet_replica.cc:333] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.005381   431 raft_consensus.cc:2243] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.006654   431 raft_consensus.cc:2272] T 9d254c50f08d431298fe57ef1dde64be P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.009618   431 tablet_replica.cc:333] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.010429   431 raft_consensus.cc:2243] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.011749   431 raft_consensus.cc:2272] T 77308d0ddc334cbfa9170c6422895ca9 P 8b2c3934407d422495728f84374c6b53 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.014438   431 tablet_replica.cc:333] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.015218   431 raft_consensus.cc:2243] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.015889   431 raft_consensus.cc:2272] T af498956d398433a8c293e398bd1255f P 8b2c3934407d422495728f84374c6b53 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.018698   431 tablet_replica.cc:333] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
W20260709 13:04:52.018532  1674 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:52.019670   431 raft_consensus.cc:2243] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.020855   431 raft_consensus.cc:2272] T 61539cc4a0c645ed9fb29927e85f3e20 P 8b2c3934407d422495728f84374c6b53 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.023756   431 tablet_replica.cc:333] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.024645   431 raft_consensus.cc:2243] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.025229   431 raft_consensus.cc:2272] T e629c77df1d84117a78341cd5a55b213 P 8b2c3934407d422495728f84374c6b53 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.028281   431 tablet_replica.cc:333] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.029089   431 raft_consensus.cc:2243] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.030277   431 raft_consensus.cc:2272] T 8fb72ffc01144a2880010ee54d6e6c44 P 8b2c3934407d422495728f84374c6b53 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:52.030344  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:52.032811   431 tablet_replica.cc:333] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.033391   431 raft_consensus.cc:2243] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.034293   431 raft_consensus.cc:2272] T 2139fc31dcac45ddbcc3f5046bd79ade P 8b2c3934407d422495728f84374c6b53 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.036648   431 tablet_replica.cc:333] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.037180   431 raft_consensus.cc:2243] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.037717   431 raft_consensus.cc:2272] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 8b2c3934407d422495728f84374c6b53 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.039825   431 tablet_replica.cc:333] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.040338   431 raft_consensus.cc:2243] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.040840   431 raft_consensus.cc:2272] T ab336cb98e24470eb828d9e9fdbcb001 P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.042901   431 tablet_replica.cc:333] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.043426   431 raft_consensus.cc:2243] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.043989   431 raft_consensus.cc:2272] T e27b39f413f94e9192458b7bef0a86b7 P 8b2c3934407d422495728f84374c6b53 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.045909   431 tablet_replica.cc:333] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53: stopping tablet replica
I20260709 13:04:52.046476   431 raft_consensus.cc:2243] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:52.047688   431 raft_consensus.cc:2272] T 5e0479a2b6a6473eac54a01fde6fc25c P 8b2c3934407d422495728f84374c6b53 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:52.050989  2262 consensus_peers.cc:597] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:52.176694  1730 debug-util.cc:398] Leaking SignalData structure 0x7b0800388ae0 after lost signal to thread 434
W20260709 13:04:53.093194 10280 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.011s	sys 0.009s
W20260709 13:04:53.212327 10280 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.012s	sys 0.009s
W20260709 13:04:53.212915  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08001cb940 after lost signal to thread 1492
W20260709 13:04:53.214092  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08002fff00 after lost signal to thread 1659
W20260709 13:04:53.215021  1730 debug-util.cc:398] Leaking SignalData structure 0x7b080033cac0 after lost signal to thread 1733
W20260709 13:04:53.219940  1730 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4be0 after lost signal to thread 2321
I20260709 13:04:53.225986 10296 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.226743 10296 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.229065 10296 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.229627 10296 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.230647  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54540 (request call id 254) took 1164 ms. Trace:
I20260709 13:04:53.232434  1456 rpcz_store.cc:276] 0709 13:04:52.066120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.066298 (+   178us) service_pool.cc:224] Handling call
0709 13:04:53.230428 (+1164130us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:04:53.234022 10296 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.241827 10292 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.242297 10296 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.231456 10297 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.242488 10292 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.243146 10297 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.245720 10297 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.238162 10290 raft_consensus.cc:493] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.257838 10290 raft_consensus.cc:515] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.236055 10294 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.269506 10299 raft_consensus.cc:493] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.269759 10292 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.270780 10290 leader_election.cc:290] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.270284 10299 raft_consensus.cc:515] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.273451 10299 leader_election.cc:290] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.273926 10299 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.274398 10299 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.276213 10299 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.276386 10290 raft_consensus.cc:493] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.268554 10294 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:53.280530 10294 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.276990 10290 raft_consensus.cc:515] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.286247 10290 leader_election.cc:290] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.288079 10290 raft_consensus.cc:493] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.288647 10290 raft_consensus.cc:515] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.291559 10290 leader_election.cc:290] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.298861 10300 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.301442 10303 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.302053 10303 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.305154 10290 raft_consensus.cc:493] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.299670 10300 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.308089 10295 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.308679 10295 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.309386 10300 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.310829 10295 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.312309 10294 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.313055 10294 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.315255 10294 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.315743 10294 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.315913 10297 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.318437 10300 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.318971 10297 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.319075 10300 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.321364 10300 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.321424 10297 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.322074 10297 raft_consensus.cc:493] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.322556 10297 raft_consensus.cc:515] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.317412 10302 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.316161 10304 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.323859 10302 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.324564 10304 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.326268 10302 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.326951 10304 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.317823 10294 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.331945 10294 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.332386 10303 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:53.340507  1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111) [suppressed 387 similar messages]
I20260709 13:04:53.349476 10295 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.358202 10295 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.361275 10295 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.363368 10301 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.347370 10302 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.351132 10300 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.353613  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.365173 10290 raft_consensus.cc:515] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.365909  2292 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 27.
I20260709 13:04:53.365763 10300 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.354360  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41b83284fac147339ac4124c2eedee11" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.354628  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4b89801fe624f179701cbd4b98f1490" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.355424  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.368876 10300 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.364301 10297 leader_election.cc:290] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.369555 10302 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.350325 10299 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.374626 10299 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.375002  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab7f997f0f54ab6bfebcbbd01c136d6" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.377152 10299 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.377861  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:53.378862  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78699c8b654944aa9788917e2a177d14" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.379078  2294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9438873b8e1a4f0780fff8b93a9d89b8" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.381629 10305 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:53.382284 10305 raft_consensus.cc:493] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.382766 10305 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:53.387997 10302 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.395211 10308 raft_consensus.cc:493] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.395906 10280 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.397109 10308 raft_consensus.cc:515] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.396847 10290 leader_election.cc:290] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.397184 10280 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.400362 10280 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.401041 10290 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.401561 10290 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.403553 10290 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.404333 10290 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.404870 10290 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.406036 10280 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.406282 10308 leader_election.cc:290] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.412075 10305 raft_consensus.cc:515] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.458488 10280 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.460712 10280 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.420030 10298 raft_consensus.cc:493] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.461449 10298 raft_consensus.cc:515] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.462745 10290 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.464011 10305 leader_election.cc:290] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.465386 10298 leader_election.cc:290] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.444681 10293 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.466224 10298 raft_consensus.cc:493] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.466450 10293 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.470288 10293 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.365193 10301 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.444053 10296 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.471810 10305 raft_consensus.cc:493] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.472460 10305 raft_consensus.cc:515] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.474720 10305 leader_election.cc:290] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.445120 10304 raft_consensus.cc:493] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.472294 10296 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.475457 10304 raft_consensus.cc:515] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.477389 10296 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.466823 10298 raft_consensus.cc:515] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.480033 10298 leader_election.cc:290] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.481180 10298 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.481812 10298 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.483906 10294 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.477595 10304 leader_election.cc:290] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.484443 10294 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.484701 10304 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.485451 10304 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.486431 10294 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.486960 10294 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.487490 10298 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.487897 10296 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.490379 10296 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.487389 10294 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.493104 10296 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.488320 10305 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.493850 10294 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.494037 10305 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.496730 10305 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.499850 10280 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.500406 10280 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.502444 10280 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.504137 10296 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.504748 10280 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.504910 10310 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.505329 10280 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.505589 10310 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:53.505014 10301 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.489215 10304 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.507690 10310 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.508067 10280 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.504778 10296 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:53.510030  2262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:39515, user_credentials={real_user=slave}} blocked reactor thread for 131051us
I20260709 13:04:53.516360 10305 raft_consensus.cc:493] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2579bae7bf1543bf9b10d68e93a6ebbb)
I20260709 13:04:53.523280 10305 raft_consensus.cc:515] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.525468 10305 leader_election.cc:290] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.532645 10280 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.533331 10280 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.533882  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd3b4e5c2bf43a7a1a186ee984fa7c4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:53.534497  2291 raft_consensus.cc:3060] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:53.535660  2288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23585cacd4d249b4aa2ca97e2fb3ea8d" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.536715  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "288b67d6ad124d2aa18d641ed4e1f3df" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.522675 10296 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:53.539043 10301 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.503796 10294 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.535899 10280 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.539561 10301 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.540117  2293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27b39f413f94e9192458b7bef0a86b7" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.540040 10294 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:53.540591  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.541076  2263 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.541755  2263 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.542032 10294 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:53.542174  2263 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.519356  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:53.542531 10280 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:53.542253  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c25c5b342d344f3ac325ab5073d1747" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:53.543190  1675 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.543309 10301 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.544292  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab336cb98e24470eb828d9e9fdbcb001" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.544605  2292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
I20260709 13:04:53.544883  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:53.543715  1675 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.545213  2292 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 25.
I20260709 13:04:53.545447  2296 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 21.
I20260709 13:04:53.543828 10280 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.546101  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd868018972045628b3604762f904e05" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" is_pre_election: true
W20260709 13:04:53.545578  1675 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.547317  1675 leader_election.cc:336] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.547812  1675 leader_election.cc:304] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.548005  2291 raft_consensus.cc:2468] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 28.
I20260709 13:04:53.548435 10280 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:53.550134  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:53.549081 10305 raft_consensus.cc:2749] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.550621  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de80dd9c151b41e88e1469985d074b81" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:53.552469  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.552567  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
W20260709 13:04:53.553243  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.553560 10294 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:04:53.553814  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.554078 10294 raft_consensus.cc:493] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.556244  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:53.560292 10305 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:53.560729 10305 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.561107 10305 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:53.556828  2263 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.562230  2263 leader_election.cc:336] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.562825  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.563295  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.565685  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.560988  1673 leader_election.cc:304] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:53.568780 10305 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:53.570398  2262 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.568378 10294 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:53.571209  2262 leader_election.cc:336] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.572140  2262 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.573233  2262 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.574095  2262 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.574903  2262 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.569554 10296 raft_consensus.cc:2804] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:53.575448  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.576395 10280 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:53.577771  2262 leader_election.cc:336] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.578492  2262 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.579531  2262 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.579962  2262 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.580731 10280 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.581590  2262 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.582048  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.582815 10280 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.585666  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d92cdb4b624c3bbc074afe1340b761" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.577045 10296 raft_consensus.cc:697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 28 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:53.588757 10296 consensus_queue.cc:237] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.592166  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1d1686090d4b3abcf51f94f1a798d4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.593107  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.593943  1702 raft_consensus.cc:2393] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 22: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
W20260709 13:04:53.570240  1675 leader_election.cc:336] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.594497  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88dfd27965e145dcb810423d7396b403" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.594861  1675 leader_election.cc:304] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.594992  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce44b855d6d442778079778cd0e68c34" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.593719  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26aae1e2780d42cda794411c9c1c0f32" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.595538  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
I20260709 13:04:53.596165  1705 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 19.
I20260709 13:04:53.596321 10310 raft_consensus.cc:2749] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:53.597766  1675 leader_election.cc:336] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.598031  2262 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.598196  1675 leader_election.cc:304] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.598907 10310 raft_consensus.cc:2749] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:53.599207  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:53.596271  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a986b2c96ad0416f8fc5927d45ebadc4" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
W20260709 13:04:53.599898  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.600327  2291 raft_consensus.cc:3060] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:53.600916  1675 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.601390  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.601720  1675 leader_election.cc:336] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.601928  1675 leader_election.cc:304] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.602507 10310 raft_consensus.cc:2749] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:53.602718  1675 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.602967  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35904a4f4e3747dd85997b0dae5ae7bc" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c" is_pre_election: true
W20260709 13:04:53.603212  1675 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.603565  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.604486  2263 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.606775  2263 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.607944 10305 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.610173  2291 raft_consensus.cc:2468] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 22.
W20260709 13:04:53.610188  1675 leader_election.cc:336] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.611222  2263 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:53.616523  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.622218  2262 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.622857  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.621055  1673 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:53.623842 10295 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.624490  2263 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.624511 10305 raft_consensus.cc:2804] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:53.625144  2263 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.625309 10305 raft_consensus.cc:697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
W20260709 13:04:53.626174  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.626720  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.626375 10305 consensus_queue.cc:237] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.627096  2263 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.628000 10301 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:53.628639 10301 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:53.629179 10301 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:53.615588  2260 leader_election.cc:304] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:53.630205  2263 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.630630  2263 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.632011  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
W20260709 13:04:53.632241  1675 leader_election.cc:336] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.632653  1675 leader_election.cc:304] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.633282 10310 raft_consensus.cc:2749] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.633560  1675 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.632146 10280 raft_consensus.cc:2749] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.636451  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.637012  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.630641  2262 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.644423 10308 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:53.646184  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.647006 10302 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:53.647830 10295 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:53.646461  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.649456 10302 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:53.649768  2262 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.650183  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.650882 10302 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.654649  2262 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.659734  2263 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.659490 10294 raft_consensus.cc:515] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:53.660409  2263 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:53.661226  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:53.661490  1703 raft_consensus.cc:2393] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2579bae7bf1543bf9b10d68e93a6ebbb in current term 26: Already voted for candidate 00f3fbc60b884078a3c219909b44f64c in this term.
I20260709 13:04:53.662441 10294 leader_election.cc:290] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196:44509)
I20260709 13:04:53.701123  1455 catalog_manager.cc:5697] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 27 to 28, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 28 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:53.714201  2262 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.715343  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.717713 10302 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.718605  2262 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.720920  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.726823  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.722198  2291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4481ad55e16e48b7a3442111d9636f55" candidate_uuid: "00f3fbc60b884078a3c219909b44f64c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb"
I20260709 13:04:53.728953 10302 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.744472  2291 raft_consensus.cc:3060] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:53.747113 10302 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.749907  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.751087 10302 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.754204  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.755417 10302 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:53.757748  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.760349 10301 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:53.760886 10301 raft_consensus.cc:493] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:53.761404 10301 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:53.777253  2260 leader_election.cc:304] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:53.784214 10295 raft_consensus.cc:2749] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:53.787613  1674 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.788892  1674 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.789575  1674 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.790755 10310 raft_consensus.cc:2749] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.793354  1674 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.793951  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.794975 10310 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:53.795297  2260 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:53.796972  1674 leader_election.cc:336] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.782028  2291 raft_consensus.cc:2468] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f3fbc60b884078a3c219909b44f64c in term 26.
I20260709 13:04:53.798763 10295 raft_consensus.cc:2749] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:53.798823  1673 leader_election.cc:304] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 
I20260709 13:04:53.800587 10310 raft_consensus.cc:2804] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:53.782563 10301 raft_consensus.cc:515] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:53.801237 10310 raft_consensus.cc:697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 LEADER]: Becoming Leader. State: Replica: 00f3fbc60b884078a3c219909b44f64c, State: Running, Role: LEADER
I20260709 13:04:53.802986 10310 consensus_queue.cc:237] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:53.827340  1675 consensus_peers.cc:597] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.827067  1674 leader_election.cc:336] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.830178 10301 leader_election.cc:290] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Requested vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:53.830241  1674 leader_election.cc:304] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.832031 10305 raft_consensus.cc:2749] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.833885  2262 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.834766  1674 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.839702  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.841035 10294 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:53.841799  1674 leader_election.cc:336] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.842547  1674 leader_election.cc:304] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.843612 10294 raft_consensus.cc:2749] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:53.845898  1674 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.853720  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.854998 10310 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:53.858263  1674 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.858896  1674 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.859969 10310 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.863984  1674 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.864665  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.896823 10310 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:53.898891  1674 leader_election.cc:336] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.899420  1674 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.900312 10310 raft_consensus.cc:2749] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.916075  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:04:53.917965  1456 catalog_manager.cc:5697] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 21 to 22, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 22 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:53.920470  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:53.903107  1674 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.920679  1456 catalog_manager.cc:5697] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c reported cstate change: term changed from 25 to 26, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 00f3fbc60b884078a3c219909b44f64c (127.0.107.195). New cstate: current_term: 26 leader_uuid: "00f3fbc60b884078a3c219909b44f64c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.921783  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.922832 10310 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:53.924387  1674 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.932637  1674 leader_election.cc:336] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.934092  1674 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.934607  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.935547 10310 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.936293  1674 leader_election.cc:336] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.936781  1674 leader_election.cc:304] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.937990 10310 raft_consensus.cc:2749] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:53.946156  1674 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.946719  1674 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.947487 10310 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.949133  1674 leader_election.cc:336] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.949680  1674 leader_election.cc:304] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.950555 10310 raft_consensus.cc:2749] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:53.952193  1674 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.952709  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.953523 10310 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:53.955163  1674 leader_election.cc:336] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.955600  1674 leader_election.cc:304] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 2579bae7bf1543bf9b10d68e93a6ebbb, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.956416 10310 raft_consensus.cc:2749] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:53.958199  1674 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.958904  1674 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.968935 10310 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:53.971105  1674 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.971825  1674 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.972685 10310 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:53.974455  1674 leader_election.cc:336] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.974979  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.975754 10310 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:53.977186  1674 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.977718  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:53.978566 10310 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:53.988456  1674 leader_election.cc:336] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.992070  1674 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:53.998083  1674 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:53.998445  2291 raft_consensus.cc:1275] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:04:53.998973  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:54.001734 10310 consensus_queue.cc:1048] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 13:04:54.012563 10310 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:54.014382  1674 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:54.015009  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:54.015878 10310 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:54.018503  1674 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:54.019217  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:54.022701  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:54.027818 10310 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:54.029531  1674 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:54.031526  1674 connection.cc:769] client connection to 127.0.107.194:39515 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:04:54.032060  1674 leader_election.cc:336] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:04:54.107558  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61539cc4a0c645ed9fb29927e85f3e20" candidate_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "00f3fbc60b884078a3c219909b44f64c"
I20260709 13:04:54.108582  1703 raft_consensus.cc:3060] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:54.110984  1675 consensus_peers.cc:597] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.119021  1703 raft_consensus.cc:2468] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2579bae7bf1543bf9b10d68e93a6ebbb in term 20.
W20260709 13:04:54.130834  1674 consensus_peers.cc:597] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:54.131421  1674 consensus_peers.cc:597] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:54.133342  2260 leader_election.cc:304] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c, 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 8b2c3934407d422495728f84374c6b53
I20260709 13:04:54.134635 10289 raft_consensus.cc:2804] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:54.135195 10289 raft_consensus.cc:697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Becoming Leader. State: Replica: 2579bae7bf1543bf9b10d68e93a6ebbb, State: Running, Role: LEADER
I20260709 13:04:54.136318 10289 consensus_queue.cc:237] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:54.158663  2291 raft_consensus.cc:1275] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:04:54.160709 10310 consensus_queue.cc:1048] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:54.160748  1456 catalog_manager.cc:5697] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb reported cstate change: term changed from 19 to 20, leader changed from 8b2c3934407d422495728f84374c6b53 (127.0.107.194) to 2579bae7bf1543bf9b10d68e93a6ebbb (127.0.107.196). New cstate: current_term: 20 leader_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.193912  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.303731  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.306469  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08003bbfe0 after lost signal to thread 434
W20260709 13:04:54.308112  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0660 after lost signal to thread 1492
W20260709 13:04:54.308899  2318 debug-util.cc:398] Leaking SignalData structure 0x7b080003d260 after lost signal to thread 1659
W20260709 13:04:54.310487  2318 debug-util.cc:398] Leaking SignalData structure 0x7b080039cbe0 after lost signal to thread 1733
W20260709 13:04:54.312353  2318 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd920 after lost signal to thread 2321
I20260709 13:04:54.355290  2291 raft_consensus.cc:1275] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Refusing update from remote peer 00f3fbc60b884078a3c219909b44f64c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:04:54.361778 10305 consensus_queue.cc:1048] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:04:54.405771  1674 consensus_peers.cc:597] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:54.424566  1675 consensus_peers.cc:597] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:54.510253  1675 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.516237  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:54.621078  1675 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:04:54.622952  1703 raft_consensus.cc:1275] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Refusing update from remote peer 2579bae7bf1543bf9b10d68e93a6ebbb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:54.624567 10289 consensus_queue.cc:1048] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:04:54.652243  2262 consensus_peers.cc:597] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:54.920498  1675 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:54.929409  2262 consensus_peers.cc:597] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.059669  1675 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:04:55.159440 10301 raft_consensus.cc:493] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.160091 10301 raft_consensus.cc:515] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.162216 10301 leader_election.cc:290] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.172250  2263 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.173090  2262 leader_election.cc:336] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.173527  2262 leader_election.cc:304] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.174862 10301 raft_consensus.cc:2749] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:55.261969  2263 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:55.264022  2262 consensus_peers.cc:597] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.289481  2262 consensus_peers.cc:597] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.292323  2263 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:04:55.293128 10301 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.295831 10301 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.299149 10301 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.315374  2263 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.318259  2262 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.319551  2262 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:55.327250  1674 consensus_peers.cc:597] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.318991 10301 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.328580 10301 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.340101 10301 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.341660 10334 raft_consensus.cc:493] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.346266 10334 raft_consensus.cc:515] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.348812 10334 leader_election.cc:290] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.351490 10301 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:55.356837  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.358201 10299 raft_consensus.cc:493] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.358875 10299 raft_consensus.cc:515] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.361189 10299 leader_election.cc:290] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.392338  1674 consensus_peers.cc:597] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.393343 10299 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.394151 10299 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.396543 10299 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.406458 10328 raft_consensus.cc:493] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.413184 10328 raft_consensus.cc:515] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.415797 10328 leader_election.cc:290] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.424332 10299 raft_consensus.cc:493] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.446702 10299 raft_consensus.cc:515] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.425971 10301 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.448191 10301 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.450570 10301 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.433089  1675 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.453548 10299 leader_election.cc:290] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.467669  1674 leader_election.cc:336] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.470074  1674 leader_election.cc:304] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:55.471160  2262 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.473178 10310 raft_consensus.cc:2749] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:55.473619 10328 raft_consensus.cc:493] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.489257 10328 raft_consensus.cc:515] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.490105  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.491010 10299 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:55.493281  2263 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.500335  2262 leader_election.cc:336] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.491597 10328 leader_election.cc:290] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.502578  1675 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.502796  2262 leader_election.cc:304] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.504688 10299 raft_consensus.cc:2749] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:55.511742  2263 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.515076  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.514987  1675 consensus_peers.cc:597] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:55.516108  1674 leader_election.cc:336] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.520764  1674 leader_election.cc:304] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.525163 10336 raft_consensus.cc:2749] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:55.477109 10301 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:55.518211  2262 leader_election.cc:336] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:55.520272  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.529476  2262 leader_election.cc:304] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:55.529626  2263 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.530323  2263 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.531134 10299 raft_consensus.cc:2749] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:55.530346  2262 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.532421  2262 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.532866  2262 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.533629 10335 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:55.528679 10301 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.539288 10336 raft_consensus.cc:493] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.539408 10328 raft_consensus.cc:493] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.539927 10336 raft_consensus.cc:515] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.540055 10328 raft_consensus.cc:515] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.542140 10336 leader_election.cc:290] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.542300 10328 leader_election.cc:290] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.542796  2263 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.549639 10301 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.550408  1675 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.552727  1675 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.559814  1674 leader_election.cc:336] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.560442  1674 leader_election.cc:304] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.561275 10328 raft_consensus.cc:2749] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:55.563567  2262 consensus_peers.cc:597] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.565128  2262 consensus_peers.cc:597] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.565652  2262 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.566112  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.566929 10301 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:55.568106  2262 leader_election.cc:336] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.568612  2262 leader_election.cc:304] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.569460 10301 raft_consensus.cc:2749] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:55.563524  1674 leader_election.cc:336] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.574447  1674 leader_election.cc:304] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:55.574558  2262 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.575397  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.576012 10328 raft_consensus.cc:2749] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:55.580034 10301 raft_consensus.cc:2749] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:55.588750  1674 consensus_peers.cc:597] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.599905  1674 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.600944 10328 raft_consensus.cc:493] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.601631 10328 raft_consensus.cc:515] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.603926 10328 leader_election.cc:290] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.613226  1675 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.622180  1674 leader_election.cc:336] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.623013  1674 leader_election.cc:304] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.623862 10328 raft_consensus.cc:2749] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:55.661774  1675 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:55.667845  1674 consensus_peers.cc:597] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.670181   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:04:55.674733  1674 consensus_peers.cc:597] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.709595 10310 raft_consensus.cc:493] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.710302 10310 raft_consensus.cc:515] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.727053 10310 leader_election.cc:290] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.730016  1674 consensus_peers.cc:597] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.737116  1675 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.738504  1674 leader_election.cc:336] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.738973  1674 leader_election.cc:304] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.752208 10336 raft_consensus.cc:2749] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:55.753901 10336 raft_consensus.cc:493] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.754402 10336 raft_consensus.cc:515] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:55.763872  1675 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.760468 10336 leader_election.cc:290] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.767092 10336 raft_consensus.cc:493] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.767784 10336 raft_consensus.cc:515] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:55.778731  1675 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.785403 10336 leader_election.cc:290] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.791487  1674 leader_election.cc:336] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.792090  1674 leader_election.cc:304] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.792913 10336 raft_consensus.cc:2749] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:55.797474  1674 leader_election.cc:336] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.798269  1674 leader_election.cc:304] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.798991 10336 raft_consensus.cc:2749] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:55.864089 10301 raft_consensus.cc:493] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.864758 10301 raft_consensus.cc:515] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:55.866813 10301 leader_election.cc:290] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.870131 10336 raft_consensus.cc:493] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.870738 10336 raft_consensus.cc:515] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.873044 10336 leader_election.cc:290] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.874563  2263 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.877981   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:04:55.883428  1675 leader_election.cc:336] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.884522  1674 leader_election.cc:336] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.885022  1674 leader_election.cc:304] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.890033 10328 raft_consensus.cc:2749] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:55.891888 10328 raft_consensus.cc:493] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.892359 10328 raft_consensus.cc:515] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:55.892719  2262 leader_election.cc:336] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.893185  2262 leader_election.cc:304] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.894582 10328 leader_election.cc:290] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:55.894542 10301 raft_consensus.cc:2749] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:55.901935  2260 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:55.902752  1674 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:55.903340  1675 leader_election.cc:336] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:55.903807  1675 leader_election.cc:304] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.905076 10336 raft_consensus.cc:2749] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:55.916898 10336 raft_consensus.cc:493] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.917474 10336 raft_consensus.cc:515] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } }
I20260709 13:04:55.919591 10336 leader_election.cc:290] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.925625  1675 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.929261  1674 leader_election.cc:336] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.929796  1674 leader_election.cc:304] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.930596 10336 raft_consensus.cc:2749] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.937680 10336 raft_consensus.cc:493] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:55.938159 10336 raft_consensus.cc:515] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:55.940292 10336 leader_election.cc:290] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:55.947917  1675 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:55.955566  1674 leader_election.cc:336] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:55.958653  1674 leader_election.cc:304] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:55.959343 10336 raft_consensus.cc:2749] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:55.972726  1674 consensus_peers.cc:597] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.977250  2260 consensus_peers.cc:597] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.009497  1675 consensus_peers.cc:597] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:04:56.011991 10336 raft_consensus.cc:493] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:56.012579 10336 raft_consensus.cc:515] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
W20260709 13:04:56.013113  1675 consensus_peers.cc:597] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:56.010182   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:56.014952 10336 leader_election.cc:290] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
I20260709 13:04:56.017536   431 tablet_replica.cc:333] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.019712   431 raft_consensus.cc:2243] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.021023   431 raft_consensus.cc:2272] T de80dd9c151b41e88e1469985d074b81 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.022630  1674 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:56.025193 10328 raft_consensus.cc:493] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:56.025749 10328 raft_consensus.cc:515] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:56.029570 10328 leader_election.cc:290] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:56.031179  2262 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:56.032420   431 tablet_replica.cc:333] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.034578   431 raft_consensus.cc:2243] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.035302   431 raft_consensus.cc:2272] T e43ddec31b504a3b9c8f06a96cd7013d P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.036118  1674 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:56.036578 10328 raft_consensus.cc:493] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
W20260709 13:04:56.033730  1675 leader_election.cc:336] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:56.037179 10328 raft_consensus.cc:515] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:56.037551  1675 leader_election.cc:304] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:56.038874 10336 raft_consensus.cc:2749] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:56.042155 10328 leader_election.cc:290] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:56.049374  1674 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:56.050705  2260 consensus_peers.cc:597] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.050781  2263 consensus_peers.cc:597] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:56.053361   431 tablet_replica.cc:333] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.054381   431 raft_consensus.cc:2243] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.055397   431 raft_consensus.cc:2272] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.059285  1675 leader_election.cc:336] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:56.059876  1675 leader_election.cc:304] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:56.061087 10328 raft_consensus.cc:2749] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:56.066401  1675 leader_election.cc:336] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:56.066936  1675 leader_election.cc:304] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:56.067588  1675 raft_consensus.cc:2686] T bcf9e0da0f934b968a7e9da19f0e0749 P 00f3fbc60b884078a3c219909b44f64c: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:04:56.070055  2260 consensus_peers.cc:597] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.071171   431 tablet_replica.cc:333] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.072290   431 raft_consensus.cc:2243] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.073061   431 raft_consensus.cc:2272] T 78699c8b654944aa9788917e2a177d14 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.076076   431 tablet_replica.cc:333] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.077257   431 raft_consensus.cc:2243] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.077971   431 raft_consensus.cc:2272] T 0d703c11931d4a2688046a3a48b71290 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.080927   431 tablet_replica.cc:333] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.081743   431 raft_consensus.cc:2243] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.082993   431 raft_consensus.cc:2272] T 7f8c2a4b167d431aac3de9d8166bab1f P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.086540   431 tablet_replica.cc:333] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.093083   431 raft_consensus.cc:2243] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.093827   431 raft_consensus.cc:2272] T 3f0ffe513d79430292b928b5fcde37fe P 00f3fbc60b884078a3c219909b44f64c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.097052   431 tablet_replica.cc:333] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.097932   431 raft_consensus.cc:2243] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.099033   431 raft_consensus.cc:2272] T 4bcf131126594cae9b8c0f94d39caaf8 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.117692 10328 raft_consensus.cc:493] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:56.118464   431 tablet_replica.cc:333] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.118301 10328 raft_consensus.cc:515] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:56.119374   431 raft_consensus.cc:2243] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.120653 10328 leader_election.cc:290] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:56.123112  2260 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.124811  1675 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:56.125607  2260 consensus_peers.cc:597] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.129978  1674 consensus_peers.cc:597] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.130489  1674 leader_election.cc:336] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:56.130929  1674 leader_election.cc:304] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:56.132143 10310 raft_consensus.cc:2749] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:56.144729  2260 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.146049   431 raft_consensus.cc:2272] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.150096   431 tablet_replica.cc:333] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.151003   431 raft_consensus.cc:2243] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.152278   431 raft_consensus.cc:2272] T 3d1d1686090d4b3abcf51f94f1a798d4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.155373   431 tablet_replica.cc:333] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.156260   431 raft_consensus.cc:2243] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.157024   431 raft_consensus.cc:2272] T c4b89801fe624f179701cbd4b98f1490 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.159780   431 tablet_replica.cc:333] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.160674   431 raft_consensus.cc:2243] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.162690   431 raft_consensus.cc:2272] T f9d92cdb4b624c3bbc074afe1340b761 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.165815   431 tablet_replica.cc:333] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.166658   431 raft_consensus.cc:2243] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.167644   431 raft_consensus.cc:2272] T 2292aebe10c64bc99c27e3976075c309 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.170426   431 tablet_replica.cc:333] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.171262   431 raft_consensus.cc:2243] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.172616   431 raft_consensus.cc:2272] T 1d5f3bc0667d44e69539243d156dac36 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.177076   431 tablet_replica.cc:333] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.177989   431 raft_consensus.cc:2243] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.179123   431 raft_consensus.cc:2272] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 00f3fbc60b884078a3c219909b44f64c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.181970   431 tablet_replica.cc:333] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.186733   431 raft_consensus.cc:2243] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.187469   431 raft_consensus.cc:2272] T e008100b0d36427c939df1036b5e0b67 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.190411  1675 consensus_peers.cc:597] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:56.190622  2260 consensus_peers.cc:597] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.191901 10310 raft_consensus.cc:493] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:56.192548 10310 raft_consensus.cc:515] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } }
I20260709 13:04:56.195340 10310 leader_election.cc:290] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:56.201027  1674 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:56.207688  1675 leader_election.cc:336] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:56.210289  1675 leader_election.cc:304] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 00f3fbc60b884078a3c219909b44f64c; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:56.211611 10328 raft_consensus.cc:2749] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:56.224754  2263 consensus_peers.cc:597] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:56.226667   431 tablet_replica.cc:333] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.227638   431 raft_consensus.cc:2243] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.228943   431 raft_consensus.cc:2272] T 1fb76969419149b19b9ea9d40ebd2c24 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.232149   431 tablet_replica.cc:333] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.233018   431 raft_consensus.cc:2243] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.233717   431 raft_consensus.cc:2272] T 9c25c5b342d344f3ac325ab5073d1747 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.235462  2260 consensus_peers.cc:597] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.237030   431 tablet_replica.cc:333] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.237942   431 raft_consensus.cc:2243] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.238982   431 raft_consensus.cc:2272] T 542a10c27e2d4480a7a262c77167ed4b P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.241864   431 tablet_replica.cc:333] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.242537   431 raft_consensus.cc:2243] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.243430   431 raft_consensus.cc:2272] T c8a9cb8cee6d447e9042f38fe30cb6e0 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.246186   431 tablet_replica.cc:333] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.246869   431 raft_consensus.cc:2243] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.247674   431 raft_consensus.cc:2272] T 012e7f60d41140d496901ad4a16e6d4d P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.252710   431 tablet_replica.cc:333] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.253515   431 raft_consensus.cc:2243] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.254221   431 raft_consensus.cc:2272] T a165cfdf51b34a5ea1bb6ece57f06bbc P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.257122   431 tablet_replica.cc:333] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.257969   431 raft_consensus.cc:2243] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.259265   431 raft_consensus.cc:2272] T 88dfd27965e145dcb810423d7396b403 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.268960  2260 consensus_peers.cc:597] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.269011   431 tablet_replica.cc:333] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.270229   431 raft_consensus.cc:2243] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.271512   431 raft_consensus.cc:2272] T a986b2c96ad0416f8fc5927d45ebadc4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.273921  2260 consensus_peers.cc:597] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.274672   431 tablet_replica.cc:333] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.275636   431 raft_consensus.cc:2243] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.277088   431 raft_consensus.cc:2272] T ec944cb8c5da4037a28a6917759780b4 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.280485   431 tablet_replica.cc:333] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.281373   431 raft_consensus.cc:2243] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.282385   431 raft_consensus.cc:2272] T 9f032b031edf4e5eb9736c18f8ba0c03 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.285290   431 tablet_replica.cc:333] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.286327   431 raft_consensus.cc:2243] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.287690   431 raft_consensus.cc:2272] T ce44b855d6d442778079778cd0e68c34 P 00f3fbc60b884078a3c219909b44f64c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.290549   431 tablet_replica.cc:333] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.291440   431 raft_consensus.cc:2243] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.292711   431 raft_consensus.cc:2272] T 35904a4f4e3747dd85997b0dae5ae7bc P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.295812   431 tablet_replica.cc:333] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.296654   431 raft_consensus.cc:2243] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.297395   431 raft_consensus.cc:2272] T bd868018972045628b3604762f904e05 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.300253   431 tablet_replica.cc:333] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.301328   431 raft_consensus.cc:2243] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.302042   431 raft_consensus.cc:2272] T 62c0efbdda23450cb269a39d62c15422 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.306107   431 tablet_replica.cc:333] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.306999   431 raft_consensus.cc:2243] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.314589   431 raft_consensus.cc:2272] T 288b67d6ad124d2aa18d641ed4e1f3df P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.317528   431 tablet_replica.cc:333] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.318424   431 raft_consensus.cc:2243] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.319120   431 raft_consensus.cc:2272] T e27b39f413f94e9192458b7bef0a86b7 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.321902   431 tablet_replica.cc:333] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.322757   431 raft_consensus.cc:2243] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.323652   431 raft_consensus.cc:2272] T 5e0479a2b6a6473eac54a01fde6fc25c P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.326650   431 tablet_replica.cc:333] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.327513   431 raft_consensus.cc:2243] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.328301   431 raft_consensus.cc:2272] T 77308d0ddc334cbfa9170c6422895ca9 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.339115   431 tablet_replica.cc:333] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.341110   431 raft_consensus.cc:2243] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.341898   431 raft_consensus.cc:2272] T ab336cb98e24470eb828d9e9fdbcb001 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.344977   431 tablet_replica.cc:333] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.345795   431 raft_consensus.cc:2243] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.346534   431 raft_consensus.cc:2272] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 00f3fbc60b884078a3c219909b44f64c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.355711   431 tablet_replica.cc:333] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.356674   431 raft_consensus.cc:2243] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.357425   431 raft_consensus.cc:2272] T c9744e1e31604277953315481c56b171 P 00f3fbc60b884078a3c219909b44f64c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.360335   431 tablet_replica.cc:333] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.361191   431 raft_consensus.cc:2243] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.362075   431 raft_consensus.cc:2272] T 41b83284fac147339ac4124c2eedee11 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.365012   431 tablet_replica.cc:333] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.365826   431 raft_consensus.cc:2243] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.367327   431 raft_consensus.cc:2272] T 4481ad55e16e48b7a3442111d9636f55 P 00f3fbc60b884078a3c219909b44f64c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.370241   431 tablet_replica.cc:333] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.371028   431 raft_consensus.cc:2243] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.371627   431 raft_consensus.cc:2272] T 61539cc4a0c645ed9fb29927e85f3e20 P 00f3fbc60b884078a3c219909b44f64c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.374325   431 tablet_replica.cc:333] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.375193   431 raft_consensus.cc:2243] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.376227   431 raft_consensus.cc:2272] T 8fb72ffc01144a2880010ee54d6e6c44 P 00f3fbc60b884078a3c219909b44f64c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.379033   431 tablet_replica.cc:333] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.379813   431 raft_consensus.cc:2243] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.380497   431 raft_consensus.cc:2272] T 9438873b8e1a4f0780fff8b93a9d89b8 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.383713   431 tablet_replica.cc:333] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.384573   431 raft_consensus.cc:2243] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.386169   431 raft_consensus.cc:2272] T e629c77df1d84117a78341cd5a55b213 P 00f3fbc60b884078a3c219909b44f64c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.389312   431 tablet_replica.cc:333] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.390136   431 raft_consensus.cc:2243] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.391486   431 raft_consensus.cc:2272] T 26aae1e2780d42cda794411c9c1c0f32 P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.394523   431 tablet_replica.cc:333] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.395308   431 raft_consensus.cc:2243] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.396183   431 raft_consensus.cc:2272] T d6beee9f16eb4c7db2d293d74dfec496 P 00f3fbc60b884078a3c219909b44f64c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.399613   431 tablet_replica.cc:333] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.400466   431 raft_consensus.cc:2243] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.402088   431 raft_consensus.cc:2272] T 309f24e44cc94815a1ad3911262a7872 P 00f3fbc60b884078a3c219909b44f64c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.405073   431 tablet_replica.cc:333] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c: stopping tablet replica
I20260709 13:04:56.405788   431 raft_consensus.cc:2243] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.406716   431 raft_consensus.cc:2272] T 2139fc31dcac45ddbcc3f5046bd79ade P 00f3fbc60b884078a3c219909b44f64c [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.416590  2260 consensus_peers.cc:597] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Couldn't send request to peer 00f3fbc60b884078a3c219909b44f64c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.823056  2263 consensus_peers.cc:597] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:56.868374 10335 raft_consensus.cc:493] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:56.868944 10335 raft_consensus.cc:515] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:56.878312  2262 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:56.883128 10335 leader_election.cc:290] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:56.883733  2263 leader_election.cc:336] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:56.884258  2263 leader_election.cc:304] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:56.885437 10335 raft_consensus.cc:2749] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:57.138486  2263 consensus_peers.cc:597] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.170647 10335 raft_consensus.cc:493] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:57.171274 10335 raft_consensus.cc:515] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.182451 10301 raft_consensus.cc:493] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:57.183076 10301 raft_consensus.cc:515] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.185161 10301 leader_election.cc:290] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:57.193488  2262 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:57.196421  2263 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.204154 10301 raft_consensus.cc:493] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.204690 10301 raft_consensus.cc:515] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.206835 10301 leader_election.cc:290] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:57.209579 10335 leader_election.cc:290] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:57.210227  2263 leader_election.cc:336] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.210685  2263 leader_election.cc:304] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:57.211547  2263 consensus_peers.cc:597] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Couldn't send request to peer 444b494bb53842b59408a7230884398a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:57.214744  2262 leader_election.cc:336] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:57.215214  2262 leader_election.cc:304] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:57.217106 10335 raft_consensus.cc:2749] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:57.217789 10335 raft_consensus.cc:2749] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:57.218092   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:04:57.223189 10301 raft_consensus.cc:493] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.223848 10301 raft_consensus.cc:515] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:57.230083  2262 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:57.232824 10301 leader_election.cc:290] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:57.238138  2263 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:57.238883  2260 leader_election.cc:336] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.241075  2260 leader_election.cc:304] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:57.242826 10301 raft_consensus.cc:2749] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:57.251173  2260 leader_election.cc:336] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.251693  2260 leader_election.cc:304] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:57.252521 10301 raft_consensus.cc:2749] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:57.258905 10301 raft_consensus.cc:493] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.259568 10301 raft_consensus.cc:515] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.263350 10301 leader_election.cc:290] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:57.270810  2263 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:57.275605  2260 leader_election.cc:336] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.276139  2260 leader_election.cc:304] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:57.276947 10301 raft_consensus.cc:2749] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.309930 10301 raft_consensus.cc:493] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.310576 10301 raft_consensus.cc:515] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.315522 10301 leader_election.cc:290] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:57.330678  2262 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:57.353719  2260 leader_election.cc:336] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.355861  2260 leader_election.cc:304] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:57.356820 10301 raft_consensus.cc:2749] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:57.368472  2262 consensus_peers.cc:597] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:57.378867   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:04:57.390000 10301 raft_consensus.cc:493] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:57.390626 10301 raft_consensus.cc:515] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.394306 10301 leader_election.cc:290] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:57.400974  2263 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.403906 10301 raft_consensus.cc:493] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.406276 10301 raft_consensus.cc:515] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } attrs { replace: true } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
W20260709 13:04:57.410972  2262 leader_election.cc:336] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:57.411543  2262 leader_election.cc:304] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:57.412381 10335 raft_consensus.cc:2749] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.411254 10301 leader_election.cc:290] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
W20260709 13:04:57.420274  2263 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:04:57.424363  2260 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.424863  2260 leader_election.cc:304] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:57.425658 10301 raft_consensus.cc:2749] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:04:57.428151  2262 leader_election.cc:336] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
W20260709 13:04:57.453483  2262 consensus_peers.cc:597] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:57.477680   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:57.480784   431 tablet_replica.cc:333] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.481873   431 raft_consensus.cc:2243] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.482997 10301 raft_consensus.cc:493] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.483187   431 raft_consensus.cc:2272] T 0ab7f997f0f54ab6bfebcbbd01c136d6 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.483597 10301 raft_consensus.cc:515] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.485877 10301 leader_election.cc:290] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:57.486533   431 tablet_replica.cc:333] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.487511   431 raft_consensus.cc:2243] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.488521   431 raft_consensus.cc:2272] T f9d92cdb4b624c3bbc074afe1340b761 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.491335  2263 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.491854   431 tablet_replica.cc:333] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.492915   431 raft_consensus.cc:2243] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.497516   431 raft_consensus.cc:2272] T c4b89801fe624f179701cbd4b98f1490 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.497994  2260 leader_election.cc:336] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.498646  2260 leader_election.cc:304] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:57.500000 10301 raft_consensus.cc:2749] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:57.501505   431 tablet_replica.cc:333] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.502597   431 raft_consensus.cc:2243] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.503273   431 raft_consensus.cc:2272] T abd3b4e5c2bf43a7a1a186ee984fa7c4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.503947 10301 raft_consensus.cc:493] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8b2c3934407d422495728f84374c6b53)
I20260709 13:04:57.504483 10301 raft_consensus.cc:515] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.506206   431 tablet_replica.cc:333] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.507345   431 raft_consensus.cc:2243] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.508724   431 raft_consensus.cc:2272] T e008100b0d36427c939df1036b5e0b67 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.512137   431 tablet_replica.cc:333] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.513197   431 raft_consensus.cc:2243] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:57.514889  2263 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.515736 10301 leader_election.cc:290] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515)
W20260709 13:04:57.515962  2262 leader_election.cc:336] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:57.516424  2262 leader_election.cc:304] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 444b494bb53842b59408a7230884398a, 8b2c3934407d422495728f84374c6b53
I20260709 13:04:57.517841   431 raft_consensus.cc:2272] T 9ce0393bceaa489d896dd65113130131 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.521027   431 tablet_replica.cc:333] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.521888   431 raft_consensus.cc:2243] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.522503   431 raft_consensus.cc:2272] T a986b2c96ad0416f8fc5927d45ebadc4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.522716 10301 raft_consensus.cc:493] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.523357 10301 raft_consensus.cc:515] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8b2c3934407d422495728f84374c6b53" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39515 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.525239   431 tablet_replica.cc:333] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.526223   431 raft_consensus.cc:2243] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.526971   431 raft_consensus.cc:2272] T c2dc331078c54a6aa71b5eab7cb93250 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.529881   431 tablet_replica.cc:333] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
W20260709 13:04:57.530581  2262 leader_election.cc:336] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111)
I20260709 13:04:57.530755   431 raft_consensus.cc:2243] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 31 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.532203   431 raft_consensus.cc:2272] T af498956d398433a8c293e398bd1255f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.532171 10301 leader_election.cc:290] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:57.535961   431 tablet_replica.cc:333] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.536937   431 raft_consensus.cc:2243] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.537777   431 raft_consensus.cc:2272] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.539153  2260 leader_election.cc:336] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.539733  2260 leader_election.cc:304] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 8b2c3934407d422495728f84374c6b53
W20260709 13:04:57.540537  2260 raft_consensus.cc:2686] T 7f8c2a4b167d431aac3de9d8166bab1f P 2579bae7bf1543bf9b10d68e93a6ebbb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:04:57.540941   431 tablet_replica.cc:333] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.541960   431 raft_consensus.cc:2243] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.543109   431 raft_consensus.cc:2272] T ab336cb98e24470eb828d9e9fdbcb001 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.545957   431 tablet_replica.cc:333] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.546761   431 raft_consensus.cc:2243] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.547863   431 raft_consensus.cc:2272] T 23585cacd4d249b4aa2ca97e2fb3ea8d P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.550940   431 tablet_replica.cc:333] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.551851   431 raft_consensus.cc:2243] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.552585   431 raft_consensus.cc:2272] T 9d254c50f08d431298fe57ef1dde64be P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.555758   431 tablet_replica.cc:333] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.556571   431 raft_consensus.cc:2243] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.558156   431 raft_consensus.cc:2272] T b1f19e2b30024564847382b872fc80e1 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.561249   431 tablet_replica.cc:333] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.562059   431 raft_consensus.cc:2243] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.562628   431 raft_consensus.cc:2272] T 88dfd27965e145dcb810423d7396b403 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.565469   431 tablet_replica.cc:333] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.566228   431 raft_consensus.cc:2243] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.566879   431 raft_consensus.cc:2272] T 7346e6ca66624e10898433387e0bbe68 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.569936   431 tablet_replica.cc:333] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.570732   431 raft_consensus.cc:2243] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.571863   431 raft_consensus.cc:2272] T 41b83284fac147339ac4124c2eedee11 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.574774   431 tablet_replica.cc:333] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.575613   431 raft_consensus.cc:2243] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.576231   431 raft_consensus.cc:2272] T 4481ad55e16e48b7a3442111d9636f55 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.578951   431 tablet_replica.cc:333] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.579737   431 raft_consensus.cc:2243] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.580906   431 raft_consensus.cc:2272] T 9438873b8e1a4f0780fff8b93a9d89b8 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.583590   431 tablet_replica.cc:333] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.584367   431 raft_consensus.cc:2243] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.584489 10335 raft_consensus.cc:493] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 00f3fbc60b884078a3c219909b44f64c)
I20260709 13:04:57.585378   431 raft_consensus.cc:2272] T bd868018972045628b3604762f904e05 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.585134 10335 raft_consensus.cc:515] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "444b494bb53842b59408a7230884398a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38031 } } peers { permanent_uuid: "00f3fbc60b884078a3c219909b44f64c" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45021 } } peers { permanent_uuid: "2579bae7bf1543bf9b10d68e93a6ebbb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 44509 } attrs { promote: false } }
I20260709 13:04:57.587425 10335 leader_election.cc:290] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 444b494bb53842b59408a7230884398a (127.0.107.193:38031), 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021)
I20260709 13:04:57.588426   431 tablet_replica.cc:333] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.590458   431 raft_consensus.cc:2243] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.591204   431 raft_consensus.cc:2272] T de80dd9c151b41e88e1469985d074b81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.593003  2263 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 444b494bb53842b59408a7230884398a (127.0.107.193:38031): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:04:57.594187   431 tablet_replica.cc:333] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.595100   431 raft_consensus.cc:2243] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.596200   431 raft_consensus.cc:2272] T 78699c8b654944aa9788917e2a177d14 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.598944   431 tablet_replica.cc:333] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.599751   431 raft_consensus.cc:2243] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.600430   431 raft_consensus.cc:2272] T 3d1d1686090d4b3abcf51f94f1a798d4 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.601007  2260 leader_election.cc:336] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
I20260709 13:04:57.601666  2260 leader_election.cc:304] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2579bae7bf1543bf9b10d68e93a6ebbb; no voters: 00f3fbc60b884078a3c219909b44f64c, 444b494bb53842b59408a7230884398a
I20260709 13:04:57.602509 10335 raft_consensus.cc:2749] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:57.603310   431 tablet_replica.cc:333] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.604104   431 raft_consensus.cc:2243] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.604818   431 raft_consensus.cc:2272] T dea1928bdd634efaa34550f21ff564f3 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.607635   431 tablet_replica.cc:333] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.608465   431 raft_consensus.cc:2243] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.609032   431 raft_consensus.cc:2272] T 26aae1e2780d42cda794411c9c1c0f32 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.611878   431 tablet_replica.cc:333] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.612550   431 raft_consensus.cc:2243] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.613216   431 raft_consensus.cc:2272] T 9b765b4fb8a24da18c3dc1ec41dd5e80 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.615903   431 tablet_replica.cc:333] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.616595   431 raft_consensus.cc:2243] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.617179   431 raft_consensus.cc:2272] T 21235338303a443e8f10b8773c68e2db P 2579bae7bf1543bf9b10d68e93a6ebbb [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.619663  2262 consensus_peers.cc:597] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb -> Peer 8b2c3934407d422495728f84374c6b53 (127.0.107.194:39515): Couldn't send request to peer 8b2c3934407d422495728f84374c6b53. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39515: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:57.619812   431 tablet_replica.cc:333] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.620728   431 raft_consensus.cc:2243] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.621366   431 raft_consensus.cc:2272] T ce44b855d6d442778079778cd0e68c34 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.624081   431 tablet_replica.cc:333] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.624795   431 raft_consensus.cc:2243] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.625584   431 raft_consensus.cc:2272] T 35904a4f4e3747dd85997b0dae5ae7bc P 2579bae7bf1543bf9b10d68e93a6ebbb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.628386   431 tablet_replica.cc:333] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.629043   431 raft_consensus.cc:2243] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.629622   431 raft_consensus.cc:2272] T 288b67d6ad124d2aa18d641ed4e1f3df P 2579bae7bf1543bf9b10d68e93a6ebbb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.632241   431 tablet_replica.cc:333] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.632902   431 raft_consensus.cc:2243] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.633986   431 raft_consensus.cc:2272] T 61539cc4a0c645ed9fb29927e85f3e20 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.636691   431 tablet_replica.cc:333] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.637352   431 raft_consensus.cc:2243] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.638455   431 raft_consensus.cc:2272] T 9c25c5b342d344f3ac325ab5073d1747 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.641093   431 tablet_replica.cc:333] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.641840   431 raft_consensus.cc:2243] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.642424   431 raft_consensus.cc:2272] T 334958c5aee349629d63e898f13ba95b P 2579bae7bf1543bf9b10d68e93a6ebbb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.645251   431 tablet_replica.cc:333] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.645921   431 raft_consensus.cc:2243] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.646580   431 raft_consensus.cc:2272] T bcd4e3d43c7d4c97a3f229082334515f P 2579bae7bf1543bf9b10d68e93a6ebbb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.649375   431 tablet_replica.cc:333] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.650058   431 raft_consensus.cc:2243] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.650718   431 raft_consensus.cc:2272] T 9df64a488003414aac04146e56487c81 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.653388   431 tablet_replica.cc:333] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb: stopping tablet replica
I20260709 13:04:57.653976   431 raft_consensus.cc:2243] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.655098   431 raft_consensus.cc:2272] T e27b39f413f94e9192458b7bef0a86b7 P 2579bae7bf1543bf9b10d68e93a6ebbb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:58.318238   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:04:58.389417   431 master.cc:562] Master@127.0.107.254:37587 shutting down...
W20260709 13:04:58.427687  1506 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:04:58.428040  1506 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:04:58.428257  1506 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:01.915560  1442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111) [suppressed 491 similar messages]
W20260709 13:05:01.919538  1442 catalog_manager.cc:4755] TS 00f3fbc60b884078a3c219909b44f64c (127.0.107.195:45021): ChangeConfig:REMOVE_PEER RPC failed for tablet f9d92cdb4b624c3bbc074afe1340b761: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45021: connect: Connection refused (error 111)
W20260709 13:05:26.839749  1445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
W20260709 13:05:26.843954  1445 catalog_manager.cc:4755] TS 444b494bb53842b59408a7230884398a (127.0.107.193:38031): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0ab7f997f0f54ab6bfebcbbd01c136d6: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38031: connect: Connection refused (error 111)
I20260709 13:05:31.607249   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:31.608384   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:31.608741   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8c0fa710867745f189aafd0e7d84fa64: stopping tablet replica
I20260709 13:05:31.838013   431 master.cc:584] Master@127.0.107.254:37587 shutdown complete.
I20260709 13:05:31.891191   431 test_util.cc:182] -----------------------------------------------
I20260709 13:05:31.891445   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601992069626-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (314822 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:05:31.895218   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36559
I20260709 13:05:31.896682   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:31.902745 10348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:31.903293 10349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:31.905885 10351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:31.907104   431 server_base.cc:1061] running on GCE node
I20260709 13:05:31.910843   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:31.911047   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:31.911170   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602331911158 us; error 0 us; skew 500 ppm
I20260709 13:05:31.911654   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:31.914247   431 webserver.cc:533] Webserver started at http://127.0.107.254:42497/ using document root <none> and password file <none>
I20260709 13:05:31.914778   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:31.915009   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:31.915294   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:31.916747   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "254dc4ec51764d408dae2397b5a8bb02"
format_stamp: "Formatted at 2026-07-09 13:05:31 on dist-test-slave-kgkf"
I20260709 13:05:31.921370   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:31.924856 10356 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:31.925649   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:31.925962   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "254dc4ec51764d408dae2397b5a8bb02"
format_stamp: "Formatted at 2026-07-09 13:05:31 on dist-test-slave-kgkf"
I20260709 13:05:31.926273   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:31.949726   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:31.951092   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:31.991899   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36559
I20260709 13:05:31.991961 10407 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36559 every 8 connection(s)
I20260709 13:05:31.996380 10408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.008811 10408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02: Bootstrap starting.
I20260709 13:05:32.013819 10408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.018471 10408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02: No bootstrap required, opened a new log
I20260709 13:05:32.020825 10408 raft_consensus.cc:359] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER }
I20260709 13:05:32.021270 10408 raft_consensus.cc:385] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.021561 10408 raft_consensus.cc:740] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254dc4ec51764d408dae2397b5a8bb02, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.022228 10408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER }
I20260709 13:05:32.022693 10408 raft_consensus.cc:399] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:32.022914 10408 raft_consensus.cc:493] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:32.023229 10408 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.028215 10408 raft_consensus.cc:515] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER }
I20260709 13:05:32.028875 10408 leader_election.cc:304] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 254dc4ec51764d408dae2397b5a8bb02; no voters: 
I20260709 13:05:32.030118 10408 leader_election.cc:290] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:32.030548 10411 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:32.032234 10411 raft_consensus.cc:697] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 1 LEADER]: Becoming Leader. State: Replica: 254dc4ec51764d408dae2397b5a8bb02, State: Running, Role: LEADER
I20260709 13:05:32.033231 10408 sys_catalog.cc:565] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:32.033138 10411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER }
I20260709 13:05:32.041204 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 254dc4ec51764d408dae2397b5a8bb02. Latest consensus state: current_term: 1 leader_uuid: "254dc4ec51764d408dae2397b5a8bb02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER } }
I20260709 13:05:32.041957 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:32.041831 10412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "254dc4ec51764d408dae2397b5a8bb02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254dc4ec51764d408dae2397b5a8bb02" member_type: VOTER } }
I20260709 13:05:32.042716 10412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:32.044094 10421 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:32.049216 10421 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:32.050757   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:32.058933 10421 catalog_manager.cc:1383] Generated new cluster ID: 7f7486a7786641a7a690ccb201b7543a
I20260709 13:05:32.059319 10421 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:32.093884 10421 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:32.095458 10421 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:32.113348 10421 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02: Generated new TSK 0
I20260709 13:05:32.114192 10421 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:32.182638   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:32.189673 10429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.191216 10430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.194116   431 server_base.cc:1061] running on GCE node
W20260709 13:05:32.194640 10432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.197404   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:32.197630   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:32.197851   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602332197815 us; error 0 us; skew 500 ppm
I20260709 13:05:32.198448   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:32.201297   431 webserver.cc:533] Webserver started at http://127.0.107.193:44317/ using document root <none> and password file <none>
I20260709 13:05:32.201833   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:32.202032   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:32.202311   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:32.203588   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "6a4b3558672241d2969ebf2c86f6d84e"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.208458   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:05:32.212106 10437 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.212883   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:05:32.213223   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "6a4b3558672241d2969ebf2c86f6d84e"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.213539   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:32.231401   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:32.232749   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:32.234274   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:32.236779   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:32.236999   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.237243   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:32.237432   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.281617   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42377
I20260709 13:05:32.281725 10499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42377 every 8 connection(s)
I20260709 13:05:32.287240   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:32.296437 10504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.298864 10505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.301267 10507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.301867   431 server_base.cc:1061] running on GCE node
I20260709 13:05:32.305019 10500 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36559
I20260709 13:05:32.305497 10500 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:32.305485   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:32.306020   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:32.306248   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602332306232 us; error 0 us; skew 500 ppm
I20260709 13:05:32.306738 10500 heartbeater.cc:507] Master 127.0.107.254:36559 requested a full tablet report, sending...
I20260709 13:05:32.307130   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:32.309541 10373 ts_manager.cc:194] Registered new tserver with Master: 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:32.310283   431 webserver.cc:533] Webserver started at http://127.0.107.194:42089/ using document root <none> and password file <none>
I20260709 13:05:32.310931   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:32.311133   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:32.311421   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:32.311689 10373 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38488
I20260709 13:05:32.312868   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.318445   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:05:32.322489 10512 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.323411   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:32.323724   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.324124   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:32.335590   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:32.336997   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:32.338557   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:32.341147   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:32.341351   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.341668   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:32.341833   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.389029   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:46713
I20260709 13:05:32.389161 10574 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:46713 every 8 connection(s)
I20260709 13:05:32.394744   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:32.403292 10578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.407102 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.413607 10575 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36559
I20260709 13:05:32.414211 10575 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:32.414677   431 server_base.cc:1061] running on GCE node
W20260709 13:05:32.415138 10581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.415405 10575 heartbeater.cc:507] Master 127.0.107.254:36559 requested a full tablet report, sending...
I20260709 13:05:32.416399   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:32.416692   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:32.416935   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602332416913 us; error 0 us; skew 500 ppm
I20260709 13:05:32.417668   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:32.418048 10373 ts_manager.cc:194] Registered new tserver with Master: e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:32.419930 10373 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38490
I20260709 13:05:32.420511   431 webserver.cc:533] Webserver started at http://127.0.107.195:41833/ using document root <none> and password file <none>
I20260709 13:05:32.421156   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:32.421427   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:32.421725   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:32.423039   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "13c7cc72a0274ec3859962b46603bb12"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.428514   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:05:32.432317 10586 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.433344   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:05:32.433637   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "13c7cc72a0274ec3859962b46603bb12"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.433892   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:32.449287   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:32.450657   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:32.452318   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:32.454826   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:32.455062   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.455365   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:32.455533   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.501513   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36903
I20260709 13:05:32.501595 10648 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36903 every 8 connection(s)
I20260709 13:05:32.507411   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:32.515578 10652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.517304 10653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.520641 10649 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36559
W20260709 13:05:32.520955 10655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.521195 10649 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:32.521333   431 server_base.cc:1061] running on GCE node
I20260709 13:05:32.522338 10649 heartbeater.cc:507] Master 127.0.107.254:36559 requested a full tablet report, sending...
I20260709 13:05:32.522384   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:32.522725   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:32.522862   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602332522851 us; error 0 us; skew 500 ppm
I20260709 13:05:32.523559   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:32.525017 10373 ts_manager.cc:194] Registered new tserver with Master: 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:32.526685   431 webserver.cc:533] Webserver started at http://127.0.107.196:43757/ using document root <none> and password file <none>
I20260709 13:05:32.527014 10373 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38504
I20260709 13:05:32.527441   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:32.527752   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:32.528141   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:32.529585   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-3-root/instance:
uuid: "13ee73323c814e939932f8e5db34fc96"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.534400   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:05:32.538343 10660 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.539211   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:05:32.539548   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-3-root
uuid: "13ee73323c814e939932f8e5db34fc96"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:32.539906   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:32.552467   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:32.553833   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:32.555238   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:32.557667   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:32.557917   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.558167   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:32.558341   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.600049   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:40563
I20260709 13:05:32.600139 10722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:40563 every 8 connection(s)
I20260709 13:05:32.614615 10723 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36559
I20260709 13:05:32.615047 10723 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:32.616045 10723 heartbeater.cc:507] Master 127.0.107.254:36559 requested a full tablet report, sending...
I20260709 13:05:32.618132 10373 ts_manager.cc:194] Registered new tserver with Master: 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:32.618326   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015019788s
I20260709 13:05:32.620256 10373 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38506
I20260709 13:05:32.646927 10373 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38510:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:05:32.893958 10688 tablet_service.cc:1511] Processing CreateTablet for tablet b4b3541d8e1e4c3faefc6d3350706237 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:32.895627 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b3541d8e1e4c3faefc6d3350706237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.905328 10614 tablet_service.cc:1511] Processing CreateTablet for tablet b4b3541d8e1e4c3faefc6d3350706237 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:32.907178 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b3541d8e1e4c3faefc6d3350706237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.909709 10539 tablet_service.cc:1511] Processing CreateTablet for tablet 09591a419b7a4edf9163e27b3861dda6 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:32.911325 10539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09591a419b7a4edf9163e27b3861dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.905628 10540 tablet_service.cc:1511] Processing CreateTablet for tablet b4b3541d8e1e4c3faefc6d3350706237 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:32.912083 10687 tablet_service.cc:1511] Processing CreateTablet for tablet 265c12d44bf04deb9f52c9bdb82cc88d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:32.913905 10687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265c12d44bf04deb9f52c9bdb82cc88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.913897 10540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b3541d8e1e4c3faefc6d3350706237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.917040 10613 tablet_service.cc:1511] Processing CreateTablet for tablet 09591a419b7a4edf9163e27b3861dda6 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:32.918519 10613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09591a419b7a4edf9163e27b3861dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.920109 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf22c286c6747518bafda752540a96c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:32.921859 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf22c286c6747518bafda752540a96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.922806 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 265c12d44bf04deb9f52c9bdb82cc88d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:32.925406 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdbdcb35eeb430e824a441d12445254 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:32.925879 10537 tablet_service.cc:1511] Processing CreateTablet for tablet 611126921768423c97934867e0e9cb44 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:32.926921 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdbdcb35eeb430e824a441d12445254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.929132 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf22c286c6747518bafda752540a96c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:32.929886 10611 tablet_service.cc:1511] Processing CreateTablet for tablet 611126921768423c97934867e0e9cb44 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:32.930804 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf22c286c6747518bafda752540a96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.932287 10611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 611126921768423c97934867e0e9cb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.934247 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 519a41f028694f6693c6c85038c54db0 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:32.934486 10684 tablet_service.cc:1511] Processing CreateTablet for tablet 519a41f028694f6693c6c85038c54db0 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:32.936045 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519a41f028694f6693c6c85038c54db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.936151 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519a41f028694f6693c6c85038c54db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.936617 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265c12d44bf04deb9f52c9bdb82cc88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.939877 10537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 611126921768423c97934867e0e9cb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.946262 10536 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdbdcb35eeb430e824a441d12445254 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:32.949321 10535 tablet_service.cc:1511] Processing CreateTablet for tablet b87f245147a44fc0a03f124aa5040d22 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:32.951356 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdbdcb35eeb430e824a441d12445254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.952067 10609 tablet_service.cc:1511] Processing CreateTablet for tablet b87f245147a44fc0a03f124aa5040d22 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:32.952913 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87f245147a44fc0a03f124aa5040d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.953665 10609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87f245147a44fc0a03f124aa5040d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.967851 10683 tablet_service.cc:1511] Processing CreateTablet for tablet b87f245147a44fc0a03f124aa5040d22 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:32.969789 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87f245147a44fc0a03f124aa5040d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.978044 10461 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdbdcb35eeb430e824a441d12445254 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:32.978243 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 09591a419b7a4edf9163e27b3861dda6 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:32.981304 10461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdbdcb35eeb430e824a441d12445254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.981091 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf22c286c6747518bafda752540a96c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:32.982640 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf22c286c6747518bafda752540a96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.983282 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09591a419b7a4edf9163e27b3861dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.984963 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 611126921768423c97934867e0e9cb44 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:32.986403 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 611126921768423c97934867e0e9cb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.989050 10460 tablet_service.cc:1511] Processing CreateTablet for tablet 519a41f028694f6693c6c85038c54db0 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:32.993933 10464 tablet_service.cc:1511] Processing CreateTablet for tablet 265c12d44bf04deb9f52c9bdb82cc88d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:32.996250 10460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 519a41f028694f6693c6c85038c54db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.998085 10464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265c12d44bf04deb9f52c9bdb82cc88d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:33.046123 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:33.142694 10459 tablet_service.cc:1511] Processing CreateTablet for tablet b9dd264c867c4296a9a2f7fd53ac51e4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:33.145083 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dd264c867c4296a9a2f7fd53ac51e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.145427 10534 tablet_service.cc:1511] Processing CreateTablet for tablet b9dd264c867c4296a9a2f7fd53ac51e4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:33.147007 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dd264c867c4296a9a2f7fd53ac51e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.155019 10614 tablet_service.cc:1511] Processing CreateTablet for tablet b9dd264c867c4296a9a2f7fd53ac51e4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:33.156662 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dd264c867c4296a9a2f7fd53ac51e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.169188 10744 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:33.168920 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 1ebe12bfeac04b27ba5070e402b0307a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:33.170557 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ebe12bfeac04b27ba5070e402b0307a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.172981 10743 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:33.178540 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 1ebe12bfeac04b27ba5070e402b0307a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:33.184012 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ebe12bfeac04b27ba5070e402b0307a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.194607 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71b4eb28f74c3db730751a1aaca656 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:33.196259 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71b4eb28f74c3db730751a1aaca656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.196703 10745 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:33.196206 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71b4eb28f74c3db730751a1aaca656 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:33.203691 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71b4eb28f74c3db730751a1aaca656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.219156 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 1ebe12bfeac04b27ba5070e402b0307a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:33.219748 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71b4eb28f74c3db730751a1aaca656 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:33.220717 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ebe12bfeac04b27ba5070e402b0307a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.221155 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71b4eb28f74c3db730751a1aaca656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.225601 10746 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:33.231001 10535 tablet_service.cc:1511] Processing CreateTablet for tablet e7a759b0b5de444fa1c1486147b76cfa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:33.232437 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a759b0b5de444fa1c1486147b76cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.235466 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 44c96c4f1f68468a8302df29e4fa399f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:33.236822 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c96c4f1f68468a8302df29e4fa399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.241050 10612 tablet_service.cc:1511] Processing CreateTablet for tablet e7a759b0b5de444fa1c1486147b76cfa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:33.242861 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a759b0b5de444fa1c1486147b76cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.243165 10746 tablet_bootstrap.cc:654] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.250988 10534 tablet_service.cc:1511] Processing CreateTablet for tablet 44c96c4f1f68468a8302df29e4fa399f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:33.252513 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c96c4f1f68468a8302df29e4fa399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.258088 10743 tablet_bootstrap.cc:654] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.259845 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 44c96c4f1f68468a8302df29e4fa399f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:33.261174 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c96c4f1f68468a8302df29e4fa399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.264243 10686 tablet_service.cc:1511] Processing CreateTablet for tablet e7a759b0b5de444fa1c1486147b76cfa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:33.265561 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a759b0b5de444fa1c1486147b76cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.251582 10744 tablet_bootstrap.cc:654] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.280529 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 68e9d448f8854ecd81d429b12890f07a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:33.281733 10745 tablet_bootstrap.cc:654] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.282526 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e9d448f8854ecd81d429b12890f07a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.302242 10610 tablet_service.cc:1511] Processing CreateTablet for tablet e9a9e5b696054a01b8aec158182b544d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:33.280840 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 68e9d448f8854ecd81d429b12890f07a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:33.303733 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a9e5b696054a01b8aec158182b544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.307971 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e9d448f8854ecd81d429b12890f07a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.314755 10500 heartbeater.cc:499] Master 127.0.107.254:36559 was elected leader, sending a full tablet report...
I20260709 13:05:33.320533 10683 tablet_service.cc:1511] Processing CreateTablet for tablet e9a9e5b696054a01b8aec158182b544d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:33.324574 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a9e5b696054a01b8aec158182b544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.325670 10462 tablet_service.cc:1511] Processing CreateTablet for tablet e9a9e5b696054a01b8aec158182b544d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:33.320533 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 68e9d448f8854ecd81d429b12890f07a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:33.336376 10743 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:33.336429 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e9d448f8854ecd81d429b12890f07a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.337359 10743 ts_tablet_manager.cc:1403] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.165s	user 0.008s	sys 0.007s
I20260709 13:05:33.340128 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a9e5b696054a01b8aec158182b544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.340615 10743 raft_consensus.cc:359] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:33.341846 10746 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:33.352654 10744 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:33.350943 10684 tablet_service.cc:1511] Processing CreateTablet for tablet acd69d2652a64ddba95c78f88d6e9a97 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:33.345829 10534 tablet_service.cc:1511] Processing CreateTablet for tablet acd69d2652a64ddba95c78f88d6e9a97 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:33.342837 10614 tablet_service.cc:1511] Processing CreateTablet for tablet acd69d2652a64ddba95c78f88d6e9a97 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:33.363948 10744 ts_tablet_manager.cc:1403] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.195s	user 0.015s	sys 0.006s
I20260709 13:05:33.363948 10746 ts_tablet_manager.cc:1403] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.139s	user 0.037s	sys 0.047s
I20260709 13:05:33.365024 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 63cd6402a88541f794b90126ed48b800 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:33.372285 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd69d2652a64ddba95c78f88d6e9a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.362216 10743 raft_consensus.cc:385] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.372817 10743 raft_consensus.cc:740] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.372480 10744 raft_consensus.cc:359] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.373351 10744 raft_consensus.cc:385] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.373745 10744 raft_consensus.cc:740] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.372574 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cd6402a88541f794b90126ed48b800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.383970 10743 consensus_queue.cc:260] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:33.388882 10649 heartbeater.cc:499] Master 127.0.107.254:36559 was elected leader, sending a full tablet report...
I20260709 13:05:33.389509 10743 ts_tablet_manager.cc:1434] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.003s
I20260709 13:05:33.390635 10743 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:33.390439 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 63cd6402a88541f794b90126ed48b800 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:33.376242 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd69d2652a64ddba95c78f88d6e9a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.377323 10746 raft_consensus.cc:359] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.378414 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd69d2652a64ddba95c78f88d6e9a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.378684 10745 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:33.392040 10746 raft_consensus.cc:385] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.392412 10746 raft_consensus.cc:740] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.392562 10745 ts_tablet_manager.cc:1403] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.196s	user 0.013s	sys 0.010s
I20260709 13:05:33.393206 10746 consensus_queue.cc:260] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.396222 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cd6402a88541f794b90126ed48b800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.375903 10744 consensus_queue.cc:260] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.400215 10745 raft_consensus.cc:359] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.402313 10745 raft_consensus.cc:385] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.402623 10723 heartbeater.cc:499] Master 127.0.107.254:36559 was elected leader, sending a full tablet report...
I20260709 13:05:33.402786 10745 raft_consensus.cc:740] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.404037 10745 consensus_queue.cc:260] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.410038 10744 ts_tablet_manager.cc:1434] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:05:33.411173 10744 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:33.413499 10575 heartbeater.cc:499] Master 127.0.107.254:36559 was elected leader, sending a full tablet report...
I20260709 13:05:33.418067 10744 tablet_bootstrap.cc:654] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.423261 10745 ts_tablet_manager.cc:1434] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:05:33.424566 10745 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:33.428887 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 63cd6402a88541f794b90126ed48b800 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:33.430217 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cd6402a88541f794b90126ed48b800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.430629 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc077c5bba0428d9350f538bd2aeeda (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:33.431914 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc077c5bba0428d9350f538bd2aeeda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.435061 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 159d5a82c8664fc79597d7f1715bf80e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:33.436395 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159d5a82c8664fc79597d7f1715bf80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.436797 10535 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc077c5bba0428d9350f538bd2aeeda (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:33.438062 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc077c5bba0428d9350f538bd2aeeda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.446089 10746 ts_tablet_manager.cc:1434] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.076s	user 0.002s	sys 0.030s
I20260709 13:05:33.447330 10746 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:33.454946 10746 tablet_bootstrap.cc:654] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.457190 10743 tablet_bootstrap.cc:654] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.467643 10754 raft_consensus.cc:493] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.476344 10754 raft_consensus.cc:515] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.484714 10754 leader_election.cc:290] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:33.485766 10753 raft_consensus.cc:493] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.487093 10753 raft_consensus.cc:515] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.490691 10684 tablet_service.cc:1511] Processing CreateTablet for tablet 159d5a82c8664fc79597d7f1715bf80e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:33.491401 10753 leader_election.cc:290] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:33.492259 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159d5a82c8664fc79597d7f1715bf80e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:33.506783 10650 tablet.cc:2406] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:33.511435 10534 tablet_service.cc:1511] Processing CreateTablet for tablet 027997962a9c49a19a76790e5f7ead90 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:33.513352 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027997962a9c49a19a76790e5f7ead90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.514432 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 4d38aaddc970432bb924dc77ac8bcfa7 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:33.515959 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d38aaddc970432bb924dc77ac8bcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.536454 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 4d38aaddc970432bb924dc77ac8bcfa7 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:33.538347 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d38aaddc970432bb924dc77ac8bcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.556869 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 027997962a9c49a19a76790e5f7ead90 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:33.558373 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027997962a9c49a19a76790e5f7ead90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.470985 10745 tablet_bootstrap.cc:654] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.596401 10744 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:33.597015 10744 ts_tablet_manager.cc:1403] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.186s	user 0.033s	sys 0.059s
I20260709 13:05:33.601810 10743 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:33.602404 10743 ts_tablet_manager.cc:1403] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.212s	user 0.011s	sys 0.003s
I20260709 13:05:33.602510 10744 raft_consensus.cc:359] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.603304 10744 raft_consensus.cc:385] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.603605 10744 raft_consensus.cc:740] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.606364 10744 consensus_queue.cc:260] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.609108 10743 raft_consensus.cc:359] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:33.614281 10743 raft_consensus.cc:385] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.615670 10743 raft_consensus.cc:740] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.615185 10535 tablet_service.cc:1511] Processing CreateTablet for tablet 64cae8b9e6b645c59124d6728f4ecb65 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:33.611346 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc077c5bba0428d9350f538bd2aeeda (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:33.612385 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 64cae8b9e6b645c59124d6728f4ecb65 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:33.618481 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 08fdd13c2ece4a958ada223d6e631c9f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:33.619421 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64cae8b9e6b645c59124d6728f4ecb65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.619060 10684 tablet_service.cc:1511] Processing CreateTablet for tablet 64cae8b9e6b645c59124d6728f4ecb65 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:33.620404 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64cae8b9e6b645c59124d6728f4ecb65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.620833 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64cae8b9e6b645c59124d6728f4ecb65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.612385 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 159d5a82c8664fc79597d7f1715bf80e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:33.609589 10744 ts_tablet_manager.cc:1434] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:05:33.625319 10744 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:33.611353 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 027997962a9c49a19a76790e5f7ead90 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:33.622017 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fdd13c2ece4a958ada223d6e631c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.622576 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc077c5bba0428d9350f538bd2aeeda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.627229 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027997962a9c49a19a76790e5f7ead90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.627933 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 159d5a82c8664fc79597d7f1715bf80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.638437 10683 tablet_service.cc:1511] Processing CreateTablet for tablet 08fdd13c2ece4a958ada223d6e631c9f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:33.639932 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fdd13c2ece4a958ada223d6e631c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.645993 10745 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:33.646593 10745 ts_tablet_manager.cc:1403] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.222s	user 0.009s	sys 0.003s
I20260709 13:05:33.649739 10745 raft_consensus.cc:359] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:33.650444 10745 raft_consensus.cc:385] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.650708 10745 raft_consensus.cc:740] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.653074 10744 tablet_bootstrap.cc:654] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.655435 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:33.651378 10745 consensus_queue.cc:260] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:33.622999 10746 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:33.617147 10743 consensus_queue.cc:260] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:33.657032 10746 ts_tablet_manager.cc:1403] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.210s	user 0.067s	sys 0.038s
I20260709 13:05:33.666596 10459 tablet_service.cc:1511] Processing CreateTablet for tablet 08fdd13c2ece4a958ada223d6e631c9f (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:33.668121 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fdd13c2ece4a958ada223d6e631c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.668640 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
W20260709 13:05:33.670346 10663 leader_election.cc:343] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:33.657414 10515 leader_election.cc:343] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.658317 10743 ts_tablet_manager.cc:1434] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.056s	user 0.003s	sys 0.004s
I20260709 13:05:33.666901 10746 raft_consensus.cc:359] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.663133 10745 ts_tablet_manager.cc:1434] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:05:33.672226 10743 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:33.673801 10746 raft_consensus.cc:385] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.674355 10746 raft_consensus.cc:740] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.675249 10746 consensus_queue.cc:260] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.681180 10743 tablet_bootstrap.cc:654] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.658855 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 4d38aaddc970432bb924dc77ac8bcfa7 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:33.684684 10745 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:33.685245 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d38aaddc970432bb924dc77ac8bcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.688476 10746 ts_tablet_manager.cc:1434] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.024s	user 0.001s	sys 0.005s
I20260709 13:05:33.689646 10746 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:33.691581 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 6eefd49b51434d4786968ac53f50899c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:33.696552 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eefd49b51434d4786968ac53f50899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.699733 10744 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:33.700353 10744 ts_tablet_manager.cc:1403] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.075s	user 0.011s	sys 0.001s
I20260709 13:05:33.701579 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 6eefd49b51434d4786968ac53f50899c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:33.703055 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eefd49b51434d4786968ac53f50899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.704219 10744 raft_consensus.cc:359] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.705030 10744 raft_consensus.cc:385] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.705219 10746 tablet_bootstrap.cc:654] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.705351 10744 raft_consensus.cc:740] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.706352 10744 consensus_queue.cc:260] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.721868 10745 tablet_bootstrap.cc:654] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.721470 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:33.727208 10744 ts_tablet_manager.cc:1434] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:05:33.727712 10743 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:33.728367 10743 ts_tablet_manager.cc:1403] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.056s	user 0.008s	sys 0.007s
W20260709 13:05:33.723634 10516 leader_election.cc:343] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.728437 10744 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:33.730456 10516 leader_election.cc:304] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:33.731374 10754 raft_consensus.cc:2749] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.731551 10746 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:33.732116 10746 ts_tablet_manager.cc:1403] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.001s
I20260709 13:05:33.733671 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:33.735457 10664 leader_election.cc:343] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.735530 10746 raft_consensus.cc:359] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.736387 10746 raft_consensus.cc:385] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.736220 10664 leader_election.cc:304] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:33.737429 10753 raft_consensus.cc:2749] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.738176 10746 raft_consensus.cc:740] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.737871 10743 raft_consensus.cc:359] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.739043 10746 consensus_queue.cc:260] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:33.742594 10745 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:33.742619 10744 tablet_bootstrap.cc:654] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.744799 10746 ts_tablet_manager.cc:1434] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:05:33.746014 10746 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:33.743278 10745 ts_tablet_manager.cc:1403] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.059s	user 0.006s	sys 0.018s
I20260709 13:05:33.750370 10743 raft_consensus.cc:385] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.750802 10745 raft_consensus.cc:359] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.751667 10745 raft_consensus.cc:385] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.752010 10745 raft_consensus.cc:740] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.750835 10743 raft_consensus.cc:740] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.753948 10746 tablet_bootstrap.cc:654] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.753716 10743 consensus_queue.cc:260] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.759186 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 6eefd49b51434d4786968ac53f50899c (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:33.761070 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eefd49b51434d4786968ac53f50899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.762914 10743 ts_tablet_manager.cc:1434] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.005s
I20260709 13:05:33.764397 10743 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:33.768213 10745 consensus_queue.cc:260] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.772015 10745 ts_tablet_manager.cc:1434] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.004s
I20260709 13:05:33.773433 10745 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:33.776135 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc7b153bdd64ea6823d2ca146d5ab61 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:33.777763 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc7b153bdd64ea6823d2ca146d5ab61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.785297 10751 raft_consensus.cc:493] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.786129 10751 raft_consensus.cc:515] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:33.795889 10752 raft_consensus.cc:493] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.796643 10744 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:33.796873 10745 tablet_bootstrap.cc:654] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.796587 10752 raft_consensus.cc:515] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.797287 10744 ts_tablet_manager.cc:1403] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.069s	user 0.028s	sys 0.022s
I20260709 13:05:33.801160 10744 raft_consensus.cc:359] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:33.816839 10744 raft_consensus.cc:385] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.816365 10746 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:33.817338 10751 leader_election.cc:290] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:33.817327 10744 raft_consensus.cc:740] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.801376 10752 leader_election.cc:290] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:33.824221 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc7b153bdd64ea6823d2ca146d5ab61 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:33.828233 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc7b153bdd64ea6823d2ca146d5ab61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.822086 10746 ts_tablet_manager.cc:1403] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.019s
I20260709 13:05:33.822368 10744 consensus_queue.cc:260] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:33.832810 10744 ts_tablet_manager.cc:1434] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.005s
I20260709 13:05:33.833357 10746 raft_consensus.cc:359] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.834249 10744 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:33.840919 10684 tablet_service.cc:1511] Processing CreateTablet for tablet 2352e31e46a8437ab6897bfbe52d7f9b (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:33.834251 10746 raft_consensus.cc:385] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.851333 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2352e31e46a8437ab6897bfbe52d7f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.857280 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc7b153bdd64ea6823d2ca146d5ab61 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:33.859117 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc7b153bdd64ea6823d2ca146d5ab61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.866073 10743 tablet_bootstrap.cc:654] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.866252 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 2352e31e46a8437ab6897bfbe52d7f9b (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:33.863190 10746 raft_consensus.cc:740] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.876829 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2352e31e46a8437ab6897bfbe52d7f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:33.887024 10746 consensus_queue.cc:260] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.894232 10746 ts_tablet_manager.cc:1434] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 13:05:33.896711 10746 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:33.895004 10744 tablet_bootstrap.cc:654] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.909951 10743 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:33.910559 10743 ts_tablet_manager.cc:1403] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.146s	user 0.019s	sys 0.011s
I20260709 13:05:33.911898 10746 tablet_bootstrap.cc:654] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.913848 10743 raft_consensus.cc:359] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.914667 10743 raft_consensus.cc:385] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.914961 10743 raft_consensus.cc:740] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.916067 10743 consensus_queue.cc:260] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:33.921231 10743 ts_tablet_manager.cc:1434] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:05:33.922335 10743 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:33.930613 10746 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:33.931922 10746 ts_tablet_manager.cc:1403] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.003s
I20260709 13:05:33.933528 10745 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:33.944168 10745 ts_tablet_manager.cc:1403] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.010s
I20260709 13:05:33.996107 10614 tablet_service.cc:1511] Processing CreateTablet for tablet e98054391f4644c9a7c7ccf6a6e2db80 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:33.999154 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98054391f4644c9a7c7ccf6a6e2db80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.002933 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:34.004487 10751 raft_consensus.cc:493] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.006093 10751 raft_consensus.cc:515] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.004594 10753 raft_consensus.cc:493] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.004734 10624 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:34.008451 10753 raft_consensus.cc:515] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.008971 10535 tablet_service.cc:1511] Processing CreateTablet for tablet 2352e31e46a8437ab6897bfbe52d7f9b (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:34.009828 10440 leader_election.cc:304] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:34.011300 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2352e31e46a8437ab6897bfbe52d7f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.011624 10752 raft_consensus.cc:2804] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.012133 10752 raft_consensus.cc:493] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.012622 10752 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.043298 10746 raft_consensus.cc:359] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.045908 10746 raft_consensus.cc:385] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.046917 10746 raft_consensus.cc:740] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.048098 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:34.041700 10459 tablet_service.cc:1511] Processing CreateTablet for tablet e98054391f4644c9a7c7ccf6a6e2db80 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:34.054646 10751 leader_election.cc:290] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:34.055583 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98054391f4644c9a7c7ccf6a6e2db80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:34.059149 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:34.065470 10685 tablet_service.cc:1511] Processing CreateTablet for tablet e98054391f4644c9a7c7ccf6a6e2db80 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:34.071048 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98054391f4644c9a7c7ccf6a6e2db80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.074937 10534 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1733523e994002a8df5fa4d29d8a72 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:34.076987 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.077594 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1733523e994002a8df5fa4d29d8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.078023 10550 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:34.077677 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1733523e994002a8df5fa4d29d8a72 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:34.079420 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1733523e994002a8df5fa4d29d8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.087110 10783 raft_consensus.cc:493] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.088292 10783 raft_consensus.cc:515] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.099742 10754 raft_consensus.cc:493] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.100924 10754 raft_consensus.cc:515] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.109215 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:34.110268 10624 raft_consensus.cc:2468] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:34.112582 10515 leader_election.cc:304] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.119949 10786 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.123095 10786 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.124424 10745 raft_consensus.cc:359] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.125690 10745 raft_consensus.cc:385] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.126055 10745 raft_consensus.cc:740] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.126991 10745 consensus_queue.cc:260] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.051615 10746 consensus_queue.cc:260] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.127913 10786 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.140609 10754 leader_election.cc:290] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:34.128908 10752 raft_consensus.cc:515] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.155992 10754 raft_consensus.cc:2804] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.156649 10754 raft_consensus.cc:493] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.157111 10754 raft_consensus.cc:3060] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.164637 10752 leader_election.cc:290] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:34.167872 10475 raft_consensus.cc:2468] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:34.174880 10783 leader_election.cc:290] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.176411 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.176980 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.177768 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.178121 10548 raft_consensus.cc:2468] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:34.179587 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:34.180444 10664 leader_election.cc:343] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.181440 10664 leader_election.cc:304] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.182708 10744 tablet_bootstrap.cc:492] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
W20260709 13:05:34.183171 10588 leader_election.cc:343] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.184237 10783 raft_consensus.cc:2804] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.184801 10783 raft_consensus.cc:493] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.185220 10783 raft_consensus.cc:3060] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.196327 10751 raft_consensus.cc:493] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.197359 10751 raft_consensus.cc:515] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.208468 10751 leader_election.cc:290] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.208703 10536 tablet_service.cc:1511] Processing CreateTablet for tablet bbad48f22c2344388089817bc703b04a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:34.209656 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1733523e994002a8df5fa4d29d8a72 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:34.210628 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbad48f22c2344388089817bc703b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.211177 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1733523e994002a8df5fa4d29d8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.213326 10753 leader_election.cc:290] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:34.209131 10698 raft_consensus.cc:2468] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:34.210510 10788 raft_consensus.cc:493] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.215161 10788 raft_consensus.cc:515] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.215863 10588 leader_election.cc:304] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:34.183496 10744 ts_tablet_manager.cc:1403] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.350s	user 0.031s	sys 0.027s
I20260709 13:05:34.217485 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.218573 10550 raft_consensus.cc:2468] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:34.220640 10590 leader_election.cc:304] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.230980 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.224694 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:34.233587 10549 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.233817 10743 tablet_bootstrap.cc:654] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.235631 10751 raft_consensus.cc:2804] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.236697 10751 raft_consensus.cc:493] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.237124 10751 raft_consensus.cc:3060] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.215548 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:34.235917 10752 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.255887 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 143638us
I20260709 13:05:34.259294 10624 raft_consensus.cc:2468] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:34.264381 10785 raft_consensus.cc:493] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.265285 10785 raft_consensus.cc:515] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.265699 10744 raft_consensus.cc:359] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.285327 10788 leader_election.cc:290] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:34.287225 10785 leader_election.cc:290] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:34.279518 10752 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.290406 10752 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.291086 10791 raft_consensus.cc:493] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.291880 10791 raft_consensus.cc:515] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.292887 10612 tablet_service.cc:1511] Processing CreateTablet for tablet bbad48f22c2344388089817bc703b04a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:34.294430 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbad48f22c2344388089817bc703b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.294312 10791 leader_election.cc:290] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.294997 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.296720 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.278280 10744 raft_consensus.cc:385] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.299249 10744 raft_consensus.cc:740] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.300071 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:34.300203 10744 consensus_queue.cc:260] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.304039 10458 tablet_service.cc:1511] Processing CreateTablet for tablet bbad48f22c2344388089817bc703b04a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:34.305706 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbad48f22c2344388089817bc703b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.305918 10744 ts_tablet_manager.cc:1434] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.088s	user 0.010s	sys 0.000s
I20260709 13:05:34.307324 10744 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:34.309618 10590 leader_election.cc:343] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:34.311527 10441 leader_election.cc:343] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:34.312302 10441 leader_election.cc:343] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.314569 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:34.315294 10475 raft_consensus.cc:2468] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:34.317581 10783 raft_consensus.cc:515] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.318866 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:34.319679 10474 raft_consensus.cc:2468] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:34.320973 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:34.320835 10537 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79f3f50a72438383af31b858f986f5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:34.321666 10475 raft_consensus.cc:2468] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:34.322460 10537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79f3f50a72438383af31b858f986f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.329425 10746 ts_tablet_manager.cc:1434] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.396s	user 0.021s	sys 0.031s
I20260709 13:05:34.318930 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:34.330588 10547 raft_consensus.cc:2468] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:34.330911 10746 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:34.332198 10441 leader_election.cc:304] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.336584 10752 raft_consensus.cc:2804] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.337002 10752 raft_consensus.cc:493] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.337319 10752 raft_consensus.cc:3060] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.342618 10783 leader_election.cc:290] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.345525 10754 raft_consensus.cc:515] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.346028 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:34.346737 10547 raft_consensus.cc:3060] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.346866 10548 raft_consensus.cc:2393] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:34.350476 10590 leader_election.cc:304] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.309744 10549 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:34.353027 10441 leader_election.cc:304] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.354007 10791 raft_consensus.cc:2804] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.354557 10791 raft_consensus.cc:697] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:34.355595 10791 consensus_queue.cc:237] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.358952 10611 tablet_service.cc:1511] Processing CreateTablet for tablet 8f11079f7def4cef8f18ca580808c5d5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:34.360569 10611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f11079f7def4cef8f18ca580808c5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.364645 10794 raft_consensus.cc:2804] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.365175 10794 raft_consensus.cc:493] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.365535 10794 raft_consensus.cc:3060] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.366196 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.366909 10698 raft_consensus.cc:2468] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:34.370304 10459 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79f3f50a72438383af31b858f986f5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:34.372346 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:34.369095 10664 leader_election.cc:304] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:34.373827 10664 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.374987 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79f3f50a72438383af31b858f986f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.376833 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:34.377085 10754 leader_election.cc:290] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:34.377527 10624 raft_consensus.cc:3060] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.380988 10753 raft_consensus.cc:2804] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.381453 10753 raft_consensus.cc:493] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.381799 10753 raft_consensus.cc:3060] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.382332 10683 tablet_service.cc:1511] Processing CreateTablet for tablet 8f11079f7def4cef8f18ca580808c5d5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:34.384001 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f11079f7def4cef8f18ca580808c5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.389521 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 8f11079f7def4cef8f18ca580808c5d5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:34.392688 10783 raft_consensus.cc:2804] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.393239 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f11079f7def4cef8f18ca580808c5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.393679 10743 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:34.394346 10743 ts_tablet_manager.cc:1403] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.472s	user 0.028s	sys 0.018s
W20260709 13:05:34.390367 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 174654us
I20260709 13:05:34.391731 10534 tablet_service.cc:1511] Processing CreateTablet for tablet 43aaaf38cfe7411aa64c01c2519884fa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:34.397807 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aaaf38cfe7411aa64c01c2519884fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.397883 10684 tablet_service.cc:1511] Processing CreateTablet for tablet 7e79f3f50a72438383af31b858f986f5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:34.399204 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87f245147a44fc0a03f124aa5040d22" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:34.399909 10698 raft_consensus.cc:3060] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.401994 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:34.402757 10623 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.407243 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 7fcc014601624e27bddc57bc6aadb67d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:34.409380 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fcc014601624e27bddc57bc6aadb67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.399375 10684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e79f3f50a72438383af31b858f986f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.393196 10783 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.407204 10613 tablet_service.cc:1511] Processing CreateTablet for tablet 43aaaf38cfe7411aa64c01c2519884fa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:34.416230 10783 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.425930 10613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aaaf38cfe7411aa64c01c2519884fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.322489 10751 raft_consensus.cc:515] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.415222 10753 raft_consensus.cc:515] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:34.420571 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 70195.3us
I20260709 13:05:34.432902 10698 raft_consensus.cc:2468] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:34.437577 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:34.441669 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 7fcc014601624e27bddc57bc6aadb67d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:34.452553 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fcc014601624e27bddc57bc6aadb67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.452414 10514 leader_election.cc:304] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.453140 10751 leader_election.cc:290] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.453409 10754 raft_consensus.cc:2804] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.453971 10754 raft_consensus.cc:697] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:34.454881 10754 consensus_queue.cc:237] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.456184 10688 tablet_service.cc:1511] Processing CreateTablet for tablet 6d59609def584fcca66488c4e7073630 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:34.456625 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:34.457616 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d59609def584fcca66488c4e7073630. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:34.458132 10440 leader_election.cc:343] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.458670 10440 leader_election.cc:304] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.461416 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:34.462143 10549 raft_consensus.cc:3060] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.446309 10623 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:34.450313 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 7fcc014601624e27bddc57bc6aadb67d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:34.463528 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.464983 10697 raft_consensus.cc:2393] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:34.465327 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.466849 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.467744 10697 raft_consensus.cc:2393] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:34.463734 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fcc014601624e27bddc57bc6aadb67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.467909 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 43aaaf38cfe7411aa64c01c2519884fa (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:34.469383 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:34.470371 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aaaf38cfe7411aa64c01c2519884fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.471752 10372 catalog_manager.cc:5697] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.476027 10791 raft_consensus.cc:2749] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.429373 10743 raft_consensus.cc:359] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.477368 10743 raft_consensus.cc:385] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.477727 10743 raft_consensus.cc:740] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.478571 10743 consensus_queue.cc:260] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.479704 10547 raft_consensus.cc:2468] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:34.467020 10753 leader_election.cc:290] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:34.481317 10743 ts_tablet_manager.cc:1434] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.002s
I20260709 13:05:34.481246 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:34.482764 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:34.483479 10473 raft_consensus.cc:3060] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.484661 10783 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.485921 10696 raft_consensus.cc:2393] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:34.486920 10783 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.488130 10439 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:34.496614 10590 leader_election.cc:343] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.500039 10791 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.499637 10664 leader_election.cc:304] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:34.502676 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:34.506940 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:34.509707 10664 leader_election.cc:343] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.515017 10783 raft_consensus.cc:2804] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.517798 10783 raft_consensus.cc:697] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:34.522650 10624 raft_consensus.cc:2468] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:34.523641 10797 raft_consensus.cc:2749] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.519037 10783 consensus_queue.cc:237] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.565305 10745 ts_tablet_manager.cc:1434] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.621s	user 0.018s	sys 0.029s
W20260709 13:05:34.565871 10745 ts_tablet_manager.cc:1467] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Tablet startup took 621ms
I20260709 13:05:34.570501 10743 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
W20260709 13:05:34.586529 10360 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 95212.3us
I20260709 13:05:34.587920 10752 raft_consensus.cc:515] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.589221 10475 raft_consensus.cc:2393] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
W20260709 13:05:34.595712 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 107816us
W20260709 13:05:34.595896 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 69976.5us
I20260709 13:05:34.597355 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 708dee8c708a4b6e91bafafed7d61c2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:34.598850 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708dee8c708a4b6e91bafafed7d61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.598505 10794 raft_consensus.cc:515] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.599975 10622 raft_consensus.cc:2393] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:34.602226 10440 leader_election.cc:304] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:34.608173 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 6d59609def584fcca66488c4e7073630 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:34.609611 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d59609def584fcca66488c4e7073630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.615252 10744 tablet_bootstrap.cc:700] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: Time spent opening tablet: real 0.107s	user 0.000s	sys 0.005s
I20260709 13:05:34.616257 10744 tablet_bootstrap.cc:654] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.617738 10536 tablet_service.cc:1511] Processing CreateTablet for tablet 6d59609def584fcca66488c4e7073630 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:34.619155 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d59609def584fcca66488c4e7073630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.621735 10746 tablet_bootstrap.cc:700] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent opening tablet: real 0.130s	user 0.002s	sys 0.018s
I20260709 13:05:34.622648 10746 tablet_bootstrap.cc:654] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.629473 10752 leader_election.cc:290] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:34.632884 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:34.635318 10743 tablet_bootstrap.cc:654] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.635591 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265c12d44bf04deb9f52c9bdb82cc88d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:34.597712 10473 raft_consensus.cc:2468] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:34.653074 10683 tablet_service.cc:1511] Processing CreateTablet for tablet 708dee8c708a4b6e91bafafed7d61c2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:34.654744 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708dee8c708a4b6e91bafafed7d61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.662521 10547 raft_consensus.cc:2393] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:34.657195 10785 raft_consensus.cc:2749] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.664362 10441 leader_election.cc:304] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.665064 10664 leader_election.cc:304] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:34.665686 10785 raft_consensus.cc:2749] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.663134 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:34.667197 10753 raft_consensus.cc:2804] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.668996 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 3dff2cbddb694bf8a854bf2a1960649d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:34.671586 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dff2cbddb694bf8a854bf2a1960649d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.672394 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 3dff2cbddb694bf8a854bf2a1960649d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:34.673743 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dff2cbddb694bf8a854bf2a1960649d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:34.566277 10745 ts_tablet_manager.cc:1469] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: Trace:
0709 13:05:32.921379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.921587 (+   208us) service_pool.cc:224] Handling call
0709 13:05:32.923759 (+  2172us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:32.936119 (+ 12360us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:33.078334 (+142215us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:33.166572 (+ 88238us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:33.166973 (+   401us) inbound_call.cc:177] Queueing success response
0709 13:05:33.772960 (+605987us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:33.944631 (+171671us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:33.945341 (+   710us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:33.946216 (+   875us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":142865,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":3087,"tablet-open.queue_time_us":606030,"thread_start_us":434637,"threads_started":1}
I20260709 13:05:34.694339 10549 raft_consensus.cc:2468] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:34.696149 10590 leader_election.cc:304] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:34.697489 10797 raft_consensus.cc:2804] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.698029 10797 raft_consensus.cc:697] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:34.698935 10797 consensus_queue.cc:237] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.689000 10369 catalog_manager.cc:5697] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.702366 10371 catalog_manager.cc:5697] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.705348 10791 raft_consensus.cc:2749] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.690927 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "519a41f028694f6693c6c85038c54db0" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:34.707186 10473 raft_consensus.cc:2393] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:34.688999 10745 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:34.712898 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 708dee8c708a4b6e91bafafed7d61c2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:34.714439 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708dee8c708a4b6e91bafafed7d61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.718427 10474 raft_consensus.cc:2468] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:34.719365 10745 tablet_bootstrap.cc:654] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.724598 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 3dff2cbddb694bf8a854bf2a1960649d (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:34.725874 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dff2cbddb694bf8a854bf2a1960649d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.687135 10794 leader_election.cc:290] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:34.730679 10664 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.731973 10783 raft_consensus.cc:2804] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.732489 10783 raft_consensus.cc:697] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:34.733657 10746 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:34.733461 10783 consensus_queue.cc:237] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:34.734293 10746 ts_tablet_manager.cc:1403] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.404s	user 0.008s	sys 0.028s
I20260709 13:05:34.750571 10746 raft_consensus.cc:359] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.757414 10794 raft_consensus.cc:493] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.758014 10794 raft_consensus.cc:515] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.761066 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:34.762756 10590 leader_election.cc:343] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.765077 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:34.765883 10698 raft_consensus.cc:2468] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:34.766746 10463 tablet_service.cc:1511] Processing CreateTablet for tablet a04a676a3431425a91e63cd315e899fe (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:34.768356 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04a676a3431425a91e63cd315e899fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.724483 10753 raft_consensus.cc:697] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:34.780488 10588 leader_election.cc:304] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:34.782087 10538 tablet_service.cc:1511] Processing CreateTablet for tablet a04a676a3431425a91e63cd315e899fe (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:34.763890 10794 leader_election.cc:290] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:34.753338 10746 raft_consensus.cc:385] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.793493 10744 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:34.794081 10744 ts_tablet_manager.cc:1403] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.487s	user 0.004s	sys 0.025s
I20260709 13:05:34.796221 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04a676a3431425a91e63cd315e899fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.800541 10794 raft_consensus.cc:2804] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.801040 10794 raft_consensus.cc:493] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.801399 10794 raft_consensus.cc:3060] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.806579 10743 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:34.806751 10458 tablet_service.cc:1511] Processing CreateTablet for tablet e01b6c33d7014320a6ee51d8f1c731cb (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:34.807404 10743 ts_tablet_manager.cc:1403] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.237s	user 0.018s	sys 0.004s
I20260709 13:05:34.807138 10753 consensus_queue.cc:237] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:34.808514 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01b6c33d7014320a6ee51d8f1c731cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.806589 10614 tablet_service.cc:1511] Processing CreateTablet for tablet a04a676a3431425a91e63cd315e899fe (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:34.811534 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04a676a3431425a91e63cd315e899fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.788429 10746 raft_consensus.cc:740] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.816708 10534 tablet_service.cc:1511] Processing CreateTablet for tablet e01b6c33d7014320a6ee51d8f1c731cb (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:34.816820 10746 consensus_queue.cc:260] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.818284 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01b6c33d7014320a6ee51d8f1c731cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.818941 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 1c33cae280ab4407bf13087a4521d135 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:34.819892 10744 raft_consensus.cc:359] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.820609 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c33cae280ab4407bf13087a4521d135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.822837 10744 raft_consensus.cc:385] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.823258 10744 raft_consensus.cc:740] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.822358 10746 ts_tablet_manager.cc:1434] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.088s	user 0.003s	sys 0.007s
I20260709 13:05:34.825269 10746 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:34.825814 10744 consensus_queue.cc:260] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.828883 10683 tablet_service.cc:1511] Processing CreateTablet for tablet 1c33cae280ab4407bf13087a4521d135 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:34.816455 10688 tablet_service.cc:1511] Processing CreateTablet for tablet e01b6c33d7014320a6ee51d8f1c731cb (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:34.834825 10791 consensus_queue.cc:1048] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:34.844306 10744 ts_tablet_manager.cc:1434] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:05:34.845532 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01b6c33d7014320a6ee51d8f1c731cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.845703 10459 tablet_service.cc:1511] Processing CreateTablet for tablet 1c33cae280ab4407bf13087a4521d135 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:34.864338 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c33cae280ab4407bf13087a4521d135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.877115 10537 tablet_service.cc:1511] Processing CreateTablet for tablet e7b9860a1dbf4923a357199faf592424 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:34.878742 10746 tablet_bootstrap.cc:654] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.878762 10537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7b9860a1dbf4923a357199faf592424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.888841 10743 raft_consensus.cc:359] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.889812 10743 raft_consensus.cc:385] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.890223 10743 raft_consensus.cc:740] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.889454 10610 tablet_service.cc:1511] Processing CreateTablet for tablet e7b9860a1dbf4923a357199faf592424 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:34.894258 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7b9860a1dbf4923a357199faf592424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.895684 10744 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:34.891390 10743 consensus_queue.cc:260] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:34.897295 10588 leader_election.cc:304] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:34.879277 10371 catalog_manager.cc:5697] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.898402 10797 raft_consensus.cc:2749] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.906746 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c33cae280ab4407bf13087a4521d135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.918574 10745 tablet_bootstrap.cc:492] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:34.919390 10745 ts_tablet_manager.cc:1403] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.231s	user 0.014s	sys 0.034s
I20260709 13:05:34.927183 10686 tablet_service.cc:1511] Processing CreateTablet for tablet e7b9860a1dbf4923a357199faf592424 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:34.927981 10791 consensus_queue.cc:1048] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:34.928849 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7b9860a1dbf4923a357199faf592424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.946210 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8fb2ab1f70442786c46c19d8323095 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:34.947913 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8fb2ab1f70442786c46c19d8323095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.950060 10745 raft_consensus.cc:359] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:34.950889 10745 raft_consensus.cc:385] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.951220 10745 raft_consensus.cc:740] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.970070 10743 ts_tablet_manager.cc:1434] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.162s	user 0.009s	sys 0.003s
I20260709 13:05:34.971832 10743 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:34.974953 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8fb2ab1f70442786c46c19d8323095 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:34.976331 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8fb2ab1f70442786c46c19d8323095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.976224 10685 tablet_service.cc:1511] Processing CreateTablet for tablet ffc7391985ae44a38ba1f2fc35104eca (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:34.978009 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc7391985ae44a38ba1f2fc35104eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.988375 10369 catalog_manager.cc:5697] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.990418 10369 catalog_manager.cc:5697] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.997457 10536 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8fb2ab1f70442786c46c19d8323095 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:34.999198 10538 tablet_service.cc:1511] Processing CreateTablet for tablet ffc7391985ae44a38ba1f2fc35104eca (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:35.004753 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc7391985ae44a38ba1f2fc35104eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.980513 10745 consensus_queue.cc:260] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.000805 10754 consensus_queue.cc:1048] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.008848 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8fb2ab1f70442786c46c19d8323095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.017216 10753 raft_consensus.cc:493] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.006274 10463 tablet_service.cc:1511] Processing CreateTablet for tablet ffc7391985ae44a38ba1f2fc35104eca (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:35.017546 10783 consensus_queue.cc:1048] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:35.017885 10753 raft_consensus.cc:515] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:35.018697 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc7391985ae44a38ba1f2fc35104eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.021072 10753 leader_election.cc:290] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:35.023015 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf22c286c6747518bafda752540a96c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:35.023746 10624 raft_consensus.cc:2468] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:35.025108 10746 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:35.025218 10663 leader_election.cc:304] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:35.025712 10746 ts_tablet_manager.cc:1403] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.201s	user 0.017s	sys 0.024s
I20260709 13:05:35.029258 10746 raft_consensus.cc:359] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:35.030045 10746 raft_consensus.cc:385] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.030405 10746 raft_consensus.cc:740] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.032894 10794 raft_consensus.cc:515] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.036099 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:35.036883 10547 raft_consensus.cc:3060] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.039620 10754 consensus_queue.cc:1048] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:35.030144 10743 tablet_bootstrap.cc:654] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.040326 10794 leader_election.cc:290] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:35.074990 10744 tablet_bootstrap.cc:654] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.077306 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b3541d8e1e4c3faefc6d3350706237" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:35.082314 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf22c286c6747518bafda752540a96c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:35.069469 10547 raft_consensus.cc:2468] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:35.087473 10590 leader_election.cc:304] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:35.088490 10794 raft_consensus.cc:2804] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.089231 10794 raft_consensus.cc:697] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:35.066941 10753 raft_consensus.cc:2804] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.097457 10791 raft_consensus.cc:493] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.098067 10791 raft_consensus.cc:515] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.095145 10753 raft_consensus.cc:493] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.104815 10753 raft_consensus.cc:3060] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.105077 10791 leader_election.cc:290] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:35.105803 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:35.107054 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:35.110064 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:35.092900 10698 raft_consensus.cc:3060] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.126427 10698 raft_consensus.cc:2468] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:35.129020 10612 tablet_service.cc:1511] Processing CreateTablet for tablet fd44cb78e12145edbcd38d9ead51f9b9 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.130429 10683 tablet_service.cc:1511] Processing CreateTablet for tablet fd44cb78e12145edbcd38d9ead51f9b9 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.144482 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44cb78e12145edbcd38d9ead51f9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.090106 10794 consensus_queue.cc:237] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.131639 10458 tablet_service.cc:1511] Processing CreateTablet for tablet fd44cb78e12145edbcd38d9ead51f9b9 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:35.135226 10534 tablet_service.cc:1511] Processing CreateTablet for tablet d4849594fca843a982888e29f557c3f4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.135733 10611 tablet_service.cc:1511] Processing CreateTablet for tablet d4849594fca843a982888e29f557c3f4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.072531 10746 consensus_queue.cc:260] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:35.156038 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44cb78e12145edbcd38d9ead51f9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.156571 10746 ts_tablet_manager.cc:1434] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.130s	user 0.003s	sys 0.005s
I20260709 13:05:35.158061 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd44cb78e12145edbcd38d9ead51f9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.160301 10473 raft_consensus.cc:2468] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:35.162501 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4849594fca843a982888e29f557c3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.163394 10611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4849594fca843a982888e29f557c3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.113207 10547 raft_consensus.cc:2393] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:35.164888 10441 leader_election.cc:304] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:35.166141 10791 raft_consensus.cc:3060] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.167368 10783 consensus_queue.cc:1048] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.147481 10685 tablet_service.cc:1511] Processing CreateTablet for tablet d4849594fca843a982888e29f557c3f4 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:35.179620 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4849594fca843a982888e29f557c3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.187240 10537 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8ad15cc9f46928e9304f151ccc0af (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:35.188792 10537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8ad15cc9f46928e9304f151ccc0af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:35.193317 10664 consensus_peers.cc:597] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:35.195222 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8ad15cc9f46928e9304f151ccc0af (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:35.160385 10746 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:35.213873 10744 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:35.214152 10743 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:35.214512 10744 ts_tablet_manager.cc:1403] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.319s	user 0.018s	sys 0.005s
I20260709 13:05:35.214864 10743 ts_tablet_manager.cc:1403] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.243s	user 0.011s	sys 0.008s
I20260709 13:05:35.222960 10743 raft_consensus.cc:359] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.223867 10743 raft_consensus.cc:385] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.224231 10743 raft_consensus.cc:740] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.224831 10745 ts_tablet_manager.cc:1434] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.305s	user 0.010s	sys 0.021s
I20260709 13:05:35.225059 10743 consensus_queue.cc:260] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.226492 10745 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:35.227674 10743 ts_tablet_manager.cc:1434] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:05:35.229483 10743 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:35.233649 10797 raft_consensus.cc:493] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.234262 10797 raft_consensus.cc:515] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.235728 10743 tablet_bootstrap.cc:654] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.245479 10688 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6d5efd76a54d0892c288f4f50190ae (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:35.236630 10797 leader_election.cc:290] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:35.246716 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:35.247135 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6d5efd76a54d0892c288f4f50190ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.145805 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8ad15cc9f46928e9304f151ccc0af (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:35.256960 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8ad15cc9f46928e9304f151ccc0af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:35.260550 10590 leader_election.cc:343] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.251737 10783 consensus_queue.cc:1048] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.262333 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8ad15cc9f46928e9304f151ccc0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.279918 10753 raft_consensus.cc:515] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:35.281350 10791 raft_consensus.cc:2749] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.301626 10459 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6d5efd76a54d0892c288f4f50190ae (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:35.302148 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6d5efd76a54d0892c288f4f50190ae (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:35.308756 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6d5efd76a54d0892c288f4f50190ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.286659 10807 consensus_queue.cc:1048] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:35.323158 10807 consensus_queue.cc:1048] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:35.324713 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf22c286c6747518bafda752540a96c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:35.325070 10753 leader_election.cc:290] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:35.325435 10475 raft_consensus.cc:3060] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.325282 10614 tablet_service.cc:1511] Processing CreateTablet for tablet 00f0aa6cf57a4d228d8214c7dca07866 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:35.325914 10459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6d5efd76a54d0892c288f4f50190ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.326524 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f0aa6cf57a4d228d8214c7dca07866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.332522 10797 consensus_queue.cc:1048] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.334481 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:35.335165 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf22c286c6747518bafda752540a96c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:35.335805 10624 raft_consensus.cc:3060] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.362314 10746 tablet_bootstrap.cc:654] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.363333 10624 raft_consensus.cc:2468] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:35.366003 10745 tablet_bootstrap.cc:654] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.366170 10475 raft_consensus.cc:2468] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:35.369751 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 00f0aa6cf57a4d228d8214c7dca07866 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:35.369710 10794 consensus_queue.cc:1048] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.374270 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 00f0aa6cf57a4d228d8214c7dca07866 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:05:35.383976 10588 leader_election.cc:343] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.400866 10743 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:35.403407 10743 ts_tablet_manager.cc:1403] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.174s	user 0.018s	sys 0.011s
I20260709 13:05:35.407325 10588 leader_election.cc:304] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:35.408725 10797 raft_consensus.cc:2749] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.423287 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f0aa6cf57a4d228d8214c7dca07866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.438864 10745 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:35.439538 10745 ts_tablet_manager.cc:1403] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.213s	user 0.014s	sys 0.011s
I20260709 13:05:35.440747 10663 leader_election.cc:304] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:35.443252 10744 raft_consensus.cc:359] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.443991 10744 raft_consensus.cc:385] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.444284 10744 raft_consensus.cc:740] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.444736 10753 raft_consensus.cc:2804] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.445263 10753 raft_consensus.cc:697] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:35.445148 10744 consensus_queue.cc:260] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.445621 10534 tablet_service.cc:1511] Processing CreateTablet for tablet 0acce5611c5e4ad4be2fedf0b6f296ea (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:35.447341 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acce5611c5e4ad4be2fedf0b6f296ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.448431 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f0aa6cf57a4d228d8214c7dca07866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.462726 10372 catalog_manager.cc:5697] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.467097 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 0acce5611c5e4ad4be2fedf0b6f296ea (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:35.468607 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acce5611c5e4ad4be2fedf0b6f296ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.472975 10686 tablet_service.cc:1511] Processing CreateTablet for tablet 0acce5611c5e4ad4be2fedf0b6f296ea (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:35.475915 10686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acce5611c5e4ad4be2fedf0b6f296ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.482295 10743 raft_consensus.cc:359] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.483033 10743 raft_consensus.cc:385] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.483304 10743 raft_consensus.cc:740] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.446165 10753 consensus_queue.cc:237] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:35.495323 10743 consensus_queue.cc:260] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.497409 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 869dce63600848149f571b6bd169206a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:35.481397 10465 tablet_service.cc:1511] Processing CreateTablet for tablet 869dce63600848149f571b6bd169206a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:35.500746 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869dce63600848149f571b6bd169206a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.503705 10746 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:35.504851 10746 ts_tablet_manager.cc:1403] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.345s	user 0.042s	sys 0.043s
I20260709 13:05:35.516215 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869dce63600848149f571b6bd169206a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.523664 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 869dce63600848149f571b6bd169206a (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:35.530041 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869dce63600848149f571b6bd169206a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.535399 10797 raft_consensus.cc:493] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.555732 10743 ts_tablet_manager.cc:1434] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.152s	user 0.004s	sys 0.003s
I20260709 13:05:35.557695 10797 raft_consensus.cc:515] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.567087 10797 leader_election.cc:290] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:35.562883 10746 raft_consensus.cc:359] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.576509 10746 raft_consensus.cc:385] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.576871 10746 raft_consensus.cc:740] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.577638 10746 consensus_queue.cc:260] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.578016 10745 raft_consensus.cc:359] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:35.578747 10745 raft_consensus.cc:385] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.579039 10745 raft_consensus.cc:740] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.579231 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 9bf69fec47094c99a317be303e0def71 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:35.580399 10745 consensus_queue.cc:260] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:35.583401 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 9bf69fec47094c99a317be303e0def71 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:35.594328 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf69fec47094c99a317be303e0def71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.595052 10745 ts_tablet_manager.cc:1434] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.144s	user 0.005s	sys 0.005s
I20260709 13:05:35.590065 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:35.596707 10474 raft_consensus.cc:2468] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:35.591060 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:35.598251 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 42d4f443226d4b09b41d1b414448b007 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.598173 10590 leader_election.cc:304] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:35.599447 10797 raft_consensus.cc:2804] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.599932 10797 raft_consensus.cc:493] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.600337 10797 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.602983 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d4f443226d4b09b41d1b414448b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.592177 10743 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:35.604296 10746 ts_tablet_manager.cc:1434] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.099s	user 0.011s	sys 0.000s
I20260709 13:05:35.605439 10746 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:35.593017 10744 ts_tablet_manager.cc:1434] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.378s	user 0.031s	sys 0.012s
I20260709 13:05:35.596231 10745 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:35.608268 10842 consensus_queue.cc:1048] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:35.608690 10744 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:35.610373 10688 tablet_service.cc:1511] Processing CreateTablet for tablet 42d4f443226d4b09b41d1b414448b007 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.611683 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d4f443226d4b09b41d1b414448b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.614938 10745 tablet_bootstrap.cc:654] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.608294 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf69fec47094c99a317be303e0def71. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:35.623970 10590 leader_election.cc:343] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.624459 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 42d4f443226d4b09b41d1b414448b007 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:35.625912 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d4f443226d4b09b41d1b414448b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.622263 10797 raft_consensus.cc:515] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.599233 10372 catalog_manager.cc:5697] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.627161 10824 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.628257 10824 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:35.630627 10824 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:35.631368 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:35.632241 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:35.634133 10516 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:35.635465 10824 raft_consensus.cc:2749] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.637142 10534 tablet_service.cc:1511] Processing CreateTablet for tablet c6a0a4ae847a4b94b968e58c01a524c2 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:35.638408 10534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a0a4ae847a4b94b968e58c01a524c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.632500 10743 tablet_bootstrap.cc:654] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.626735 10538 tablet_service.cc:1511] Processing CreateTablet for tablet 9bf69fec47094c99a317be303e0def71 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:35.635628 10744 tablet_bootstrap.cc:654] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.652974 10683 tablet_service.cc:1511] Processing CreateTablet for tablet c6a0a4ae847a4b94b968e58c01a524c2 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:35.658479 10463 tablet_service.cc:1511] Processing CreateTablet for tablet 828be85a64dc4ceaaa111481681d3550 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:35.660405 10612 tablet_service.cc:1511] Processing CreateTablet for tablet c6a0a4ae847a4b94b968e58c01a524c2 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:35.663166 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf69fec47094c99a317be303e0def71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.667680 10746 tablet_bootstrap.cc:654] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.668497 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a0a4ae847a4b94b968e58c01a524c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.672396 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a0a4ae847a4b94b968e58c01a524c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.674386 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828be85a64dc4ceaaa111481681d3550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.681567 10547 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.691617 10536 tablet_service.cc:1511] Processing CreateTablet for tablet 828be85a64dc4ceaaa111481681d3550 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:35.693028 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828be85a64dc4ceaaa111481681d3550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.708752 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 828be85a64dc4ceaaa111481681d3550 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:35.709487 10745 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:35.710018 10745 ts_tablet_manager.cc:1403] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.114s	user 0.010s	sys 0.004s
I20260709 13:05:35.710035 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828be85a64dc4ceaaa111481681d3550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.719825 10842 consensus_queue.cc:1048] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.730494 10797 leader_election.cc:290] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:35.731036 10743 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:35.731587 10743 ts_tablet_manager.cc:1403] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.167s	user 0.018s	sys 0.003s
I20260709 13:05:35.732508 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:35.732508 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:35.734099 10474 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:35.736466 10590 leader_election.cc:343] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.739758 10538 tablet_service.cc:1511] Processing CreateTablet for tablet cf7697bdcf9f49ed9129a8fd785b67be (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:35.740960 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7697bdcf9f49ed9129a8fd785b67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.742621 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 559deb5d3dd44d6f8cd2e98f479b9168 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.743844 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559deb5d3dd44d6f8cd2e98f479b9168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.742637 10683 tablet_service.cc:1511] Processing CreateTablet for tablet cf7697bdcf9f49ed9129a8fd785b67be (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:35.764492 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 559deb5d3dd44d6f8cd2e98f479b9168 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.765826 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7697bdcf9f49ed9129a8fd785b67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.766070 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559deb5d3dd44d6f8cd2e98f479b9168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.764492 10463 tablet_service.cc:1511] Processing CreateTablet for tablet cf7697bdcf9f49ed9129a8fd785b67be (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:35.767333 10463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7697bdcf9f49ed9129a8fd785b67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.768293 10745 raft_consensus.cc:359] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.769845 10745 raft_consensus.cc:385] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.770151 10745 raft_consensus.cc:740] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.789954 10743 raft_consensus.cc:359] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.790733 10743 raft_consensus.cc:385] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.791015 10743 raft_consensus.cc:740] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.791978 10745 consensus_queue.cc:260] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.804494 10745 ts_tablet_manager.cc:1434] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.094s	user 0.005s	sys 0.007s
I20260709 13:05:35.805657 10745 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:35.807924 10791 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.808467 10791 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.810658 10791 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:35.811966 10610 tablet_service.cc:1511] Processing CreateTablet for tablet 3353aa19c6e4404fbcbd2f5e810074c5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:35.812708 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:35.813251 10610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3353aa19c6e4404fbcbd2f5e810074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.793629 10474 raft_consensus.cc:2468] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:35.814162 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:35.797857 10537 tablet_service.cc:1511] Processing CreateTablet for tablet 3353aa19c6e4404fbcbd2f5e810074c5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:35.791734 10743 consensus_queue.cc:260] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.816141 10537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3353aa19c6e4404fbcbd2f5e810074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.797801 10462 tablet_service.cc:1511] Processing CreateTablet for tablet 559deb5d3dd44d6f8cd2e98f479b9168 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:35.817793 10462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559deb5d3dd44d6f8cd2e98f479b9168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.817317 10622 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:35.822857 10465 tablet_service.cc:1511] Processing CreateTablet for tablet b08b155b1b294e4ea4a4b29755b98651 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:35.824091 10590 leader_election.cc:304] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:35.824416 10440 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:35.824342 10465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08b155b1b294e4ea4a4b29755b98651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.816152 10688 tablet_service.cc:1511] Processing CreateTablet for tablet 3353aa19c6e4404fbcbd2f5e810074c5 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:35.826208 10791 raft_consensus.cc:2804] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.826529 10688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3353aa19c6e4404fbcbd2f5e810074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.826615 10791 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.827196 10791 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.832067 10797 raft_consensus.cc:2804] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.832677 10797 raft_consensus.cc:697] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
W20260709 13:05:35.833068 10441 leader_election.cc:343] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.833506 10797 consensus_queue.cc:237] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.837707 10614 tablet_service.cc:1511] Processing CreateTablet for tablet b08b155b1b294e4ea4a4b29755b98651 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:35.839231 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08b155b1b294e4ea4a4b29755b98651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.837801 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 02a164c68f31403ab1ce21952d49ed2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:35.840866 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a164c68f31403ab1ce21952d49ed2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.836520 10683 tablet_service.cc:1511] Processing CreateTablet for tablet 02a164c68f31403ab1ce21952d49ed2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:35.842499 10683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a164c68f31403ab1ce21952d49ed2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.845970 10538 tablet_service.cc:1511] Processing CreateTablet for tablet b08b155b1b294e4ea4a4b29755b98651 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:35.846987 10538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08b155b1b294e4ea4a4b29755b98651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.855540 10743 ts_tablet_manager.cc:1434] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.124s	user 0.005s	sys 0.005s
I20260709 13:05:35.857246 10743 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:35.863584 10743 tablet_bootstrap.cc:654] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.869558 10536 tablet_service.cc:1511] Processing CreateTablet for tablet 02a164c68f31403ab1ce21952d49ed2e (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:35.870836 10746 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:35.873814 10746 ts_tablet_manager.cc:1403] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.269s	user 0.012s	sys 0.002s
I20260709 13:05:35.873001 10536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a164c68f31403ab1ce21952d49ed2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.886260 10753 raft_consensus.cc:493] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.886837 10753 raft_consensus.cc:515] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.887408 10746 raft_consensus.cc:359] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.888149 10746 raft_consensus.cc:385] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.888401 10746 raft_consensus.cc:740] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.889093 10753 leader_election.cc:290] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:35.891726 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:35.893216 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:35.893587 10745 tablet_bootstrap.cc:654] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.893903 10622 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:35.889086 10746 consensus_queue.cc:260] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.895853 10474 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:35.896100 10663 leader_election.cc:304] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:35.896948 10840 consensus_queue.cc:1048] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:35.898881 10612 tablet_service.cc:1511] Processing CreateTablet for tablet 225f2db0ded7415b81e387822b6e3856 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:35.897130 10753 raft_consensus.cc:2804] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.900233 10612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225f2db0ded7415b81e387822b6e3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.900295 10753 raft_consensus.cc:493] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.900794 10753 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.901711 10744 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:35.902235 10744 ts_tablet_manager.cc:1403] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.294s	user 0.020s	sys 0.015s
I20260709 13:05:35.908322 10458 tablet_service.cc:1511] Processing CreateTablet for tablet 225f2db0ded7415b81e387822b6e3856 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:35.910121 10458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225f2db0ded7415b81e387822b6e3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.915500 10685 tablet_service.cc:1511] Processing CreateTablet for tablet 225f2db0ded7415b81e387822b6e3856 (DEFAULT_TABLE table=test-workload [id=b99c7455decf43268bad04e2afb334bb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:35.916422 10744 raft_consensus.cc:359] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:35.916955 10685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225f2db0ded7415b81e387822b6e3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.917176 10744 raft_consensus.cc:385] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.917470 10744 raft_consensus.cc:740] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.917660 10753 raft_consensus.cc:515] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:35.918201 10744 consensus_queue.cc:260] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:35.927481 10746 ts_tablet_manager.cc:1434] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:05:35.929003 10746 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:35.953401 10744 ts_tablet_manager.cc:1434] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 13:05:35.935376 10791 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:35.960171 10369 catalog_manager.cc:5697] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.962751 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:35.963297 10753 leader_election.cc:290] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:35.964627 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:35.966148 10743 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:35.967981 10746 tablet_bootstrap.cc:654] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.999604 10743 ts_tablet_manager.cc:1403] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.143s	user 0.016s	sys 0.004s
I20260709 13:05:35.969866 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:36.002599 10474 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.002844 10743 raft_consensus.cc:359] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.003834 10743 raft_consensus.cc:385] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.004139 10743 raft_consensus.cc:740] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.961073 10791 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:35.963497 10624 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.974181 10744 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:35.970824 10441 leader_election.cc:343] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.993789 10831 consensus_queue.cc:1048] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.986688 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:36.013528 10744 tablet_bootstrap.cc:654] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.013870 10622 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.013563 10624 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:36.015322 10474 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:35.995683 10745 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.017860 10663 leader_election.cc:304] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:36.018270 10745 ts_tablet_manager.cc:1403] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.020s
I20260709 13:05:36.018874 10831 raft_consensus.cc:2804] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.019512 10831 raft_consensus.cc:697] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:36.020449 10831 consensus_queue.cc:237] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.024415 10744 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.024830 10622 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:36.025007 10744 ts_tablet_manager.cc:1403] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.001s
I20260709 13:05:36.026253 10440 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.028256 10743 consensus_queue.cc:260] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.027663 10791 raft_consensus.cc:2804] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.029708 10791 raft_consensus.cc:697] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:36.030490 10743 ts_tablet_manager.cc:1434] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:05:36.030622 10791 consensus_queue.cc:237] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.033644 10745 raft_consensus.cc:359] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.034446 10745 raft_consensus.cc:385] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.038693 10745 raft_consensus.cc:740] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.040567 10745 consensus_queue.cc:260] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.035671 10743 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.034264 10744 raft_consensus.cc:359] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.043612 10744 raft_consensus.cc:385] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.044001 10744 raft_consensus.cc:740] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.044695 10744 consensus_queue.cc:260] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.046895 10744 ts_tablet_manager.cc:1434] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.001s
I20260709 13:05:36.047204 10745 ts_tablet_manager.cc:1434] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 13:05:36.048496 10745 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.049531 10743 tablet_bootstrap.cc:654] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.050292 10744 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.051868 10746 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.052490 10746 ts_tablet_manager.cc:1403] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.124s	user 0.015s	sys 0.006s
I20260709 13:05:36.060398 10744 tablet_bootstrap.cc:654] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.060328 10746 raft_consensus.cc:359] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.061371 10746 raft_consensus.cc:385] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.063045 10746 raft_consensus.cc:740] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.064014 10746 consensus_queue.cc:260] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.069985 10746 ts_tablet_manager.cc:1434] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:36.072679 10746 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:36.072750 10743 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.070922 10371 catalog_manager.cc:5697] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.073388 10743 ts_tablet_manager.cc:1403] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.000s
I20260709 13:05:36.083750 10744 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.084309 10744 ts_tablet_manager.cc:1403] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.008s
I20260709 13:05:36.086122 10743 raft_consensus.cc:359] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.086994 10743 raft_consensus.cc:385] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.087273 10743 raft_consensus.cc:740] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.087183 10744 raft_consensus.cc:359] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.088021 10744 raft_consensus.cc:385] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.088307 10744 raft_consensus.cc:740] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.088250 10743 consensus_queue.cc:260] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.090564 10743 ts_tablet_manager.cc:1434] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:36.091048 10744 consensus_queue.cc:260] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.093756 10744 ts_tablet_manager.cc:1434] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:36.094578 10824 raft_consensus.cc:493] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.095290 10743 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.095119 10824 raft_consensus.cc:515] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.096654 10744 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.097522 10824 leader_election.cc:290] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:36.098783 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:36.102293 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.108448 10472 raft_consensus.cc:2393] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:36.109447 10746 tablet_bootstrap.cc:654] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.111348 10516 leader_election.cc:304] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:36.112344 10824 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.103070 10744 tablet_bootstrap.cc:654] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.120801 10824 raft_consensus.cc:2749] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.105239 10743 tablet_bootstrap.cc:654] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.126744 10746 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.082644 10745 tablet_bootstrap.cc:654] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.104259 10372 catalog_manager.cc:5697] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.127280 10746 ts_tablet_manager.cc:1403] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.001s
I20260709 13:05:36.133245 10745 tablet_bootstrap.cc:492] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.133792 10745 ts_tablet_manager.cc:1403] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.000s
W20260709 13:05:36.131014 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:36.142606 10746 raft_consensus.cc:359] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.143399 10746 raft_consensus.cc:385] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.143679 10746 raft_consensus.cc:740] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.144590 10745 raft_consensus.cc:359] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.145386 10745 raft_consensus.cc:385] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.145736 10745 raft_consensus.cc:740] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.146497 10745 consensus_queue.cc:260] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.144650 10746 consensus_queue.cc:260] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.147125 10824 raft_consensus.cc:493] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.148613 10744 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.147627 10824 raft_consensus.cc:515] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.149147 10744 ts_tablet_manager.cc:1403] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.053s	user 0.004s	sys 0.011s
I20260709 13:05:36.151413 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ebe12bfeac04b27ba5070e402b0307a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.152145 10698 raft_consensus.cc:2468] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:36.152107 10744 raft_consensus.cc:359] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.152942 10744 raft_consensus.cc:385] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.153251 10744 raft_consensus.cc:740] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.153923 10514 leader_election.cc:304] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:36.153978 10744 consensus_queue.cc:260] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.156467 10744 ts_tablet_manager.cc:1434] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:05:36.157126 10746 ts_tablet_manager.cc:1434] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 13:05:36.157879 10744 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.159636 10743 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.160189 10743 ts_tablet_manager.cc:1403] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.000s
I20260709 13:05:36.163178 10743 raft_consensus.cc:359] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.163981 10743 raft_consensus.cc:385] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.164249 10743 raft_consensus.cc:740] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.166132 10744 tablet_bootstrap.cc:654] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.166407 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ebe12bfeac04b27ba5070e402b0307a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.167124 10474 raft_consensus.cc:2468] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:36.169178 10746 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:36.172013 10824 leader_election.cc:290] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.176092 10743 consensus_queue.cc:260] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.178449 10743 ts_tablet_manager.cc:1434] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.004s
I20260709 13:05:36.179657 10867 raft_consensus.cc:2804] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.180179 10867 raft_consensus.cc:493] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.180496 10867 raft_consensus.cc:3060] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.184639 10743 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.190697 10744 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.191314 10744 ts_tablet_manager.cc:1403] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.003s
I20260709 13:05:36.194376 10744 raft_consensus.cc:359] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.195140 10744 raft_consensus.cc:385] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.195403 10744 raft_consensus.cc:740] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.197561 10745 ts_tablet_manager.cc:1434] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.063s	user 0.021s	sys 0.000s
I20260709 13:05:36.197368 10867 raft_consensus.cc:515] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.199919 10746 tablet_bootstrap.cc:654] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.202273 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ebe12bfeac04b27ba5070e402b0307a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.202979 10698 raft_consensus.cc:3060] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.203336 10867 leader_election.cc:290] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.191636 10743 tablet_bootstrap.cc:654] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.209550 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ebe12bfeac04b27ba5070e402b0307a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:36.210376 10474 raft_consensus.cc:3060] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.211939 10746 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.212476 10746 ts_tablet_manager.cc:1403] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.006s
I20260709 13:05:36.215619 10746 raft_consensus.cc:359] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.216512 10746 raft_consensus.cc:385] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.216804 10746 raft_consensus.cc:740] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.217175 10797 raft_consensus.cc:493] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.217708 10797 raft_consensus.cc:515] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.219210 10743 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.224808 10745 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.221160 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a9e5b696054a01b8aec158182b544d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.220732 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a9e5b696054a01b8aec158182b544d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.219841 10797 leader_election.cc:290] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.227921 10696 raft_consensus.cc:2468] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:36.219743 10743 ts_tablet_manager.cc:1403] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.005s
I20260709 13:05:36.229590 10588 leader_election.cc:304] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:36.224045 10698 raft_consensus.cc:2468] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:36.230489 10797 raft_consensus.cc:2804] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.230862 10797 raft_consensus.cc:493] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.231194 10797 raft_consensus.cc:3060] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.231151 10514 leader_election.cc:304] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:36.232386 10867 raft_consensus.cc:2804] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.233202 10867 raft_consensus.cc:697] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:36.220397 10474 raft_consensus.cc:2468] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:36.234263 10867 consensus_queue.cc:237] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.238097 10745 tablet_bootstrap.cc:654] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.238449 10746 consensus_queue.cc:260] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.238912 10744 consensus_queue.cc:260] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.241365 10472 raft_consensus.cc:2468] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:36.242820 10744 ts_tablet_manager.cc:1434] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:05:36.244668 10744 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.240027 10797 raft_consensus.cc:515] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.249459 10797 leader_election.cc:290] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.251155 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a9e5b696054a01b8aec158182b544d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:36.252496 10472 raft_consensus.cc:3060] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.254864 10744 tablet_bootstrap.cc:654] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.240427 10743 raft_consensus.cc:359] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.260074 10743 raft_consensus.cc:385] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.260430 10743 raft_consensus.cc:740] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.261294 10743 consensus_queue.cc:260] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.261543 10372 catalog_manager.cc:5697] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.263504 10743 ts_tablet_manager.cc:1434] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:05:36.264812 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a9e5b696054a01b8aec158182b544d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.265206 10472 raft_consensus.cc:2468] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:36.265565 10698 raft_consensus.cc:3060] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.268838 10590 leader_election.cc:304] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.271750 10797 raft_consensus.cc:2804] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.272583 10797 raft_consensus.cc:697] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:36.273749 10746 ts_tablet_manager.cc:1434] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.001s
I20260709 13:05:36.275058 10746 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:36.273761 10797 consensus_queue.cc:237] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.280455 10743 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.286090 10746 tablet_bootstrap.cc:654] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.292425 10698 raft_consensus.cc:2468] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:36.292425 10745 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.294154 10745 ts_tablet_manager.cc:1403] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.070s	user 0.021s	sys 0.011s
I20260709 13:05:36.296005 10746 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.296675 10746 ts_tablet_manager.cc:1403] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 13:05:36.298308 10745 raft_consensus.cc:359] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.301717 10745 raft_consensus.cc:385] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.302027 10745 raft_consensus.cc:740] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.302983 10745 consensus_queue.cc:260] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.304204 10842 consensus_queue.cc:1048] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.305960 10745 ts_tablet_manager.cc:1434] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:05:36.306834 10746 raft_consensus.cc:359] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.309281 10745 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.309959 10746 raft_consensus.cc:385] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.310115 10807 raft_consensus.cc:493] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.310530 10746 raft_consensus.cc:740] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.310678 10807 raft_consensus.cc:515] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.311646 10746 consensus_queue.cc:260] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.314661 10746 ts_tablet_manager.cc:1434] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:05:36.315611 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a759b0b5de444fa1c1486147b76cfa" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:36.316035 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a759b0b5de444fa1c1486147b76cfa" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:36.316705 10549 raft_consensus.cc:2468] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:36.316926 10807 leader_election.cc:290] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:36.317091 10745 tablet_bootstrap.cc:654] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.316735 10622 raft_consensus.cc:2468] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:36.319413 10663 leader_election.cc:304] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:36.320456 10744 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.320869 10746 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:36.320960 10744 ts_tablet_manager.cc:1403] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.077s	user 0.023s	sys 0.020s
I20260709 13:05:36.321985 10743 tablet_bootstrap.cc:654] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.324312 10744 raft_consensus.cc:359] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.325119 10744 raft_consensus.cc:385] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.325450 10744 raft_consensus.cc:740] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.320946 10807 raft_consensus.cc:2804] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.336705 10744 consensus_queue.cc:260] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.341260 10807 raft_consensus.cc:493] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.342010 10807 raft_consensus.cc:3060] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.342324 10369 catalog_manager.cc:5697] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.346418 10744 ts_tablet_manager.cc:1434] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 13:05:36.356688 10744 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.351341 10791 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.360268 10746 tablet_bootstrap.cc:654] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.360110 10791 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.362699 10791 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.363350 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.364091 10698 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:36.365140 10439 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.372365 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:36.374010 10441 leader_election.cc:343] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.376192 10791 raft_consensus.cc:2804] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.376744 10791 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.377074 10743 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.377147 10791 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.377689 10743 ts_tablet_manager.cc:1403] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.003s
I20260709 13:05:36.380658 10743 raft_consensus.cc:359] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.381364 10743 raft_consensus.cc:385] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.381642 10743 raft_consensus.cc:740] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.382421 10743 consensus_queue.cc:260] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.377140 10879 consensus_queue.cc:1048] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:36.381582 10744 tablet_bootstrap.cc:654] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.385110 10743 ts_tablet_manager.cc:1434] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:36.386859 10743 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.388230 10791 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.389348 10745 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.389834 10745 ts_tablet_manager.cc:1403] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.007s
I20260709 13:05:36.391587 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.392215 10698 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.394548 10743 tablet_bootstrap.cc:654] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.383414 10746 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.398689 10791 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.400576 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:36.408684 10746 ts_tablet_manager.cc:1403] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.003s
I20260709 13:05:36.408962 10698 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:36.411670 10807 raft_consensus.cc:515] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.412357 10744 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.412835 10744 ts_tablet_manager.cc:1403] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.002s
I20260709 13:05:36.412930 10439 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.414081 10791 raft_consensus.cc:2804] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.414593 10791 raft_consensus.cc:697] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:36.414633 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a759b0b5de444fa1c1486147b76cfa" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:36.415341 10549 raft_consensus.cc:3060] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.415628 10791 consensus_queue.cc:237] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.421142 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a759b0b5de444fa1c1486147b76cfa" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:36.422034 10624 raft_consensus.cc:3060] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.402033 10745 raft_consensus.cc:359] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.423667 10745 raft_consensus.cc:385] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.424573 10840 consensus_queue.cc:1048] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.426198 10549 raft_consensus.cc:2468] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:36.427098 10746 raft_consensus.cc:359] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.428013 10744 raft_consensus.cc:359] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.430545 10744 raft_consensus.cc:385] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.435911 10744 raft_consensus.cc:740] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.436756 10744 consensus_queue.cc:260] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.438710 10744 ts_tablet_manager.cc:1434] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.002s
I20260709 13:05:36.427665 10807 leader_election.cc:290] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:36.439020 10372 catalog_manager.cc:5697] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.429301 10664 leader_election.cc:304] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:36.441982 10744 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.444785 10840 raft_consensus.cc:2804] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.445415 10840 raft_consensus.cc:697] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:36.446568 10840 consensus_queue.cc:237] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.449219 10744 tablet_bootstrap.cc:654] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.428128 10745 raft_consensus.cc:740] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.450651 10745 consensus_queue.cc:260] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.452440 10547 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.454216 10791 consensus_queue.cc:1048] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.455090 10624 raft_consensus.cc:2468] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:36.458076 10743 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.458667 10743 ts_tablet_manager.cc:1403] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.032s
I20260709 13:05:36.459985 10879 raft_consensus.cc:493] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.460517 10879 raft_consensus.cc:515] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.462985 10879 leader_election.cc:290] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:36.464874 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:36.466238 10590 leader_election.cc:343] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.466084 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.467206 10698 raft_consensus.cc:2468] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:36.467497 10743 raft_consensus.cc:359] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.468653 10743 raft_consensus.cc:385] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.469066 10743 raft_consensus.cc:740] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.469249 10588 leader_election.cc:304] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.469967 10743 consensus_queue.cc:260] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.470567 10879 raft_consensus.cc:2804] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.471176 10879 raft_consensus.cc:493] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.471607 10879 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.472616 10743 ts_tablet_manager.cc:1434] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 13:05:36.477298 10743 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.483946 10753 consensus_queue.cc:1048] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.429173 10746 raft_consensus.cc:385] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.491892 10746 raft_consensus.cc:740] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.497670 10549 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.492614 10746 consensus_queue.cc:260] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.494953 10547 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:36.504252 10745 ts_tablet_manager.cc:1434] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.002s
I20260709 13:05:36.505348 10745 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.512696 10879 raft_consensus.cc:515] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.514943 10879 leader_election.cc:290] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:36.515952 10746 ts_tablet_manager.cc:1434] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.001s
I20260709 13:05:36.516517 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:36.517035 10746 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:36.518105 10590 leader_election.cc:343] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.519731 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.520381 10698 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.523469 10745 tablet_bootstrap.cc:654] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.527452 10743 tablet_bootstrap.cc:654] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.528823 10746 tablet_bootstrap.cc:654] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.530148 10892 consensus_queue.cc:1048] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.535248 10698 raft_consensus.cc:2468] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:36.540382 10588 leader_election.cc:304] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.541601 10879 raft_consensus.cc:2804] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.542400 10879 raft_consensus.cc:697] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:36.539523 10369 catalog_manager.cc:5697] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.543239 10879 consensus_queue.cc:237] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.560142 10743 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.560668 10743 ts_tablet_manager.cc:1403] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.014s
I20260709 13:05:36.564075 10743 raft_consensus.cc:359] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.564661 10743 raft_consensus.cc:385] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.564911 10743 raft_consensus.cc:740] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.565702 10743 consensus_queue.cc:260] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.567788 10743 ts_tablet_manager.cc:1434] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:36.568763 10743 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.575558 10743 tablet_bootstrap.cc:654] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.575611 10744 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.576237 10744 ts_tablet_manager.cc:1403] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.135s	user 0.024s	sys 0.057s
I20260709 13:05:36.579018 10744 raft_consensus.cc:359] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.579634 10745 tablet_bootstrap.cc:492] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.579735 10744 raft_consensus.cc:385] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.580153 10744 raft_consensus.cc:740] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.580263 10745 ts_tablet_manager.cc:1403] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.000s
I20260709 13:05:36.581103 10744 consensus_queue.cc:260] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.581022 10368 catalog_manager.cc:5697] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.582751 10746 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.583282 10746 ts_tablet_manager.cc:1403] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.000s
I20260709 13:05:36.583066 10745 raft_consensus.cc:359] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.583506 10744 ts_tablet_manager.cc:1434] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:05:36.583806 10745 raft_consensus.cc:385] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.584414 10745 raft_consensus.cc:740] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.584681 10892 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.585291 10892 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.590683 10744 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.594209 10745 consensus_queue.cc:260] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.592269 10746 raft_consensus.cc:359] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.592783 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.595355 10698 raft_consensus.cc:2468] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:36.594172 10840 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.596648 10840 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.598752 10840 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.599480 10439 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.600390 10785 raft_consensus.cc:2804] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.600765 10785 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.601082 10785 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.601712 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:36.602365 10743 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.602821 10743 ts_tablet_manager.cc:1403] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.003s
I20260709 13:05:36.603322 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.604713 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:36.605702 10743 raft_consensus.cc:359] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.606415 10743 raft_consensus.cc:385] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.606689 10743 raft_consensus.cc:740] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
W20260709 13:05:36.607101 10441 leader_election.cc:343] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.593720 10892 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.607477 10743 consensus_queue.cc:260] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:36.608042 10664 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:36.608767 10664 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.609164 10746 raft_consensus.cc:385] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.611929 10746 raft_consensus.cc:740] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.612679 10746 consensus_queue.cc:260] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.614712 10746 ts_tablet_manager.cc:1434] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.003s
I20260709 13:05:36.609203 10664 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.616184 10840 raft_consensus.cc:2749] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.616338 10744 tablet_bootstrap.cc:654] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.610055 10743 ts_tablet_manager.cc:1434] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:36.611016 10785 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.618124 10743 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:36.619024 10785 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.619668 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.620319 10698 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.625784 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:36.629690 10441 leader_election.cc:343] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.631366 10785 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.632086 10785 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.633549 10743 tablet_bootstrap.cc:654] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.596508 10745 ts_tablet_manager.cc:1434] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:05:36.636693 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:36.646700 10624 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:36.648865 10440 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.637524 10785 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:36.653694 10785 raft_consensus.cc:2804] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.655182 10785 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.656293 10785 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.639662 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:36.662545 10441 leader_election.cc:343] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.679450 10867 raft_consensus.cc:493] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.681592 10745 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.685740 10840 raft_consensus.cc:493] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.686515 10840 raft_consensus.cc:515] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.701221 10866 consensus_queue.cc:1048] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.709254 10746 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:36.680210 10867 raft_consensus.cc:515] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.722879 10744 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.723466 10744 ts_tablet_manager.cc:1403] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.139s	user 0.044s	sys 0.065s
I20260709 13:05:36.723677 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:36.724020 10698 raft_consensus.cc:2468] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:36.726153 10439 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.726513 10744 raft_consensus.cc:359] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.727218 10744 raft_consensus.cc:385] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.727486 10744 raft_consensus.cc:740] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.728312 10744 consensus_queue.cc:260] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.730175 10791 raft_consensus.cc:2804] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.730455 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159d5a82c8664fc79597d7f1715bf80e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:36.731124 10624 raft_consensus.cc:2468] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:36.733603 10744 ts_tablet_manager.cc:1434] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:36.733520 10663 leader_election.cc:304] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:36.734423 10807 raft_consensus.cc:2804] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.734781 10807 raft_consensus.cc:493] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.735128 10807 raft_consensus.cc:3060] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.744450 10840 leader_election.cc:290] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.745572 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159d5a82c8664fc79597d7f1715bf80e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.746914 10744 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.753229 10745 tablet_bootstrap.cc:654] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.756083 10744 tablet_bootstrap.cc:654] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.756902 10791 raft_consensus.cc:697] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:36.758270 10791 consensus_queue.cc:237] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:36.745334 10867 leader_election.cc:290] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:36.748039 10474 raft_consensus.cc:2468] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:36.761006 10879 raft_consensus.cc:493] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.761499 10879 raft_consensus.cc:515] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.762485 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.763305 10696 raft_consensus.cc:2393] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:36.763620 10879 leader_election.cc:290] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:36.767791 10514 leader_election.cc:304] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:36.769659 10807 raft_consensus.cc:515] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.771574 10746 tablet_bootstrap.cc:654] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.775854 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:36.778297 10867 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:36.781032 10590 leader_election.cc:343] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.782341 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.784571 10880 consensus_queue.cc:1048] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:36.785863 10588 leader_election.cc:343] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.786430 10880 raft_consensus.cc:493] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.786535 10588 leader_election.cc:304] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.787818 10842 raft_consensus.cc:2749] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.787308 10880 raft_consensus.cc:515] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.791370 10785 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.795131 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:36.792228 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.796062 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:36.797098 10590 leader_election.cc:343] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:36.797472 10441 leader_election.cc:343] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:36.798071 10590 leader_election.cc:343] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.798662 10590 leader_election.cc:304] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.799515 10807 leader_election.cc:290] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.799695 10842 raft_consensus.cc:2749] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.799752 10880 leader_election.cc:290] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:36.805112 10866 consensus_queue.cc:1048] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.805572 10785 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:36.810761 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159d5a82c8664fc79597d7f1715bf80e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:36.811420 10624 raft_consensus.cc:3060] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.814451 10867 raft_consensus.cc:2749] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.817876 10745 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:36.817879 10743 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.819633 10743 ts_tablet_manager.cc:1403] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.202s	user 0.016s	sys 0.008s
I20260709 13:05:36.819635 10745 ts_tablet_manager.cc:1403] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.138s	user 0.018s	sys 0.022s
I20260709 13:05:36.828624 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "159d5a82c8664fc79597d7f1715bf80e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:36.831609 10474 raft_consensus.cc:3060] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.832223 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:36.833109 10622 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.834798 10880 consensus_queue.cc:1048] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.840698 10743 raft_consensus.cc:359] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.841637 10743 raft_consensus.cc:385] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.842383 10743 raft_consensus.cc:740] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.848596 10474 raft_consensus.cc:2468] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:36.848645 10880 raft_consensus.cc:493] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.850152 10880 raft_consensus.cc:515] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.852928 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:36.853799 10696 raft_consensus.cc:2468] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:36.855273 10588 leader_election.cc:304] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:36.856202 10842 raft_consensus.cc:2804] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.856585 10842 raft_consensus.cc:493] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.856968 10842 raft_consensus.cc:3060] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.856707 10840 consensus_queue.cc:1048] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.865502 10842 raft_consensus.cc:515] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.868727 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:36.869549 10696 raft_consensus.cc:3060] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.874357 10831 consensus_queue.cc:1048] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.880241 10743 consensus_queue.cc:260] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.883131 10743 ts_tablet_manager.cc:1434] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:05:36.885623 10744 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:36.886354 10744 ts_tablet_manager.cc:1403] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.140s	user 0.030s	sys 0.033s
I20260709 13:05:36.884327 10745 raft_consensus.cc:359] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.887025 10745 raft_consensus.cc:385] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.887377 10745 raft_consensus.cc:740] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.888242 10745 consensus_queue.cc:260] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.896376 10743 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
W20260709 13:05:36.843544 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 58736.4us
I20260709 13:05:36.909111 10664 leader_election.cc:304] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:36.913971 10745 ts_tablet_manager.cc:1434] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:05:36.920116 10745 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:36.924118 10622 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:36.925441 10440 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:36.926636 10901 raft_consensus.cc:2804] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.927116 10901 raft_consensus.cc:697] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:36.927716 10624 raft_consensus.cc:2468] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:36.927968 10901 consensus_queue.cc:237] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:36.915187 10807 raft_consensus.cc:2804] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.929610 10807 raft_consensus.cc:697] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:36.930519 10807 consensus_queue.cc:237] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:36.931895 10743 tablet_bootstrap.cc:654] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.934062 10892 consensus_queue.cc:1048] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:36.936259 10880 leader_election.cc:290] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:36.932753 10744 raft_consensus.cc:359] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.940109 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:36.940840 10744 raft_consensus.cc:385] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.941175 10744 raft_consensus.cc:740] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.941532 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:36.941891 10744 consensus_queue.cc:260] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:36.943845 10744 ts_tablet_manager.cc:1434] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
W20260709 13:05:36.945114 10590 leader_election.cc:343] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:36.946447 10590 leader_election.cc:343] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.948961 10745 tablet_bootstrap.cc:654] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.947342 10744 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:36.966908 10743 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:36.967738 10743 ts_tablet_manager.cc:1403] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.004s
I20260709 13:05:36.961577 10696 raft_consensus.cc:2468] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:36.973737 10746 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:36.957283 10368 catalog_manager.cc:5697] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.036062 10746 ts_tablet_manager.cc:1403] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.327s	user 0.018s	sys 0.021s
I20260709 13:05:37.042932 10842 leader_election.cc:290] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
W20260709 13:05:37.040371 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 71176.9us
I20260709 13:05:37.047209 10745 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:37.048357 10745 ts_tablet_manager.cc:1403] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.129s	user 0.021s	sys 0.008s
I20260709 13:05:37.071552 10745 raft_consensus.cc:359] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.077818 10745 raft_consensus.cc:385] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.078207 10745 raft_consensus.cc:740] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.080895 10745 consensus_queue.cc:260] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.079907 10588 leader_election.cc:304] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:37.090755 10744 tablet_bootstrap.cc:654] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.098886 10745 ts_tablet_manager.cc:1434] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:05:37.109923 10745 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:37.125196 10746 raft_consensus.cc:359] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.125950 10746 raft_consensus.cc:385] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.126240 10746 raft_consensus.cc:740] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.127481 10791 consensus_queue.cc:1048] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.128574 10866 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.131621 10746 consensus_queue.cc:260] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.129127 10866 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.141860 10746 ts_tablet_manager.cc:1434] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.000s
I20260709 13:05:37.143042 10926 raft_consensus.cc:2804] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.150331 10746 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.150534 10866 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.150872 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.152081 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.153046 10623 raft_consensus.cc:2393] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:37.154901 10515 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:37.157030 10746 tablet_bootstrap.cc:654] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.163977 10866 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.170778 10745 tablet_bootstrap.cc:654] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.172283 10743 raft_consensus.cc:359] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.172998 10743 raft_consensus.cc:385] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.173246 10743 raft_consensus.cc:740] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.173930 10743 consensus_queue.cc:260] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.175480 10743 ts_tablet_manager.cc:1434] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.207s	user 0.005s	sys 0.002s
I20260709 13:05:37.176525 10743 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.170154 10926 raft_consensus.cc:697] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:37.189558 10926 consensus_queue.cc:237] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.188735 10743 tablet_bootstrap.cc:654] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.196664 10744 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:37.197208 10744 ts_tablet_manager.cc:1403] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.250s	user 0.024s	sys 0.003s
I20260709 13:05:37.203998 10746 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.204571 10746 ts_tablet_manager.cc:1403] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.005s
I20260709 13:05:37.205684 10744 raft_consensus.cc:359] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.206451 10744 raft_consensus.cc:385] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.206738 10744 raft_consensus.cc:740] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.207437 10744 consensus_queue.cc:260] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.207574 10746 raft_consensus.cc:359] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:37.207930 10441 consensus_peers.cc:597] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.208331 10746 raft_consensus.cc:385] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.208647 10746 raft_consensus.cc:740] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.209432 10746 consensus_queue.cc:260] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.209678 10842 consensus_queue.cc:1048] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.212577 10746 ts_tablet_manager.cc:1434] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:37.213795 10922 consensus_queue.cc:1048] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.214557 10746 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.218349 10744 ts_tablet_manager.cc:1434] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:05:37.219408 10744 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:37.225023 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:37.251556 10866 raft_consensus.cc:2749] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.252198 10785 consensus_queue.cc:1048] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:37.260473 10746 tablet_bootstrap.cc:654] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.262952 10840 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.263669 10924 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.263523 10840 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.264247 10924 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.266379 10840 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.266830 10924 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.267534 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.268190 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.269150 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.275738 10698 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:37.269416 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.277204 10623 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
W20260709 13:05:37.277889 10664 leader_election.cc:343] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:37.279096 10590 leader_election.cc:343] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.280224 10588 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.281597 10663 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.283311 10373 catalog_manager.cc:5697] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.284494 10926 raft_consensus.cc:2804] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.284955 10926 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.285288 10926 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.283254 10840 raft_consensus.cc:2804] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.293298 10840 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.293678 10840 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.294552 10369 catalog_manager.cc:5697] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.301654 10926 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.269688 10743 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.308532 10743 ts_tablet_manager.cc:1403] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.001s
I20260709 13:05:37.312366 10840 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.316746 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:37.318497 10664 leader_election.cc:343] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.319159 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:37.321177 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:37.322328 10698 raft_consensus.cc:2393] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:37.324401 10926 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.326671 10745 tablet_bootstrap.cc:492] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:37.326841 10840 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:37.328696 10590 leader_election.cc:343] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.327318 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:37.330701 10590 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.331421 10623 raft_consensus.cc:2393] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:37.331910 10926 raft_consensus.cc:2749] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.332840 10663 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.334246 10925 raft_consensus.cc:2749] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.329609 10745 ts_tablet_manager.cc:1403] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.020s
I20260709 13:05:37.327294 10743 raft_consensus.cc:359] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.339794 10743 raft_consensus.cc:385] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.342000 10743 raft_consensus.cc:740] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.347328 10743 consensus_queue.cc:260] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.347719 10368 catalog_manager.cc:5697] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.352298 10743 ts_tablet_manager.cc:1434] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:05:37.353685 10743 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.338963 10745 raft_consensus.cc:359] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.357210 10745 raft_consensus.cc:385] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.357687 10745 raft_consensus.cc:740] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.358831 10744 tablet_bootstrap.cc:654] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.358641 10745 consensus_queue.cc:260] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.363425 10745 ts_tablet_manager.cc:1434] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:05:37.364818 10785 raft_consensus.cc:493] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.365036 10745 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:37.365408 10785 raft_consensus.cc:515] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.366393 10743 tablet_bootstrap.cc:654] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.367810 10785 leader_election.cc:290] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.369516 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.369530 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08fdd13c2ece4a958ada223d6e631c9f" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.370819 10698 raft_consensus.cc:2393] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:37.372581 10439 leader_election.cc:304] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:37.373638 10785 raft_consensus.cc:3060] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.380796 10785 raft_consensus.cc:2749] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.387218 10743 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.388011 10743 ts_tablet_manager.cc:1403] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.000s
I20260709 13:05:37.394145 10743 raft_consensus.cc:359] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.395119 10743 raft_consensus.cc:385] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.395452 10743 raft_consensus.cc:740] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.396219 10743 consensus_queue.cc:260] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.398699 10743 ts_tablet_manager.cc:1434] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:37.400422 10743 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.406553 10743 tablet_bootstrap.cc:654] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.420248 10746 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.420789 10746 ts_tablet_manager.cc:1403] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.207s	user 0.058s	sys 0.071s
I20260709 13:05:37.422605 10745 tablet_bootstrap.cc:654] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.423489 10746 raft_consensus.cc:359] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.424232 10746 raft_consensus.cc:385] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.424530 10746 raft_consensus.cc:740] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.425206 10746 consensus_queue.cc:260] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.429229 10744 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:37.429737 10744 ts_tablet_manager.cc:1403] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.211s	user 0.010s	sys 0.002s
I20260709 13:05:37.436367 10746 ts_tablet_manager.cc:1434] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:37.436795 10743 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.437361 10743 ts_tablet_manager.cc:1403] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.020s
I20260709 13:05:37.437429 10746 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.434095 10744 raft_consensus.cc:359] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.439540 10744 raft_consensus.cc:385] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.439850 10744 raft_consensus.cc:740] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.440465 10743 raft_consensus.cc:359] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.440668 10744 consensus_queue.cc:260] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.441212 10743 raft_consensus.cc:385] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.441710 10743 raft_consensus.cc:740] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.442569 10743 consensus_queue.cc:260] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.445232 10743 ts_tablet_manager.cc:1434] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:05:37.445324 10744 ts_tablet_manager.cc:1434] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 13:05:37.445693 10746 tablet_bootstrap.cc:654] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.446321 10744 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:37.446498 10743 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.449995 10840 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.456840 10746 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.457396 10746 ts_tablet_manager.cc:1403] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:05:37.460570 10746 raft_consensus.cc:359] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.461256 10746 raft_consensus.cc:385] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.461535 10746 raft_consensus.cc:740] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.461829 10791 consensus_queue.cc:1048] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.462311 10746 consensus_queue.cc:260] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.464078 10746 ts_tablet_manager.cc:1434] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:05:37.450663 10840 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.466239 10746 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.465999 10926 raft_consensus.cc:493] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.472128 10926 raft_consensus.cc:515] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.486086 10745 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:37.486562 10745 ts_tablet_manager.cc:1403] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.122s	user 0.002s	sys 0.010s
I20260709 13:05:37.488245 10926 leader_election.cc:290] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.489862 10840 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:37.490621 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.485363 10791 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.504135 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.514221 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:37.514485 10588 leader_election.cc:343] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.495568 10791 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.517197 10791 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:37.517993 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.518281 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:37.518643 10590 leader_election.cc:343] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.519150 10590 leader_election.cc:304] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:37.519464 10664 leader_election.cc:343] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.517798 10746 tablet_bootstrap.cc:654] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.518668 10866 raft_consensus.cc:493] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.520723 10866 raft_consensus.cc:515] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.521186 10831 raft_consensus.cc:493] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.521689 10831 raft_consensus.cc:515] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.522907 10866 leader_election.cc:290] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.523478 10923 raft_consensus.cc:2749] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.523739 10831 leader_election.cc:290] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.525964 10743 tablet_bootstrap.cc:654] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.518683 10698 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:37.524719 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.525313 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.524403 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.528236 10622 raft_consensus.cc:2468] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:37.528514 10623 raft_consensus.cc:2468] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:37.529857 10515 leader_election.cc:304] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:37.530284 10663 leader_election.cc:304] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:37.530750 10866 raft_consensus.cc:2804] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.531183 10866 raft_consensus.cc:493] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.531404 10439 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:37.526609 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.531539 10866 raft_consensus.cc:3060] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.527626 10472 raft_consensus.cc:2468] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:37.535838 10791 raft_consensus.cc:2804] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.538157 10744 tablet_bootstrap.cc:654] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.539065 10909 raft_consensus.cc:493] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.539678 10909 raft_consensus.cc:515] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.542604 10743 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.542502 10909 leader_election.cc:290] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.543601 10743 ts_tablet_manager.cc:1403] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.001s
I20260709 13:05:37.543818 10791 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.544253 10791 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.546126 10921 consensus_queue.cc:1048] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:37.521229 10664 leader_election.cc:343] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.547734 10664 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.548832 10743 raft_consensus.cc:359] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.549160 10840 raft_consensus.cc:2749] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.549542 10743 raft_consensus.cc:385] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.549873 10743 raft_consensus.cc:740] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.549973 10840 consensus_queue.cc:1048] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.550581 10743 consensus_queue.cc:260] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.550848 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.550165 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.551645 10550 raft_consensus.cc:2468] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:37.536013 10831 raft_consensus.cc:2804] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.560312 10743 ts_tablet_manager.cc:1434] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:05:37.561427 10743 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
W20260709 13:05:37.562541 10441 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.564730 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.565600 10624 raft_consensus.cc:2468] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:37.569737 10743 tablet_bootstrap.cc:654] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.555462 10831 raft_consensus.cc:493] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.571048 10831 raft_consensus.cc:3060] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.572744 10745 raft_consensus.cc:359] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.573747 10745 raft_consensus.cc:385] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.574095 10745 raft_consensus.cc:740] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.575933 10515 leader_election.cc:304] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:37.576817 10909 raft_consensus.cc:2804] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.577207 10909 raft_consensus.cc:493] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.577567 10909 raft_consensus.cc:3060] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.569788 10925 consensus_queue.cc:1048] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:05:37.582736 10926 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.583288 10926 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.585379 10926 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:37.587740 10909 raft_consensus.cc:515] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.589883 10909 leader_election.cc:290] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.575032 10745 consensus_queue.cc:260] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.596312 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:37.600592 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.604519 10923 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.605209 10923 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.606925 10866 raft_consensus.cc:515] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.599507 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.612267 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:37.612289 10926 consensus_queue.cc:1048] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:05:37.617497 10624 raft_consensus.cc:3060] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.618408 10831 raft_consensus.cc:515] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.612550 10923 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:37.611454 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.619590 10698 raft_consensus.cc:2393] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:37.612509 10744 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:37.624012 10744 ts_tablet_manager.cc:1403] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.178s	user 0.014s	sys 0.000s
W20260709 13:05:37.610340 10590 leader_election.cc:343] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.636214 10831 leader_election.cc:290] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:37.636468 10590 leader_election.cc:343] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.637748 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:37.643666 10745 ts_tablet_manager.cc:1434] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.157s	user 0.008s	sys 0.002s
I20260709 13:05:37.646499 10745 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:37.629739 10791 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.647271 10624 raft_consensus.cc:2468] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:37.655230 10515 leader_election.cc:304] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:37.656105 10909 raft_consensus.cc:2804] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.657615 10746 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.656592 10909 raft_consensus.cc:697] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:37.658226 10746 ts_tablet_manager.cc:1403] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.193s	user 0.054s	sys 0.027s
I20260709 13:05:37.658844 10866 leader_election.cc:290] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.648695 10791 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:37.660233 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:37.661003 10622 raft_consensus.cc:3060] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.661458 10743 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.661976 10743 ts_tablet_manager.cc:1403] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.101s	user 0.015s	sys 0.004s
I20260709 13:05:37.649662 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:37.664050 10698 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.663702 10744 raft_consensus.cc:359] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.664538 10744 raft_consensus.cc:385] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.664839 10744 raft_consensus.cc:740] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.665562 10744 consensus_queue.cc:260] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.650686 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:37.668026 10744 ts_tablet_manager.cc:1434] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:05:37.668092 10548 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.669011 10746 raft_consensus.cc:359] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.669426 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027997962a9c49a19a76790e5f7ead90" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:37.672680 10472 raft_consensus.cc:3060] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.658879 10909 consensus_queue.cc:237] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.670825 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d38aaddc970432bb924dc77ac8bcfa7" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:37.671061 10744 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:37.676844 10743 raft_consensus.cc:359] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.672019 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.677577 10743 raft_consensus.cc:385] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.677886 10743 raft_consensus.cc:740] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
W20260709 13:05:37.678094 10590 leader_election.cc:343] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.678570 10590 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.679325 10923 raft_consensus.cc:2749] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.680186 10746 raft_consensus.cc:385] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.653244 10745 tablet_bootstrap.cc:654] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.680470 10746 raft_consensus.cc:740] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.681699 10743 consensus_queue.cc:260] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:37.683243 10590 leader_election.cc:343] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.683799 10590 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.675519 10548 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:37.688798 10743 ts_tablet_manager.cc:1434] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:05:37.684693 10928 raft_consensus.cc:2749] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.688565 10746 consensus_queue.cc:260] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.689868 10743 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.688280 10664 leader_election.cc:304] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.695982 10746 ts_tablet_manager.cc:1434] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 13:05:37.697014 10746 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.697751 10472 raft_consensus.cc:2468] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:37.698040 10441 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:37.698943 10928 raft_consensus.cc:493] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.699043 10516 leader_election.cc:304] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:37.699944 10892 raft_consensus.cc:2804] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.700374 10866 raft_consensus.cc:2804] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.701182 10866 raft_consensus.cc:697] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:37.702065 10866 consensus_queue.cc:237] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.703732 10746 tablet_bootstrap.cc:654] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.699539 10928 raft_consensus.cc:515] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.707330 10928 leader_election.cc:290] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:37.707692 10840 raft_consensus.cc:2749] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.700420 10892 raft_consensus.cc:697] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:37.708737 10892 consensus_queue.cc:237] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.710415 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.712057 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.712809 10474 raft_consensus.cc:2468] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:37.712673 10922 consensus_queue.cc:1048] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.713119 10622 raft_consensus.cc:2468] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
W20260709 13:05:37.712150 10588 leader_election.cc:343] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.720458 10698 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:37.730082 10744 tablet_bootstrap.cc:654] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.735176 10590 leader_election.cc:304] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:37.737051 10928 raft_consensus.cc:2804] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.737519 10928 raft_consensus.cc:493] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.737957 10928 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.740327 10743 tablet_bootstrap.cc:654] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.765199 10928 raft_consensus.cc:515] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.785616 10745 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:37.778483 10746 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.786207 10745 ts_tablet_manager.cc:1403] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.140s	user 0.016s	sys 0.034s
I20260709 13:05:37.786393 10746 ts_tablet_manager.cc:1403] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.090s	user 0.022s	sys 0.013s
I20260709 13:05:37.789021 10745 raft_consensus.cc:359] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.789752 10745 raft_consensus.cc:385] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.790024 10745 raft_consensus.cc:740] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.790591 10369 catalog_manager.cc:5697] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.791849 10745 consensus_queue.cc:260] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.794382 10744 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:37.794884 10744 ts_tablet_manager.cc:1403] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.000s
I20260709 13:05:37.795135 10745 ts_tablet_manager.cc:1434] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:37.794505 10369 catalog_manager.cc:5697] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.795359 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:37.796038 10472 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.796133 10745 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:37.797614 10744 raft_consensus.cc:359] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.798261 10744 raft_consensus.cc:385] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.798528 10744 raft_consensus.cc:740] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.802472 10745 tablet_bootstrap.cc:654] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.803211 10472 raft_consensus.cc:2468] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:37.792735 10743 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.793205 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:37.804179 10743 ts_tablet_manager.cc:1403] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.000s
I20260709 13:05:37.804585 10590 leader_election.cc:304] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:37.805485 10922 raft_consensus.cc:2804] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.805950 10922 raft_consensus.cc:697] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:37.806804 10743 raft_consensus.cc:359] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.807631 10743 raft_consensus.cc:385] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.807909 10743 raft_consensus.cc:740] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.806770 10922 consensus_queue.cc:237] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.808558 10743 consensus_queue.cc:260] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.810037 10743 ts_tablet_manager.cc:1434] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:37.810918 10743 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.813756 10746 raft_consensus.cc:359] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.824541 10746 raft_consensus.cc:385] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.824977 10746 raft_consensus.cc:740] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.822405 10743 tablet_bootstrap.cc:654] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.823247 10368 catalog_manager.cc:5697] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.815129 10373 catalog_manager.cc:5697] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.814958 10928 leader_election.cc:290] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:37.814996 10744 consensus_queue.cc:260] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.841801 10698 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.844465 10746 consensus_queue.cc:260] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.847697 10746 ts_tablet_manager.cc:1434] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.001s
I20260709 13:05:37.848747 10746 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.848598 10698 raft_consensus.cc:2468] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:37.851961 10743 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.852538 10743 ts_tablet_manager.cc:1403] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.004s
I20260709 13:05:37.855305 10746 tablet_bootstrap.cc:654] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.855283 10743 raft_consensus.cc:359] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.856194 10743 raft_consensus.cc:385] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.856495 10743 raft_consensus.cc:740] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.857371 10743 consensus_queue.cc:260] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.859927 10743 ts_tablet_manager.cc:1434] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:37.860939 10743 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.865425 10744 ts_tablet_manager.cc:1434] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 13:05:37.866744 10746 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.867116 10743 tablet_bootstrap.cc:654] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.867229 10746 ts_tablet_manager.cc:1403] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.019s	user 0.003s	sys 0.013s
I20260709 13:05:37.870060 10746 raft_consensus.cc:359] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.870730 10746 raft_consensus.cc:385] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.870981 10746 raft_consensus.cc:740] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.871589 10746 consensus_queue.cc:260] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:37.876740 10746 ts_tablet_manager.cc:1434] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:37.877761 10746 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.884691 10743 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.885303 10746 tablet_bootstrap.cc:654] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.885368 10743 ts_tablet_manager.cc:1403] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.011s
I20260709 13:05:37.884691 10745 tablet_bootstrap.cc:492] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:37.888988 10743 raft_consensus.cc:359] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.889751 10745 ts_tablet_manager.cc:1403] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.094s	user 0.019s	sys 0.005s
I20260709 13:05:37.890172 10743 raft_consensus.cc:385] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.890558 10743 raft_consensus.cc:740] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.891396 10743 consensus_queue.cc:260] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.892791 10745 raft_consensus.cc:359] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.897882 10745 raft_consensus.cc:385] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.898202 10745 raft_consensus.cc:740] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.898875 10745 consensus_queue.cc:260] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.899338 10743 ts_tablet_manager.cc:1434] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:05:37.897114 10746 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:37.900171 10746 ts_tablet_manager.cc:1403] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.002s
I20260709 13:05:37.900621 10743 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:37.903105 10753 raft_consensus.cc:493] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.904067 10928 raft_consensus.cc:493] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.904542 10928 raft_consensus.cc:515] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.905370 10866 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.905856 10866 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.897505 10744 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:37.906776 10928 leader_election.cc:290] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:37.907971 10866 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:37.908691 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.908875 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.910560 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.911373 10743 tablet_bootstrap.cc:654] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:37.911373 10590 leader_election.cc:343] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.903630 10753 raft_consensus.cc:515] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.912552 10516 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:37.913874 10866 raft_consensus.cc:2749] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:37.915072 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.915640 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:37.915957 10698 raft_consensus.cc:2468] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:37.918418 10744 tablet_bootstrap.cc:654] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.916105 10746 raft_consensus.cc:359] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.925683 10746 raft_consensus.cc:385] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.926442 10746 raft_consensus.cc:740] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.920485 10588 leader_election.cc:304] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.927626 10746 consensus_queue.cc:260] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:37.931255 10746 ts_tablet_manager.cc:1434] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:05:37.932677 10746 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:37.940623 10744 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
W20260709 13:05:37.940555 10664 leader_election.cc:343] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.941483 10744 ts_tablet_manager.cc:1403] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.018s
I20260709 13:05:37.944322 10744 raft_consensus.cc:359] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.944494 10753 leader_election.cc:290] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:37.945253 10744 raft_consensus.cc:385] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.945567 10744 raft_consensus.cc:740] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.946240 10746 tablet_bootstrap.cc:654] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.946367 10744 consensus_queue.cc:260] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.947141 10928 raft_consensus.cc:2804] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.951359 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:37.947718 10928 raft_consensus.cc:493] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.952899 10928 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.958961 10922 raft_consensus.cc:493] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.959707 10922 raft_consensus.cc:515] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.956753 10744 ts_tablet_manager.cc:1434] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:37.963536 10922 leader_election.cc:290] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:37.963721 10744 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:37.965368 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f11079f7def4cef8f18ca580808c5d5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:37.967010 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f11079f7def4cef8f18ca580808c5d5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:37.967674 10698 raft_consensus.cc:2468] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:37.958966 10746 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
W20260709 13:05:37.968626 10590 leader_election.cc:343] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:37.970508 10588 leader_election.cc:304] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:37.971354 10922 raft_consensus.cc:2804] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:37.971712 10922 raft_consensus.cc:493] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.972080 10922 raft_consensus.cc:3060] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.973209 10745 ts_tablet_manager.cc:1434] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 13:05:37.976765 10745 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:37.991506 10928 raft_consensus.cc:515] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:37.992806 10622 raft_consensus.cc:2393] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:37.994639 10663 leader_election.cc:304] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:37.974699 10744 tablet_bootstrap.cc:654] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.968714 10746 ts_tablet_manager.cc:1403] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.008s
I20260709 13:05:37.995754 10753 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.999009 10746 raft_consensus.cc:359] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:37.999706 10746 raft_consensus.cc:385] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.000267 10746 raft_consensus.cc:740] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.001015 10746 consensus_queue.cc:260] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.002874 10743 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:37.989007 10922 raft_consensus.cc:515] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.003399 10743 ts_tablet_manager.cc:1403] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.072s
I20260709 13:05:38.005636 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.006350 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
W20260709 13:05:38.007162 10590 leader_election.cc:343] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.006309 10743 raft_consensus.cc:359] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.007735 10743 raft_consensus.cc:385] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.008078 10743 raft_consensus.cc:740] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.009044 10743 consensus_queue.cc:260] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.008097 10745 tablet_bootstrap.cc:654] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.010977 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f11079f7def4cef8f18ca580808c5d5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:38.011632 10696 raft_consensus.cc:3060] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.011904 10922 leader_election.cc:290] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:38.012061 10928 leader_election.cc:290] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:38.003300 10746 ts_tablet_manager.cc:1434] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:05:38.014081 10746 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:38.015439 10743 ts_tablet_manager.cc:1434] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:05:38.016729 10743 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:38.019021 10696 raft_consensus.cc:2468] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:38.020334 10588 leader_election.cc:304] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:38.022264 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f11079f7def4cef8f18ca580808c5d5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:38.023064 10928 raft_consensus.cc:2804] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.024003 10928 raft_consensus.cc:697] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:38.024981 10743 tablet_bootstrap.cc:654] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.023064 10474 raft_consensus.cc:3060] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.025113 10928 consensus_queue.cc:237] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.035804 10745 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.036475 10745 ts_tablet_manager.cc:1403] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.000s
I20260709 13:05:38.037147 10746 tablet_bootstrap.cc:654] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.040185 10745 raft_consensus.cc:359] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.041184 10745 raft_consensus.cc:385] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.041554 10745 raft_consensus.cc:740] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.042470 10745 consensus_queue.cc:260] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.044852 10745 ts_tablet_manager.cc:1434] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:38.045881 10745 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.047416 10753 raft_consensus.cc:2749] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.050861 10743 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.051576 10743 ts_tablet_manager.cc:1403] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:05:38.052531 10746 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:38.058405 10746 ts_tablet_manager.cc:1403] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.001s
I20260709 13:05:38.055337 10743 raft_consensus.cc:359] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.056094 10698 raft_consensus.cc:2468] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:38.061018 10743 raft_consensus.cc:385] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.062657 10588 leader_election.cc:304] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.065141 10743 raft_consensus.cc:740] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.065891 10743 consensus_queue.cc:260] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.066799 10928 raft_consensus.cc:2804] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.067668 10743 ts_tablet_manager.cc:1434] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:05:38.063220 10745 tablet_bootstrap.cc:654] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.068114 10744 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:38.062325 10368 catalog_manager.cc:5697] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.067824 10928 raft_consensus.cc:697] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:38.069757 10743 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:38.070070 10928 consensus_queue.cc:237] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.070598 10892 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.071478 10892 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.060719 10474 raft_consensus.cc:2468] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:38.074968 10892 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.076403 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.076622 10922 raft_consensus.cc:493] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.077080 10922 raft_consensus.cc:515] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:38.078176 10441 leader_election.cc:343] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.079211 10922 leader_election.cc:290] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:38.082084 10743 tablet_bootstrap.cc:654] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.068648 10744 ts_tablet_manager.cc:1403] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.105s	user 0.037s	sys 0.023s
I20260709 13:05:38.082448 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.064007 10746 raft_consensus.cc:359] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.084419 10746 raft_consensus.cc:385] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.084751 10746 raft_consensus.cc:740] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.074877 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.086812 10746 consensus_queue.cc:260] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.089749 10744 raft_consensus.cc:359] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.090816 10744 raft_consensus.cc:385] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.091262 10744 raft_consensus.cc:740] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.091413 10746 ts_tablet_manager.cc:1434] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:05:38.094643 10746 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:38.096148 10892 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.096803 10892 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.100998 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:38.101176 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.101714 10622 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
W20260709 13:05:38.102623 10441 leader_election.cc:343] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.103401 10440 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.104413 10901 raft_consensus.cc:2804] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.104935 10901 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.105175 10791 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.106978 10791 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.099656 10892 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.110183 10791 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.111234 10744 consensus_queue.cc:260] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.112083 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:38.112819 10624 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:38.113262 10744 ts_tablet_manager.cc:1434] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:05:38.114455 10744 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:38.105505 10590 leader_election.cc:343] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.116145 10440 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.119095 10866 consensus_queue.cc:1048] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.122833 10698 raft_consensus.cc:2468] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:38.105389 10901 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.104020 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.128252 10791 raft_consensus.cc:2804] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.127897 10892 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.142613 10439 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.143697 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.144052 10785 raft_consensus.cc:2804] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.144510 10548 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:38.144665 10785 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.145263 10785 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.147320 10909 consensus_queue.cc:1048] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.150779 10746 tablet_bootstrap.cc:654] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.137862 10791 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:38.157373 10588 leader_election.cc:343] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.161578 10588 leader_election.cc:304] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:38.163949 10922 raft_consensus.cc:2749] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.156306 10791 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.141438 10892 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.178540 10368 catalog_manager.cc:5697] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.181962 10892 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.183065 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:38.183736 10622 raft_consensus.cc:2468] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:38.176420 10909 consensus_queue.cc:1048] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.185045 10440 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.191290 10922 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.192211 10922 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.194765 10791 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.196385 10909 consensus_queue.cc:1048] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.196779 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.198055 10922 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
W20260709 13:05:38.198711 10590 leader_election.cc:343] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.199201 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:38.200822 10590 leader_election.cc:343] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.201301 10590 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.201740 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:38.202344 10623 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.202711 10928 raft_consensus.cc:2749] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.200155 10791 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.203810 10791 raft_consensus.cc:2804] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.204211 10791 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.204825 10791 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:38.215999 10980 log.cc:927] Time spent T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:05:38.229288 10744 tablet_bootstrap.cc:654] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.230209 10901 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.234236 10745 tablet_bootstrap.cc:492] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.234843 10745 ts_tablet_manager.cc:1403] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.189s	user 0.010s	sys 0.012s
I20260709 13:05:38.238332 10743 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.239580 10743 ts_tablet_manager.cc:1403] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.170s	user 0.025s	sys 0.023s
I20260709 13:05:38.237098 10901 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.254407 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.254739 10745 raft_consensus.cc:359] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.255666 10745 raft_consensus.cc:385] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.256666 10745 raft_consensus.cc:740] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.264122 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:38.265122 10620 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.269332 10745 consensus_queue.cc:260] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.272166 10922 raft_consensus.cc:493] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.297043 10745 ts_tablet_manager.cc:1434] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.000s
I20260709 13:05:38.302062 10745 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.310704 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.311422 10548 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.301127 10922 raft_consensus.cc:515] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.334385 10744 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:38.337172 10744 ts_tablet_manager.cc:1403] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.223s	user 0.022s	sys 0.014s
I20260709 13:05:38.340154 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.340976 10928 raft_consensus.cc:493] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.341565 10928 raft_consensus.cc:515] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.343448 10745 tablet_bootstrap.cc:654] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.346518 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.329381 10983 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.347357 10983 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.349020 10791 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.349675 10983 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
W20260709 13:05:38.349344 10590 leader_election.cc:343] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.356995 10985 consensus_queue.cc:1048] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.358067 10743 raft_consensus.cc:359] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.358776 10743 raft_consensus.cc:385] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.359048 10743 raft_consensus.cc:740] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.360466 10791 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.359700 10743 consensus_queue.cc:260] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:38.351719 10984 log.cc:927] Time spent T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:05:38.354259 10928 leader_election.cc:290] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:38.372834 10743 ts_tablet_manager.cc:1434] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.133s	user 0.007s	sys 0.000s
I20260709 13:05:38.377799 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.378728 10697 raft_consensus.cc:2468] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:38.361346 10922 leader_election.cc:290] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:38.385667 10743 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:38.388293 10439 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.394987 10983 raft_consensus.cc:2804] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.395435 10983 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.411281 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.414319 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.416163 10623 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
W20260709 13:05:38.400826 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 52567.7us
I20260709 13:05:38.418674 10548 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:38.420185 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.398416 10983 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.422050 10620 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
W20260709 13:05:38.423494 10590 leader_election.cc:343] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.425246 10743 tablet_bootstrap.cc:654] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.429523 10994 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.430140 10994 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.432456 10994 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:38.373731 10785 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.444680 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.445940 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.451189 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
W20260709 13:05:38.442859 10441 leader_election.cc:343] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.452332 10623 raft_consensus.cc:2393] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:38.455295 10744 raft_consensus.cc:359] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.458323 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.459776 10698 raft_consensus.cc:2393] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
W20260709 13:05:38.460974 10588 leader_election.cc:343] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.461479 10588 leader_election.cc:304] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.462123 10992 consensus_queue.cc:1048] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.464711 10547 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.465878 10992 raft_consensus.cc:2749] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:38.467296 10588 leader_election.cc:343] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.467881 10588 leader_election.cc:304] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:38.468331 10441 leader_election.cc:343] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.469700 10441 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:38.470074 10901 consensus_queue.cc:1048] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.469117 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:38.471869 10694 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.474078 10985 raft_consensus.cc:2804] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.475521 10985 raft_consensus.cc:697] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:38.476413 10985 consensus_queue.cc:237] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:38.480033 10441 leader_election.cc:343] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:38.482451 10441 leader_election.cc:343] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.481016 10983 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.488490 10983 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.490310 10548 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.491079 10746 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:38.492504 10440 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.491675 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.492797 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:38.491640 10514 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:38.493722 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:38.529780 10697 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.530045 10623 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.530246 10985 raft_consensus.cc:2804] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.494508 10922 raft_consensus.cc:2749] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.496820 10745 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.533969 10745 ts_tablet_manager.cc:1403] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.234s	user 0.010s	sys 0.036s
I20260709 13:05:38.534045 10909 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.535758 10992 consensus_queue.cc:1048] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:38.539783 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 48229.1us
I20260709 13:05:38.457947 10785 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.507131 10744 raft_consensus.cc:385] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:38.509584 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:38.509624 10694 raft_consensus.cc:2468] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:38.518858 10746 ts_tablet_manager.cc:1403] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.425s	user 0.010s	sys 0.014s
I20260709 13:05:38.523756 10743 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.542903 10744 raft_consensus.cc:740] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.544462 10744 consensus_queue.cc:260] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.545424 10743 ts_tablet_manager.cc:1403] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.171s	user 0.018s	sys 0.009s
I20260709 13:05:38.542232 10985 raft_consensus.cc:697] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:38.547043 10985 consensus_queue.cc:237] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.549881 10744 ts_tablet_manager.cc:1434] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.212s	user 0.005s	sys 0.002s
I20260709 13:05:38.550817 10744 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:38.550982 10924 consensus_queue.cc:1048] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.556106 10745 raft_consensus.cc:359] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.557020 10745 raft_consensus.cc:385] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.557325 10745 raft_consensus.cc:740] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.558470 10745 consensus_queue.cc:260] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.560860 10746 raft_consensus.cc:359] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.561091 10439 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.561620 10746 raft_consensus.cc:385] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.561906 10746 raft_consensus.cc:740] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.562258 10785 raft_consensus.cc:2804] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.562795 10785 raft_consensus.cc:697] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:38.562568 10746 consensus_queue.cc:260] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.563632 10785 consensus_queue.cc:237] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.568444 10744 tablet_bootstrap.cc:654] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.584661 10753 raft_consensus.cc:493] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.585206 10753 raft_consensus.cc:515] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.586896 10623 raft_consensus.cc:2468] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:38.587320 10753 leader_election.cc:290] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.589865 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.590546 10475 raft_consensus.cc:2468] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:38.591224 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.591904 10664 leader_election.cc:304] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.592054 10440 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:38.592880 10664 leader_election.cc:343] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.593180 10901 raft_consensus.cc:2804] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.593665 10901 raft_consensus.cc:697] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:38.593840 10988 raft_consensus.cc:2804] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:38.594208 10988 raft_consensus.cc:493] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.578946 10743 raft_consensus.cc:359] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.595165 10743 raft_consensus.cc:385] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.588066 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.591626 10745 ts_tablet_manager.cc:1434] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:05:38.596726 10746 ts_tablet_manager.cc:1434] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:05:38.597651 10746 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:38.597695 10745 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.595525 10743 raft_consensus.cc:740] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.601747 10743 consensus_queue.cc:260] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.594532 10988 raft_consensus.cc:3060] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.594523 10901 consensus_queue.cc:237] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.578575 10697 raft_consensus.cc:2468] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:38.608253 10743 ts_tablet_manager.cc:1434] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.002s
I20260709 13:05:38.603047 10549 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.613168 10743 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
W20260709 13:05:38.615553 10997 log.cc:927] Time spent T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:05:38.616380 10922 consensus_queue.cc:1048] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:38.624554 10439 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:38.626215 10928 consensus_queue.cc:1048] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.633026 10745 tablet_bootstrap.cc:654] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.643554 10985 raft_consensus.cc:2804] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:38.645006 10441 leader_election.cc:343] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.677245 10743 tablet_bootstrap.cc:654] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.674181 10746 tablet_bootstrap.cc:654] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.688596 10928 consensus_queue.cc:1048] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.663668 10549 raft_consensus.cc:2468] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:38.724674 10988 raft_consensus.cc:515] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.728650 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:38.729465 10472 raft_consensus.cc:3060] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:38.730998 10368 catalog_manager.cc:5697] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.732813 10368 catalog_manager.cc:5697] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.734607 10368 catalog_manager.cc:5697] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.737390 10368 catalog_manager.cc:5697] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.695271 10909 raft_consensus.cc:2749] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.748369 10745 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.749091 10745 ts_tablet_manager.cc:1403] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.152s	user 0.013s	sys 0.009s
I20260709 13:05:38.752562 10472 raft_consensus.cc:2468] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:38.752630 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:38.754092 10664 leader_election.cc:304] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
W20260709 13:05:38.755282 10664 leader_election.cc:343] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.755487 10745 raft_consensus.cc:359] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.756111 10753 raft_consensus.cc:2804] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:38.756554 10745 raft_consensus.cc:385] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.756724 10753 raft_consensus.cc:697] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:38.756994 10745 raft_consensus.cc:740] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.757987 10753 consensus_queue.cc:237] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.765614 10745 consensus_queue.cc:260] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.773247 10745 ts_tablet_manager.cc:1434] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:05:38.786806 10745 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.765017 10988 leader_election.cc:290] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.796109 10985 raft_consensus.cc:697] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:38.807024 10745 tablet_bootstrap.cc:654] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.808405 10985 consensus_queue.cc:237] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.815868 10744 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:38.816448 10744 ts_tablet_manager.cc:1403] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.266s	user 0.031s	sys 0.041s
I20260709 13:05:38.819250 10744 raft_consensus.cc:359] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.820144 10744 raft_consensus.cc:385] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.820415 10744 raft_consensus.cc:740] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.821318 10744 consensus_queue.cc:260] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.827035 10744 ts_tablet_manager.cc:1434] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:38.828379 10744 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:38.832490 10992 raft_consensus.cc:493] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.833251 10992 raft_consensus.cc:515] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.835731 10992 leader_election.cc:290] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:38.836227 10746 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:38.845983 10746 ts_tablet_manager.cc:1403] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.248s	user 0.011s	sys 0.010s
I20260709 13:05:38.849291 10745 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.850189 10745 ts_tablet_manager.cc:1403] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.007s
I20260709 13:05:38.836236 10744 tablet_bootstrap.cc:654] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.836854 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:38.853078 10588 leader_election.cc:343] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.853183 10745 raft_consensus.cc:359] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.854328 10745 raft_consensus.cc:385] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.854737 10746 raft_consensus.cc:359] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.855433 10746 raft_consensus.cc:385] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.855710 10746 raft_consensus.cc:740] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.854741 10745 raft_consensus.cc:740] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.842193 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.856858 10746 consensus_queue.cc:260] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:38.858112 10590 leader_election.cc:343] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.858565 10590 leader_election.cc:304] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.859391 10746 ts_tablet_manager.cc:1434] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:05:38.859640 10992 raft_consensus.cc:2749] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.859920 10744 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:38.860466 10746 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:38.860539 10744 ts_tablet_manager.cc:1403] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.000s
I20260709 13:05:38.840947 10743 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.863421 10744 raft_consensus.cc:359] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.865286 10744 raft_consensus.cc:385] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.865582 10744 raft_consensus.cc:740] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.866281 10744 consensus_queue.cc:260] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.864297 10745 consensus_queue.cc:260] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:38.868909 10745 ts_tablet_manager.cc:1434] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:05:38.869827 10745 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.873656 10370 catalog_manager.cc:5697] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.875955 10745 tablet_bootstrap.cc:654] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.879559 10743 ts_tablet_manager.cc:1403] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.268s	user 0.027s	sys 0.021s
I20260709 13:05:38.883822 10744 ts_tablet_manager.cc:1434] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:05:38.884162 10746 tablet_bootstrap.cc:654] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.884261 10743 raft_consensus.cc:359] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.884882 10744 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:38.885128 10743 raft_consensus.cc:385] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.885547 10743 raft_consensus.cc:740] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.886294 10743 consensus_queue.cc:260] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.889958 10743 ts_tablet_manager.cc:1434] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:38.890837 10743 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:38.892860 10744 tablet_bootstrap.cc:654] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.896618 10743 tablet_bootstrap.cc:654] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.905114 10373 catalog_manager.cc:5697] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.913877 10743 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.914484 10743 ts_tablet_manager.cc:1403] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.000s
I20260709 13:05:38.917464 10743 raft_consensus.cc:359] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.918066 10743 raft_consensus.cc:385] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.918296 10743 raft_consensus.cc:740] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.919013 10743 consensus_queue.cc:260] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.920951 10745 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:38.921506 10745 ts_tablet_manager.cc:1403] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.011s
I20260709 13:05:38.924170 10745 raft_consensus.cc:359] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.924767 10992 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.925288 10992 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.926384 10746 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:38.926927 10746 ts_tablet_manager.cc:1403] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.010s
I20260709 13:05:38.929111 10992 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:38.929782 10743 ts_tablet_manager.cc:1434] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:05:38.930169 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.930800 10743 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:38.930706 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:38.931706 10590 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.931514 10746 raft_consensus.cc:359] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.932277 10746 raft_consensus.cc:385] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:38.932493 10590 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.933053 10590 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:38.934491 10992 raft_consensus.cc:2749] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.924820 10745 raft_consensus.cc:385] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.932569 10746 raft_consensus.cc:740] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.935216 10745 raft_consensus.cc:740] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.936576 10745 consensus_queue.cc:260] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.939145 10745 ts_tablet_manager.cc:1434] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:38.940490 10745 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:38.941356 10744 tablet_bootstrap.cc:492] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:38.939795 10743 tablet_bootstrap.cc:654] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.941844 10744 ts_tablet_manager.cc:1403] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.004s
I20260709 13:05:38.939731 10746 consensus_queue.cc:260] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.944559 10746 ts_tablet_manager.cc:1434] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:38.949460 10745 tablet_bootstrap.cc:654] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.953022 10746 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:38.953078 10743 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:38.953823 10743 ts_tablet_manager.cc:1403] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.002s
I20260709 13:05:38.956748 10992 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.957195 10992 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:38.959188 10992 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:38.959893 10746 tablet_bootstrap.cc:654] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.960969 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:38.962612 10588 leader_election.cc:343] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.967128 10744 raft_consensus.cc:359] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.968133 10744 raft_consensus.cc:385] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.968464 10744 raft_consensus.cc:740] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.968866 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.969450 10744 consensus_queue.cc:260] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:38.970633 10590 leader_election.cc:343] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:38.971083 10590 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:38.980062 10992 raft_consensus.cc:2749] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:38.984309 10983 consensus_queue.cc:1048] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:38.986768 10922 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.987321 10922 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:38.989454 10922 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:38.990923 10909 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.991465 10909 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:38.993541 10909 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:38.994469 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:38.995092 10698 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:38.996052 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:38.996402 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:38.996727 10550 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:38.997246 10624 raft_consensus.cc:2393] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:38.997637 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:38.998855 10588 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:38.999748 10922 raft_consensus.cc:2804] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.977056 10744 ts_tablet_manager.cc:1434] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:05:39.000249 10922 raft_consensus.cc:493] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.000875 10922 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.001361 10744 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:39.004539 10516 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:39.007004 10746 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:39.006438 10743 raft_consensus.cc:359] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.007527 10746 ts_tablet_manager.cc:1403] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.003s
I20260709 13:05:39.010357 10746 raft_consensus.cc:359] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.011005 10746 raft_consensus.cc:385] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.011274 10746 raft_consensus.cc:740] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.007336 11016 consensus_queue.cc:1048] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.013360 10995 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.014321 10745 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:39.014823 10745 ts_tablet_manager.cc:1403] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.075s	user 0.008s	sys 0.015s
I20260709 13:05:39.007572 10743 raft_consensus.cc:385] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.016515 10743 raft_consensus.cc:740] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.017236 10909 raft_consensus.cc:493] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.017174 10743 consensus_queue.cc:260] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.017807 10909 raft_consensus.cc:515] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.020818 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.021606 10623 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:39.021948 10743 ts_tablet_manager.cc:1434] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:05:39.021730 10745 raft_consensus.cc:359] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.022467 10745 raft_consensus.cc:385] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.022759 10745 raft_consensus.cc:740] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.023406 10745 consensus_queue.cc:260] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.023697 10743 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:39.023901 10515 leader_election.cc:304] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:39.025522 10745 ts_tablet_manager.cc:1434] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:39.025887 10994 raft_consensus.cc:2804] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.026726 10994 raft_consensus.cc:493] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.027043 10994 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.027225 10745 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:39.027906 10909 leader_election.cc:290] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:39.029057 10744 tablet_bootstrap.cc:654] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.030205 10995 raft_consensus.cc:2749] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.030193 10922 raft_consensus.cc:515] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.024302 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:39.033102 10698 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:39.037860 10745 tablet_bootstrap.cc:654] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.038841 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:39.039441 10698 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.039723 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eefd49b51434d4786968ac53f50899c" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:39.041208 10550 raft_consensus.cc:3060] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.046638 10698 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:39.046803 10994 raft_consensus.cc:515] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.047907 10588 leader_election.cc:304] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:39.048688 10928 raft_consensus.cc:2804] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.049168 10928 raft_consensus.cc:697] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:39.049979 10928 consensus_queue.cc:237] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.026083 10746 consensus_queue.cc:260] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.054405 10746 ts_tablet_manager.cc:1434] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.000s
I20260709 13:05:39.055234 10745 tablet_bootstrap.cc:492] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:39.057870 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:39.058358 10922 leader_election.cc:290] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:39.058526 10623 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.059576 10745 ts_tablet_manager.cc:1403] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 13:05:39.062342 10745 raft_consensus.cc:359] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.062947 10745 raft_consensus.cc:385] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.063190 10745 raft_consensus.cc:740] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.066311 10743 tablet_bootstrap.cc:654] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.066403 10623 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:39.078120 10515 leader_election.cc:304] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:39.068166 10994 leader_election.cc:290] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:39.068079 10745 consensus_queue.cc:260] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.069242 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:39.079370 10695 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.079954 10994 raft_consensus.cc:2804] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.080845 10994 raft_consensus.cc:697] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
W20260709 13:05:39.073127 11034 log.cc:927] Time spent T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:39.084208 10994 consensus_queue.cc:237] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.086614 10745 ts_tablet_manager.cc:1434] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:05:39.092365 10745 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:39.089648 11016 consensus_queue.cc:1048] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.096204 10695 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:39.101128 10745 tablet_bootstrap.cc:654] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.102896 10983 consensus_queue.cc:1048] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.104475 10983 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.104969 10983 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.109691 10550 raft_consensus.cc:2468] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:39.109808 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.110682 10744 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:39.110612 10624 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:39.111159 10983 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.111620 10744 ts_tablet_manager.cc:1403] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.110s	user 0.018s	sys 0.021s
I20260709 13:05:39.055580 10746 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:39.111999 10440 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.114940 11016 raft_consensus.cc:2804] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.115315 11016 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.115655 11016 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.116803 10744 raft_consensus.cc:359] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.117626 10744 raft_consensus.cc:385] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.118322 10744 raft_consensus.cc:740] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.119045 10745 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:39.119205 10744 consensus_queue.cc:260] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.119534 10745 ts_tablet_manager.cc:1403] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:05:39.121172 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.123049 10745 raft_consensus.cc:359] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.123713 10745 raft_consensus.cc:385] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.124008 10745 raft_consensus.cc:740] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.124665 10745 consensus_queue.cc:260] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.125622 10744 ts_tablet_manager.cc:1434] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:39.126961 10744 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:39.130070 10441 leader_election.cc:343] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.130496 10368 catalog_manager.cc:5697] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 2, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.131888 10745 ts_tablet_manager.cc:1434] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:05:39.132956 10745 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:39.130471 11016 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.137512 11016 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.138326 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:39.138978 10550 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.139556 10745 tablet_bootstrap.cc:654] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.140470 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:39.141183 10624 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.148492 10744 tablet_bootstrap.cc:654] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.153990 10983 consensus_queue.cc:1048] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.158288 11020 raft_consensus.cc:493] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.158886 11020 raft_consensus.cc:515] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.162214 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.162923 10548 raft_consensus.cc:2468] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:39.164572 10664 leader_election.cc:304] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:39.165421 10753 raft_consensus.cc:2804] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.165788 10753 raft_consensus.cc:493] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.166126 10753 raft_consensus.cc:3060] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.172545 11020 leader_election.cc:290] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.173889 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.174806 10620 raft_consensus.cc:2468] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:39.172400 10753 raft_consensus.cc:515] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.180644 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:39.181340 10548 raft_consensus.cc:3060] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.184413 10744 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:39.184962 10983 consensus_queue.cc:1048] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.187628 10995 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.188371 10995 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.189482 10744 ts_tablet_manager.cc:1403] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 13:05:39.191742 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.192449 10744 raft_consensus.cc:359] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.192623 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.193151 10744 raft_consensus.cc:385] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.193462 10744 raft_consensus.cc:740] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.194106 10744 consensus_queue.cc:260] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.195013 10995 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.195909 10516 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:39.197958 10547 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.205905 10892 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.206525 10892 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.208717 10892 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:39.210115 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:39.211018 10695 raft_consensus.cc:2468] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:39.212499 10995 raft_consensus.cc:2749] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.213204 10439 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.216579 11020 raft_consensus.cc:493] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.217321 11020 raft_consensus.cc:515] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.220460 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.221407 10620 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:39.223992 11020 leader_election.cc:290] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.224964 10663 leader_election.cc:304] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:39.220412 10743 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:39.226876 10743 ts_tablet_manager.cc:1403] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.203s	user 0.022s	sys 0.008s
I20260709 13:05:39.227656 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.215405 11016 raft_consensus.cc:2804] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.228359 10472 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:39.228691 11016 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.227465 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.229264 11016 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.230597 10746 tablet_bootstrap.cc:700] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent opening tablet: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:05:39.232879 10988 raft_consensus.cc:2804] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.233330 10988 raft_consensus.cc:493] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.233788 10988 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.236510 10831 raft_consensus.cc:493] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.237128 10831 raft_consensus.cc:515] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.238073 11020 consensus_queue.cc:1048] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.242992 10746 tablet_bootstrap.cc:654] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.253669 10753 leader_election.cc:290] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.233809 10892 consensus_queue.cc:1048] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:05:39.251065 10368 catalog_manager.cc:5697] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.279482 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dff2cbddb694bf8a854bf2a1960649d" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:39.280287 10621 raft_consensus.cc:3060] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:39.284330 10441 leader_election.cc:343] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.287163 10546 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.287489 10988 raft_consensus.cc:515] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.289705 10621 raft_consensus.cc:2468] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.300263 10548 raft_consensus.cc:2468] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.302315 10744 ts_tablet_manager.cc:1434] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.112s	user 0.006s	sys 0.000s
I20260709 13:05:39.304962 10744 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:39.308005 10831 leader_election.cc:290] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.321367 10624 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:39.323979 10440 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.325299 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.325493 10983 raft_consensus.cc:2804] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.326259 10983 raft_consensus.cc:697] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:39.327343 10983 consensus_queue.cc:237] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.319319 10664 leader_election.cc:304] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:39.331682 10550 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:39.332752 10831 raft_consensus.cc:2804] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:39.333020 10664 leader_election.cc:343] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.334836 10831 raft_consensus.cc:697] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:39.335711 10831 consensus_queue.cc:237] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.347361 10743 raft_consensus.cc:359] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.350764 10743 raft_consensus.cc:385] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.351503 10743 raft_consensus.cc:740] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.355142 10743 consensus_queue.cc:260] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.353160 10909 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.368386 10743 ts_tablet_manager.cc:1434] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.141s	user 0.005s	sys 0.002s
I20260709 13:05:39.370352 10743 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:39.374331 10746 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:39.374943 10746 ts_tablet_manager.cc:1403] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.320s	user 0.011s	sys 0.007s
I20260709 13:05:39.377698 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:39.377880 10988 leader_election.cc:290] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.378444 10472 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.365794 10909 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.377781 10746 raft_consensus.cc:359] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.383644 10746 raft_consensus.cc:385] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.384016 10746 raft_consensus.cc:740] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.384677 10746 consensus_queue.cc:260] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.399971 10892 consensus_queue.cc:1048] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.406100 10746 ts_tablet_manager.cc:1434] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.001s
I20260709 13:05:39.407279 10746 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:39.408890 10745 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:39.409484 10745 ts_tablet_manager.cc:1403] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.277s	user 0.084s	sys 0.103s
I20260709 13:05:39.384241 10922 raft_consensus.cc:493] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.431465 11016 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.412040 10922 raft_consensus.cc:515] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:39.434041 11016 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:39.443994 10992 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.444684 10992 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.444862 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.445923 10622 raft_consensus.cc:2468] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:39.446956 10992 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
W20260709 13:05:39.448923 10664 consensus_peers.cc:597] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:39.451522 10922 leader_election.cc:290] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:39.456813 10663 leader_election.cc:304] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:39.457340 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.458585 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.461064 10909 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.461791 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.462198 10745 raft_consensus.cc:359] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.462931 10745 raft_consensus.cc:385] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.463201 10745 raft_consensus.cc:740] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.463944 10745 consensus_queue.cc:260] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.465287 10472 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.466135 10745 ts_tablet_manager.cc:1434] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:05:39.470103 10745 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
W20260709 13:05:39.468518 10590 leader_election.cc:343] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:39.471815 10590 leader_election.cc:343] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.472282 10590 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:39.473857 10590 leader_election.cc:343] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:39.471412 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 66387.6us
I20260709 13:05:39.475958 10992 raft_consensus.cc:2749] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:39.467829 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 51760.6us
I20260709 13:05:39.476634 10831 raft_consensus.cc:2804] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.489509 10696 raft_consensus.cc:1275] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:39.491938 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:39.492666 10664 leader_election.cc:304] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.493526 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:39.495092 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.501484 10696 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.501835 11048 raft_consensus.cc:2804] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.502429 11048 raft_consensus.cc:697] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:39.503340 11048 consensus_queue.cc:237] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.495810 10831 raft_consensus.cc:493] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.523259 10744 tablet_bootstrap.cc:654] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.556416 10743 tablet_bootstrap.cc:654] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.531912 10588 leader_election.cc:343] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.533047 11054 consensus_queue.cc:1048] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.534909 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:39.561213 10992 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.561702 10622 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.536945 10368 catalog_manager.cc:5697] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.541492 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:39.507640 10909 consensus_queue.cc:1048] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:05:39.543969 10901 consensus_queue.cc:1048] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.547923 10922 consensus_queue.cc:1048] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.549714 10746 tablet_bootstrap.cc:654] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.555892 10745 tablet_bootstrap.cc:654] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.565275 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 56881.7us
I20260709 13:05:39.559630 10831 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.559864 10588 leader_election.cc:304] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:39.592651 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:39.593457 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 61568.8us
I20260709 13:05:39.561757 10992 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.611203 11052 raft_consensus.cc:2749] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.614586 10696 raft_consensus.cc:2468] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:39.616551 10439 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.618629 11016 raft_consensus.cc:2804] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.619148 11016 raft_consensus.cc:697] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:39.620038 11016 consensus_queue.cc:237] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:39.652139 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 54503.7us
I20260709 13:05:39.657143 10514 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:39.658604 10995 raft_consensus.cc:2749] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.659637 10992 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.673260 10744 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:39.674134 10744 ts_tablet_manager.cc:1403] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.370s	user 0.010s	sys 0.010s
I20260709 13:05:39.689836 10622 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.690016 10831 raft_consensus.cc:515] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.691574 10994 consensus_queue.cc:1048] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:39.692822 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.700876 10746 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:39.712220 10746 ts_tablet_manager.cc:1403] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.305s	user 0.017s	sys 0.005s
W20260709 13:05:39.675860 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 81012.1us
I20260709 13:05:39.740947 10745 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:39.741787 10745 ts_tablet_manager.cc:1403] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.272s	user 0.022s	sys 0.002s
I20260709 13:05:39.727068 10909 raft_consensus.cc:493] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.748651 10831 leader_election.cc:290] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.757669 10745 raft_consensus.cc:359] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.758428 10745 raft_consensus.cc:385] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.758695 10745 raft_consensus.cc:740] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.753708 10909 raft_consensus.cc:515] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.761325 10746 raft_consensus.cc:359] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.790688 10746 raft_consensus.cc:385] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.791040 10746 raft_consensus.cc:740] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.791750 10746 consensus_queue.cc:260] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.792531 10909 leader_election.cc:290] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:39.796798 10744 raft_consensus.cc:359] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:39.782058 10441 leader_election.cc:343] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.782585 10985 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:39.815897 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 68158.3us
I20260709 13:05:39.818678 10368 catalog_manager.cc:5697] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.784250 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:39.820814 10473 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.788542 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:39.759354 10745 consensus_queue.cc:260] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:39.780030 10590 leader_election.cc:343] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:39.824342 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 44358.7us
I20260709 13:05:39.825913 10745 ts_tablet_manager.cc:1434] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.003s
I20260709 13:05:39.798611 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:39.803629 10546 raft_consensus.cc:1275] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:39.826810 10622 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.827039 10745 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:39.804991 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:39.806293 10744 raft_consensus.cc:385] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.828833 10744 raft_consensus.cc:740] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.829562 10744 consensus_queue.cc:260] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:39.803633 10753 consensus_queue.cc:1048] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:39.808064 10746 ts_tablet_manager.cc:1434] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.002s
I20260709 13:05:39.835493 10746 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:39.838392 10622 raft_consensus.cc:2468] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.840055 11056 consensus_queue.cc:1048] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.831672 10744 ts_tablet_manager.cc:1434] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.157s	user 0.008s	sys 0.002s
W20260709 13:05:39.806308 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 79695.4us
W20260709 13:05:39.850099 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 68081.6us
I20260709 13:05:39.846864 10985 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.854198 10985 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:39.861366 10744 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:39.870276 10588 leader_election.cc:343] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.870882 10588 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:39.842197 11069 consensus_queue.cc:1048] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:39.876287 10922 raft_consensus.cc:2749] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.880391 10663 leader_election.cc:304] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:39.884986 10550 raft_consensus.cc:3060] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.885785 10988 raft_consensus.cc:2804] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.886384 10988 raft_consensus.cc:697] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:39.887265 10988 consensus_queue.cc:237] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:39.888314 10545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:39.892604 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:39.893532 10623 raft_consensus.cc:2468] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:39.896534 10440 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:39.898777 10901 raft_consensus.cc:2804] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.899290 10901 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.900179 10744 tablet_bootstrap.cc:654] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.899713 10901 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.928188 10746 tablet_bootstrap.cc:654] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.934628 10745 tablet_bootstrap.cc:654] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:39.936367 10441 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.951006 10473 raft_consensus.cc:2468] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:39.951668 10743 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:39.908113 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d59609def584fcca66488c4e7073630" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:39.956204 10743 ts_tablet_manager.cc:1403] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.587s	user 0.036s	sys 0.054s
I20260709 13:05:39.962489 10743 raft_consensus.cc:359] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.963456 10743 raft_consensus.cc:385] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.971868 10743 raft_consensus.cc:740] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.973049 10743 consensus_queue.cc:260] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:39.979341 10373 catalog_manager.cc:5697] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.985323 10743 ts_tablet_manager.cc:1434] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:05:39.986848 10743 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:40.005712 10791 consensus_queue.cc:1048] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:05:40.025360 10514 leader_election.cc:304] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:40.026705 10994 raft_consensus.cc:2749] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:40.049314 11084 log.cc:927] Time spent T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:05:40.037232 11016 consensus_queue.cc:1048] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:05:40.056911 10744 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:40.002970 10901 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.058283 10744 ts_tablet_manager.cc:1403] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.197s	user 0.034s	sys 0.042s
I20260709 13:05:40.062510 10901 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:40.064312 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.065026 10743 tablet_bootstrap.cc:654] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.066713 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:40.064951 10623 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:40.081023 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:40.094216 11076 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.095005 11076 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.098681 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.099929 10696 raft_consensus.cc:2393] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
W20260709 13:05:40.084075 10441 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.100234 11069 consensus_queue.cc:1048] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.106922 10744 raft_consensus.cc:359] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.122435 10744 raft_consensus.cc:385] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.122920 10744 raft_consensus.cc:740] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.122331 10369 catalog_manager.cc:5697] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.123616 10744 consensus_queue.cc:260] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.125875 10744 ts_tablet_manager.cc:1434] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 13:05:40.126891 10744 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:40.131989 11069 consensus_queue.cc:1048] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.134974 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:40.133328 11076 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:40.136832 10516 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:40.138167 11088 log.cc:927] Time spent T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:40.140326 11076 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.153311 10743 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:40.164320 10743 ts_tablet_manager.cc:1403] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.178s	user 0.008s	sys 0.007s
I20260709 13:05:40.167472 10743 raft_consensus.cc:359] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.172313 10743 raft_consensus.cc:385] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.172655 10743 raft_consensus.cc:740] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.173352 10743 consensus_queue.cc:260] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.175200 11016 consensus_queue.cc:1048] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.152895 10368 catalog_manager.cc:5697] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.157681 10745 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:40.176801 10745 ts_tablet_manager.cc:1403] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.350s	user 0.025s	sys 0.004s
I20260709 13:05:40.158890 10623 raft_consensus.cc:2468] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:40.188549 10440 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.189791 11016 raft_consensus.cc:2804] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.190323 11016 raft_consensus.cc:697] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:40.189803 10746 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:40.191013 10746 ts_tablet_manager.cc:1403] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.356s	user 0.018s	sys 0.007s
I20260709 13:05:40.191207 11016 consensus_queue.cc:237] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.193740 10746 raft_consensus.cc:359] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.194461 10746 raft_consensus.cc:385] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.194769 10746 raft_consensus.cc:740] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.171582 11076 raft_consensus.cc:2749] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.201164 10791 consensus_queue.cc:1048] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.195483 10746 consensus_queue.cc:260] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.206741 10746 ts_tablet_manager.cc:1434] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:05:40.207757 10746 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:40.181736 10745 raft_consensus.cc:359] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.210402 10745 raft_consensus.cc:385] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.210682 10745 raft_consensus.cc:740] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.175192 10743 ts_tablet_manager.cc:1434] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:40.213641 10744 tablet_bootstrap.cc:654] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.224164 10745 consensus_queue.cc:260] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.226730 10745 ts_tablet_manager.cc:1434] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.002s
I20260709 13:05:40.229121 10743 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:40.229872 10746 tablet_bootstrap.cc:654] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.234941 10745 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:40.238010 10746 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:40.238659 10746 ts_tablet_manager.cc:1403] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 13:05:40.241501 10746 raft_consensus.cc:359] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.242162 10746 raft_consensus.cc:385] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.242380 10746 raft_consensus.cc:740] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.243016 10746 consensus_queue.cc:260] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.245162 10746 ts_tablet_manager.cc:1434] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:40.246181 10746 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:40.251236 10744 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:40.256486 10746 tablet_bootstrap.cc:654] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.257302 10744 ts_tablet_manager.cc:1403] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.131s	user 0.009s	sys 0.004s
I20260709 13:05:40.261191 10744 raft_consensus.cc:359] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.261888 10744 raft_consensus.cc:385] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.262163 10744 raft_consensus.cc:740] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.263600 10744 consensus_queue.cc:260] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.265884 10744 ts_tablet_manager.cc:1434] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:40.266846 10744 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:40.267968 10743 tablet_bootstrap.cc:654] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.272198 10745 tablet_bootstrap.cc:654] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.273736 10744 tablet_bootstrap.cc:654] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.283890 11076 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.275914 10746 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:40.285233 10746 ts_tablet_manager.cc:1403] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.000s
I20260709 13:05:40.284756 11076 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.288414 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.289119 10697 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:40.291081 11048 raft_consensus.cc:493] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.288231 10746 raft_consensus.cc:359] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.291637 11048 raft_consensus.cc:515] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.292130 10746 raft_consensus.cc:385] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.292541 10746 raft_consensus.cc:740] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.293251 10746 consensus_queue.cc:260] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.295717 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:40.301810 10664 leader_election.cc:343] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.302265 10746 ts_tablet_manager.cc:1434] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:40.296532 11048 leader_election.cc:290] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:40.298269 10901 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.303092 10901 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.303655 10746 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:40.306036 10901 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:40.296244 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:40.307087 10620 raft_consensus.cc:2468] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:40.308979 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:40.309643 10546 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.311298 10746 tablet_bootstrap.cc:654] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.312534 10441 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:40.314146 11016 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.314524 11016 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.314855 11016 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.314800 10663 leader_election.cc:304] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.316110 11048 raft_consensus.cc:2804] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.316468 11048 raft_consensus.cc:493] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.316768 11048 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.322908 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.323695 10696 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.323195 10369 catalog_manager.cc:5697] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.322829 11016 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.326486 11016 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:40.290889 10514 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:40.327792 10994 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.328177 10994 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.328508 10994 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.329975 10743 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:40.330631 10743 ts_tablet_manager.cc:1403] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.119s	user 0.027s	sys 0.011s
I20260709 13:05:40.332533 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:40.333199 10697 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.334090 10743 raft_consensus.cc:359] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.335026 10743 raft_consensus.cc:385] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.335345 10743 raft_consensus.cc:740] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.335568 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:40.334137 11048 raft_consensus.cc:515] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.338820 11048 leader_election.cc:290] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:40.339489 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:40.341032 10664 leader_election.cc:343] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.343564 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.344338 10623 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.344306 10743 consensus_queue.cc:260] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.347472 10743 ts_tablet_manager.cc:1434] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:40.348161 11076 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:40.352139 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:40.352999 10473 raft_consensus.cc:2393] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:40.354781 10994 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.355257 10743 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:40.356068 10546 raft_consensus.cc:2393] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:40.359848 10697 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:40.361373 10994 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:40.361500 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:40.362324 10743 tablet_bootstrap.cc:654] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.362485 10696 raft_consensus.cc:2393] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:40.363179 10439 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.364482 11016 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.364948 11016 raft_consensus.cc:697] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:40.365794 11016 consensus_queue.cc:237] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.369338 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:40.369742 10746 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:40.370503 10746 ts_tablet_manager.cc:1403] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.007s
I20260709 13:05:40.372962 10516 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:40.373468 10746 raft_consensus.cc:359] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.374189 10746 raft_consensus.cc:385] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.374461 10746 raft_consensus.cc:740] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.375604 10623 raft_consensus.cc:2468] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:40.375710 10746 consensus_queue.cc:260] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.376763 10663 leader_election.cc:304] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.377884 10994 raft_consensus.cc:2749] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.378046 10746 ts_tablet_manager.cc:1434] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:40.377972 11048 raft_consensus.cc:2804] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.380443 11048 raft_consensus.cc:697] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:40.380805 10746 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:40.381287 11048 consensus_queue.cc:237] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.388236 10988 consensus_queue.cc:1048] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.389256 10745 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:40.389822 10745 ts_tablet_manager.cc:1403] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.162s	user 0.011s	sys 0.017s
I20260709 13:05:40.392520 10745 raft_consensus.cc:359] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.393198 10745 raft_consensus.cc:385] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.393471 10745 raft_consensus.cc:740] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.395524 10744 tablet_bootstrap.cc:492] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:40.396100 10744 ts_tablet_manager.cc:1403] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.009s
I20260709 13:05:40.401641 10744 raft_consensus.cc:359] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.403937 10744 raft_consensus.cc:385] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.404213 10744 raft_consensus.cc:740] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.404961 10744 consensus_queue.cc:260] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.405887 10746 tablet_bootstrap.cc:654] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.406611 10744 ts_tablet_manager.cc:1434] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:05:40.407408 10745 consensus_queue.cc:260] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.408319 10994 raft_consensus.cc:493] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.408838 10994 raft_consensus.cc:515] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.412868 11067 raft_consensus.cc:493] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.413780 11067 raft_consensus.cc:515] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.415758 10744 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:40.420958 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:40.421738 10623 raft_consensus.cc:2468] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:40.421223 11067 leader_election.cc:290] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.424199 10994 leader_election.cc:290] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.425029 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:40.424409 10368 catalog_manager.cc:5697] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.426589 10515 leader_election.cc:304] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:40.427726 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.428269 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.428476 10697 raft_consensus.cc:2468] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
W20260709 13:05:40.430160 10588 leader_election.cc:343] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.431484 10994 raft_consensus.cc:2804] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.431941 10744 tablet_bootstrap.cc:654] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.432016 10994 raft_consensus.cc:493] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.432612 10994 raft_consensus.cc:3060] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:40.434465 10590 leader_election.cc:343] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.434953 10590 leader_election.cc:304] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.435839 10745 ts_tablet_manager.cc:1434] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.000s
I20260709 13:05:40.436908 11067 raft_consensus.cc:2749] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.437817 10745 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:40.438176 11067 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.446458 10994 raft_consensus.cc:515] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.448092 10745 tablet_bootstrap.cc:654] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.450449 10994 leader_election.cc:290] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.453965 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:40.454658 10697 raft_consensus.cc:3060] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.455266 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.456032 10622 raft_consensus.cc:3060] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.466195 10697 raft_consensus.cc:2468] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:40.468196 10514 leader_election.cc:304] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:40.469218 11076 raft_consensus.cc:2804] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.469730 11076 raft_consensus.cc:697] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:40.470608 11076 consensus_queue.cc:237] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.439285 11067 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.477502 10746 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:40.478107 10746 ts_tablet_manager.cc:1403] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.098s	user 0.008s	sys 0.004s
I20260709 13:05:40.478758 11067 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:40.485684 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:40.487545 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:40.489462 10590 leader_election.cc:343] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:40.490775 10590 leader_election.cc:343] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.491216 10590 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.490007 11016 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.500268 11016 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.499552 10622 raft_consensus.cc:2468] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:40.502717 11016 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:40.504252 10744 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:40.504526 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:40.504833 10744 ts_tablet_manager.cc:1403] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.003s
I20260709 13:05:40.507850 10746 raft_consensus.cc:359] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.504854 11048 consensus_queue.cc:1048] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:40.509244 10746 raft_consensus.cc:385] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.509683 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:40.510442 10623 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
W20260709 13:05:40.512760 10441 leader_election.cc:343] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.515203 11067 raft_consensus.cc:2749] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.518314 10743 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:40.518864 10743 ts_tablet_manager.cc:1403] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.164s	user 0.010s	sys 0.012s
I20260709 13:05:40.523144 10440 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.524468 11054 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.524892 11054 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.525251 11054 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.509725 10746 raft_consensus.cc:740] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.532871 10743 raft_consensus.cc:359] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:40.533726 10743 raft_consensus.cc:385] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.539075 10369 catalog_manager.cc:5697] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.542043 10746 consensus_queue.cc:260] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.544091 10746 ts_tablet_manager.cc:1434] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.000s
I20260709 13:05:40.585934 10746 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:40.588593 10440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 43702.9us
I20260709 13:05:40.600049 10744 raft_consensus.cc:359] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.600936 10744 raft_consensus.cc:385] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.601238 10744 raft_consensus.cc:740] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.601955 10744 consensus_queue.cc:260] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.605576 11054 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.606494 10791 raft_consensus.cc:493] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.620173 10791 raft_consensus.cc:515] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.621543 11054 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:40.622208 10791 leader_election.cc:290] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:40.606954 10901 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.623124 10901 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.625329 10744 ts_tablet_manager.cc:1434] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.120s	user 0.006s	sys 0.000s
I20260709 13:05:40.625627 10901 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.626441 10744 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:40.628172 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.629711 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.630411 10901 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.609382 11068 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.631222 11068 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:40.631624 10439 leader_election.cc:343] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.633599 11068 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:40.609337 10745 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:40.634491 10745 ts_tablet_manager.cc:1403] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.197s	user 0.061s	sys 0.063s
I20260709 13:05:40.637650 10745 raft_consensus.cc:359] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.638396 10745 raft_consensus.cc:385] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.638697 10745 raft_consensus.cc:740] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.639197 10744 tablet_bootstrap.cc:654] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.615291 10992 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.640043 10901 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.640316 10992 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.639436 10745 consensus_queue.cc:260] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.643838 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:40.608886 10441 consensus_peers.cc:597] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.644609 10697 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.630481 10696 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.645855 10901 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.647327 10745 ts_tablet_manager.cc:1434] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:05:40.648130 10439 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:40.648425 10745 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:40.650249 11054 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.651170 11054 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.651559 11054 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.653627 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:40.655373 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:40.643430 10992 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:40.656373 10439 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:40.660868 10901 raft_consensus.cc:2804] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.661947 10901 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.662444 10901 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:40.661463 10441 leader_election.cc:343] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.534030 10743 raft_consensus.cc:740] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.670028 10368 catalog_manager.cc:5697] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.671244 10441 leader_election.cc:304] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.680665 11016 raft_consensus.cc:2749] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:40.681138 10441 leader_election.cc:343] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.693542 11054 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.673298 10743 consensus_queue.cc:260] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:40.708988 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 91399.7us
I20260709 13:05:40.712600 10743 ts_tablet_manager.cc:1434] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.193s	user 0.010s	sys 0.001s
I20260709 13:05:40.715121 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:40.719724 10744 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:40.720405 10744 ts_tablet_manager.cc:1403] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.094s	user 0.015s	sys 0.010s
I20260709 13:05:40.713106 11054 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.722157 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.722743 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:40.743158 10696 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:40.743949 10619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.719312 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:40.746061 10624 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.726877 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:40.747020 10695 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.719822 10620 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:40.728324 10901 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.749883 10901 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:40.733863 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.750640 10622 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.753460 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:40.754246 10697 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.759492 10588 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
W20260709 13:05:40.761137 10588 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.761325 11067 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.761897 11067 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.762310 11067 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.746945 10743 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:40.736274 10745 tablet_bootstrap.cc:654] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.752055 10746 tablet_bootstrap.cc:654] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.725366 10744 raft_consensus.cc:359] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.768877 10744 raft_consensus.cc:385] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.769236 10744 raft_consensus.cc:740] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.772380 10744 consensus_queue.cc:260] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:40.790081 10744 ts_tablet_manager.cc:1434] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.003s
I20260709 13:05:40.791162 10744 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:40.814361 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:40.817764 10620 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.830039 10791 consensus_queue.cc:1048] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.837687 10695 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:40.839479 10439 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:40.851195 10791 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.851919 10791 raft_consensus.cc:697] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:40.852849 10791 consensus_queue.cc:237] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:40.854036 10745 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:40.854600 10745 ts_tablet_manager.cc:1403] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.206s	user 0.028s	sys 0.000s
I20260709 13:05:40.857724 10745 raft_consensus.cc:359] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.858440 10745 raft_consensus.cc:385] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.858722 10745 raft_consensus.cc:740] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.859542 10745 consensus_queue.cc:260] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.862139 10745 ts_tablet_manager.cc:1434] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:40.863665 10745 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:40.869793 10473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:40.873696 10620 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:40.875022 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:40.878765 10440 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:40.886606 10743 tablet_bootstrap.cc:654] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.889020 10622 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
W20260709 13:05:40.850831 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 128177us
I20260709 13:05:40.893350 10697 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
W20260709 13:05:40.894407 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 55085.6us
W20260709 13:05:40.904160 10590 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.904753 10590 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:40.905668 11068 raft_consensus.cc:2749] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.904654 11016 raft_consensus.cc:2804] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:40.914992 10664 consensus_peers.cc:597] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.915822 10744 tablet_bootstrap.cc:654] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.924569 10440 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:40.933473 10901 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.934517 10988 consensus_queue.cc:1048] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.932299 11016 raft_consensus.cc:697] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:40.982203 10901 raft_consensus.cc:697] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:40.989048 10901 consensus_queue.cc:237] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:40.991724 10745 tablet_bootstrap.cc:654] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:40.980844 10440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 47040us
I20260709 13:05:40.992381 11016 consensus_queue.cc:237] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:40.981144 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 48377.5us
I20260709 13:05:41.011498 11076 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.024703 11076 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.015568 11067 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.026124 11115 consensus_queue.cc:1048] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.027864 10743 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:41.028577 10743 ts_tablet_manager.cc:1403] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.283s	user 0.008s	sys 0.013s
I20260709 13:05:41.037266 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:41.055642 10746 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.056223 10746 ts_tablet_manager.cc:1403] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.471s	user 0.026s	sys 0.014s
I20260709 13:05:41.061056 11067 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:41.062590 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:41.064424 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:41.065270 10695 raft_consensus.cc:2393] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:41.072710 10588 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:41.075315 10744 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.075928 10744 ts_tablet_manager.cc:1403] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.285s	user 0.027s	sys 0.007s
I20260709 13:05:41.073907 10992 raft_consensus.cc:2749] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.066726 11076 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:41.081816 11076 consensus_queue.cc:1048] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.085825 10743 raft_consensus.cc:359] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.086907 10743 raft_consensus.cc:385] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.087262 10743 raft_consensus.cc:740] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.117779 10743 consensus_queue.cc:260] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.123112 10743 ts_tablet_manager.cc:1434] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 13:05:41.129092 10745 tablet_bootstrap.cc:492] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:41.129649 10745 ts_tablet_manager.cc:1403] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.266s	user 0.024s	sys 0.020s
I20260709 13:05:41.142678 10745 raft_consensus.cc:359] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.143472 10745 raft_consensus.cc:385] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.143951 10745 raft_consensus.cc:740] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.150242 11068 raft_consensus.cc:493] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.154703 11068 raft_consensus.cc:515] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.155107 10901 consensus_queue.cc:1048] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.155329 10746 raft_consensus.cc:359] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.156098 10746 raft_consensus.cc:385] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.156383 10746 raft_consensus.cc:740] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.156845 11068 leader_election.cc:290] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:41.157081 10746 consensus_queue.cc:260] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.160687 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:41.161275 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:41.163048 10746 ts_tablet_manager.cc:1434] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.000s
W20260709 13:05:41.163021 10590 leader_election.cc:343] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.164533 10746 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.169237 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:41.174310 10588 leader_election.cc:343] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.174757 10588 leader_election.cc:304] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.175941 11068 raft_consensus.cc:2749] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.181296 10744 raft_consensus.cc:359] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.181991 10744 raft_consensus.cc:385] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.182229 10744 raft_consensus.cc:740] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.182754 10744 consensus_queue.cc:260] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.185223 10515 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:41.277029 10743 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:41.295262 10744 ts_tablet_manager.cc:1434] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.219s	user 0.004s	sys 0.003s
I20260709 13:05:41.296227 10746 tablet_bootstrap.cc:654] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.296356 10744 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
W20260709 13:05:41.301191 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:41.306613 10744 tablet_bootstrap.cc:654] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.306895 10743 tablet_bootstrap.cc:654] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.316918 10994 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.317624 10994 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:41.318099 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 132996us
W20260709 13:05:41.322220 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 185917us
I20260709 13:05:41.314225 11132 raft_consensus.cc:2749] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.354626 11139 raft_consensus.cc:493] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.355459 11139 raft_consensus.cc:515] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.343242 10994 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:41.346184 10471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:41.388008 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 46713us
I20260709 13:05:41.372509 11115 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.422992 10746 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.412029 11115 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.430120 11115 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
W20260709 13:05:41.432412 11130 log.cc:927] Time spent T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.359s	user 0.006s	sys 0.000s
I20260709 13:05:41.434764 11139 leader_election.cc:290] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:41.435572 10745 consensus_queue.cc:260] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.437955 10745 ts_tablet_manager.cc:1434] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.308s	user 0.013s	sys 0.004s
I20260709 13:05:41.438931 10745 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:41.439616 11132 consensus_queue.cc:1048] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.423595 10746 ts_tablet_manager.cc:1403] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.259s	user 0.013s	sys 0.014s
I20260709 13:05:41.456250 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:41.457690 10547 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.460397 10619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:41.459976 10369 catalog_manager.cc:5697] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.461256 10619 raft_consensus.cc:2393] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:41.461791 10369 catalog_manager.cc:5697] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.462920 10515 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:41.465674 11135 log.cc:927] Time spent T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:05:41.466293 10994 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.466812 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:41.467479 10475 raft_consensus.cc:2468] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:41.472352 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:41.475204 10439 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.476395 11129 log.cc:927] Time spent T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.176s	user 0.003s	sys 0.000s
W20260709 13:05:41.476639 11136 log.cc:927] Time spent T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.194s	user 0.001s	sys 0.001s
W20260709 13:05:41.479375 11133 log.cc:927] Time spent T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:41.483734 10746 raft_consensus.cc:359] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.510686 10746 raft_consensus.cc:385] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.510941 10746 raft_consensus.cc:740] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.511613 10746 consensus_queue.cc:260] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.488803 11123 consensus_queue.cc:1048] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
W20260709 13:05:41.512738 10664 leader_election.cc:343] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.488937 11115 consensus_queue.cc:1048] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:05:41.502729 10744 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.517452 10664 leader_election.cc:304] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.517773 10744 ts_tablet_manager.cc:1403] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.222s	user 0.040s	sys 0.048s
I20260709 13:05:41.520810 10744 raft_consensus.cc:359] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.521502 10744 raft_consensus.cc:385] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.521780 10744 raft_consensus.cc:740] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.522629 11112 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.522434 10744 consensus_queue.cc:260] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.523120 11112 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.525229 11112 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:41.526533 10744 ts_tablet_manager.cc:1434] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:41.526669 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:41.527647 10744 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:41.530135 11137 raft_consensus.cc:2804] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.530504 11137 raft_consensus.cc:493] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.530823 11137 raft_consensus.cc:3060] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.508909 10743 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:41.535521 10743 ts_tablet_manager.cc:1403] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.262s	user 0.014s	sys 0.006s
I20260709 13:05:41.537957 10745 tablet_bootstrap.cc:654] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.541069 10590 leader_election.cc:343] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.542423 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:41.546727 11137 raft_consensus.cc:515] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.549788 10746 ts_tablet_manager.cc:1434] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.102s	user 0.006s	sys 0.000s
I20260709 13:05:41.586460 10746 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.556602 10744 tablet_bootstrap.cc:654] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.562247 11137 leader_election.cc:290] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:41.493150 10994 raft_consensus.cc:2749] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:41.568387 10590 leader_election.cc:343] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.570851 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:41.625038 10475 raft_consensus.cc:3060] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.579042 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:41.581086 11132 raft_consensus.cc:493] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.629009 10664 leader_election.cc:343] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.628800 11132 raft_consensus.cc:515] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.631121 11132 leader_election.cc:290] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:41.600633 10619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:41.631841 10619 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:41.615559 11139 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.632746 11139 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.633090 10475 raft_consensus.cc:2468] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:41.634213 10746 tablet_bootstrap.cc:654] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.634151 10590 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.634958 11139 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
W20260709 13:05:41.635166 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 66820.1us
I20260709 13:05:41.635954 11113 raft_consensus.cc:2749] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.636567 10664 leader_election.cc:304] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.638262 11147 raft_consensus.cc:2804] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.638904 11147 raft_consensus.cc:697] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:41.639845 11120 consensus_queue.cc:1048] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.640039 11147 consensus_queue.cc:237] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.648789 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:41.553566 10743 raft_consensus.cc:359] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.651324 10743 raft_consensus.cc:385] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.651342 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:41.655799 10743 raft_consensus.cc:740] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.654649 10619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:41.656625 10743 consensus_queue.cc:260] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.658826 10743 ts_tablet_manager.cc:1434] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.123s	user 0.004s	sys 0.000s
I20260709 13:05:41.685787 10743 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
W20260709 13:05:41.690270 10664 leader_election.cc:343] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.691309 10664 leader_election.cc:343] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.691862 10664 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.693463 11139 raft_consensus.cc:2749] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.694144 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:41.700440 10745 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:41.701210 10745 ts_tablet_manager.cc:1403] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.262s	user 0.024s	sys 0.020s
I20260709 13:05:41.704505 11123 consensus_queue.cc:1048] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.707854 10440 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:41.709120 11122 raft_consensus.cc:2804] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.709529 11122 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.709882 11122 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.710899 11115 consensus_queue.cc:1048] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.713207 11123 consensus_queue.cc:1048] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.720083 10514 leader_election.cc:304] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:41.719635 10369 catalog_manager.cc:5697] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.722658 11131 raft_consensus.cc:2749] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.724915 10745 raft_consensus.cc:359] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.727699 10745 raft_consensus.cc:385] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.728109 10745 raft_consensus.cc:740] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.729120 10745 consensus_queue.cc:260] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.734948 10745 ts_tablet_manager.cc:1434] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:05:41.736245 10746 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.736536 10745 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:41.736805 10746 ts_tablet_manager.cc:1403] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.151s	user 0.023s	sys 0.071s
I20260709 13:05:41.739622 10746 raft_consensus.cc:359] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.740317 10746 raft_consensus.cc:385] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.740573 10746 raft_consensus.cc:740] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.741238 10746 consensus_queue.cc:260] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.743192 10746 ts_tablet_manager.cc:1434] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:05:41.744249 10746 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.718920 11122 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.749908 10746 tablet_bootstrap.cc:654] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.750319 10744 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.750900 10744 ts_tablet_manager.cc:1403] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.223s	user 0.008s	sys 0.012s
I20260709 13:05:41.753643 10744 raft_consensus.cc:359] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.758571 10744 raft_consensus.cc:385] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.758991 10744 raft_consensus.cc:740] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.759797 10744 consensus_queue.cc:260] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.757059 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:41.760838 10543 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.758128 11122 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
W20260709 13:05:41.762919 10439 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.757814 10743 tablet_bootstrap.cc:654] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.775096 10745 tablet_bootstrap.cc:654] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.764825 10744 ts_tablet_manager.cc:1434] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:05:41.782855 10744 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:41.793156 10746 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.793857 10746 ts_tablet_manager.cc:1403] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.013s
I20260709 13:05:41.801743 10744 tablet_bootstrap.cc:654] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.808014 10746 raft_consensus.cc:359] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.808940 10746 raft_consensus.cc:385] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.809285 10746 raft_consensus.cc:740] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.810002 10746 consensus_queue.cc:260] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.810886 10745 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:41.811408 10745 ts_tablet_manager.cc:1403] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.001s
I20260709 13:05:41.812363 10746 ts_tablet_manager.cc:1434] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.004s
I20260709 13:05:41.813553 10746 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.814177 10745 raft_consensus.cc:359] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.814832 10745 raft_consensus.cc:385] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.815088 10745 raft_consensus.cc:740] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.815822 10745 consensus_queue.cc:260] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.818558 10745 ts_tablet_manager.cc:1434] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:41.819482 10745 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:41.819813 10746 tablet_bootstrap.cc:654] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.822779 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:41.823323 11120 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.823424 10621 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.823972 11120 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.826114 11120 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:41.826903 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:41.827582 10695 raft_consensus.cc:2468] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:41.829752 10439 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:41.830602 10745 tablet_bootstrap.cc:654] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.830670 11120 raft_consensus.cc:2804] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.831223 11120 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.831560 11120 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.832641 10621 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:41.841423 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:41.844944 10441 leader_election.cc:343] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.846556 10745 tablet_bootstrap.cc:492] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:41.847049 10745 ts_tablet_manager.cc:1403] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.002s
I20260709 13:05:41.850589 10745 raft_consensus.cc:359] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.851243 10745 raft_consensus.cc:385] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.851495 10745 raft_consensus.cc:740] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.853576 10744 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.854362 10746 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.854106 10745 consensus_queue.cc:260] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.855085 10746 ts_tablet_manager.cc:1403] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.004s
I20260709 13:05:41.855618 11120 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.856503 10744 ts_tablet_manager.cc:1403] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.008s
I20260709 13:05:41.857232 10745 ts_tablet_manager.cc:1434] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:41.857749 11120 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:41.858640 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:41.861011 10695 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.861327 10745 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:41.863022 10744 raft_consensus.cc:359] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.863658 10744 raft_consensus.cc:385] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.863929 10744 raft_consensus.cc:740] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.859063 10746 raft_consensus.cc:359] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.859428 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:41.864993 10744 consensus_queue.cc:260] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.865686 10746 raft_consensus.cc:385] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.866065 10746 raft_consensus.cc:740] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
W20260709 13:05:41.866171 10441 leader_election.cc:343] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.866853 10746 consensus_queue.cc:260] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.867378 10744 ts_tablet_manager.cc:1434] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:41.868448 10744 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:41.868374 10695 raft_consensus.cc:2468] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:41.869355 10746 ts_tablet_manager.cc:1434] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 13:05:41.869596 10440 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:41.870435 10746 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.869614 10439 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:41.871754 11123 raft_consensus.cc:2804] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.872238 11123 raft_consensus.cc:697] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:41.873087 11123 consensus_queue.cc:237] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.874740 10744 tablet_bootstrap.cc:654] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.876132 11122 raft_consensus.cc:2804] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.876664 11122 raft_consensus.cc:697] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:41.877465 10746 tablet_bootstrap.cc:654] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.878840 11122 consensus_queue.cc:237] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.873049 10745 tablet_bootstrap.cc:654] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.897890 10744 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.898463 10744 ts_tablet_manager.cc:1403] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.010s
I20260709 13:05:41.901458 10744 raft_consensus.cc:359] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.908305 10744 raft_consensus.cc:385] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.908607 10744 raft_consensus.cc:740] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.909390 10744 consensus_queue.cc:260] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:41.911875 10744 ts_tablet_manager.cc:1434] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:41.912403 10745 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:41.914295 10745 ts_tablet_manager.cc:1403] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.002s
I20260709 13:05:41.912810 10744 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:41.919796 10745 raft_consensus.cc:359] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.920528 10745 raft_consensus.cc:385] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.920794 10745 raft_consensus.cc:740] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.924109 10746 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.924696 10746 ts_tablet_manager.cc:1403] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.027s
I20260709 13:05:41.921566 10745 consensus_queue.cc:260] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.927038 10745 ts_tablet_manager.cc:1434] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:05:41.928078 10746 raft_consensus.cc:359] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.928438 10745 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:41.928882 10746 raft_consensus.cc:385] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.929241 10746 raft_consensus.cc:740] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.929945 10746 consensus_queue.cc:260] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.933182 10744 tablet_bootstrap.cc:654] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.933369 10746 ts_tablet_manager.cc:1434] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:41.936822 10745 tablet_bootstrap.cc:654] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.923404 10368 catalog_manager.cc:5697] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.938571 10746 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.944320 10743 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:41.944851 10743 ts_tablet_manager.cc:1403] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.285s	user 0.008s	sys 0.009s
I20260709 13:05:41.945257 10744 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:41.945408 10746 tablet_bootstrap.cc:654] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.945753 10744 ts_tablet_manager.cc:1403] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.000s
I20260709 13:05:41.950322 10744 raft_consensus.cc:359] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.954561 10744 raft_consensus.cc:385] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.954916 10744 raft_consensus.cc:740] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.955881 10744 consensus_queue.cc:260] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.958441 10743 raft_consensus.cc:359] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:41.959175 10743 raft_consensus.cc:385] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.959331 10744 ts_tablet_manager.cc:1434] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:41.960763 10746 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:41.964471 10746 ts_tablet_manager.cc:1403] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.002s
I20260709 13:05:41.966279 10744 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:41.967494 10746 raft_consensus.cc:359] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.959461 10743 raft_consensus.cc:740] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.968501 10746 raft_consensus.cc:385] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.970664 10746 raft_consensus.cc:740] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.971453 10746 consensus_queue.cc:260] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.973517 10746 ts_tablet_manager.cc:1434] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:05:41.974961 10746 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:41.980576 10744 tablet_bootstrap.cc:654] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.980710 10746 tablet_bootstrap.cc:654] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.994473 11123 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.995255 11123 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:41.996953 11147 raft_consensus.cc:493] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.997449 11131 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.997962 11131 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:41.998317 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:41.998953 10543 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:41.997491 11147 raft_consensus.cc:515] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.000506 10441 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:42.001670 10746 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:42.001803 11122 raft_consensus.cc:2804] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.002266 11122 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.002313 10746 ts_tablet_manager.cc:1403] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.007s
I20260709 13:05:42.002702 11122 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.002772 11147 leader_election.cc:290] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.004086 11131 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.006909 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.007591 10550 raft_consensus.cc:2468] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:42.004993 11147 raft_consensus.cc:493] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.008342 11147 raft_consensus.cc:515] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.009658 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:42.010458 10695 raft_consensus.cc:2393] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:42.011945 11123 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.014935 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.014997 11147 leader_election.cc:290] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:42.007529 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.016954 11122 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.017269 10664 leader_election.cc:304] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:42.018860 10472 raft_consensus.cc:2468] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:42.019254 10746 raft_consensus.cc:359] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.020082 10746 raft_consensus.cc:385] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.020334 10746 raft_consensus.cc:740] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.020968 10516 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:42.023990 10746 consensus_queue.cc:260] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.024266 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:42.025097 10550 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.025308 11122 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.028126 11163 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.029445 10746 ts_tablet_manager.cc:1434] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:05:42.029738 11147 raft_consensus.cc:2804] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.030098 11147 raft_consensus.cc:493] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.030450 10746 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:42.036018 10743 consensus_queue.cc:260] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.036841 10746 tablet_bootstrap.cc:654] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.030411 11147 raft_consensus.cc:3060] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.037428 11113 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.037916 11113 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.040036 11113 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:42.041345 10664 leader_election.cc:304] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.023095 10370 catalog_manager.cc:5697] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.042148 11139 raft_consensus.cc:2804] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.048837 11139 raft_consensus.cc:493] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.049285 11139 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.049746 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.050416 10622 raft_consensus.cc:2468] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:42.053675 11147 raft_consensus.cc:515] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.053187 10370 catalog_manager.cc:5697] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.043335 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.056529 10472 raft_consensus.cc:2393] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:42.058653 11147 leader_election.cc:290] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.044049 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.046406 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.061053 10590 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.062417 11112 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.066797 11163 raft_consensus.cc:2749] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.069754 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:42.100589 10543 raft_consensus.cc:3060] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.101158 11139 raft_consensus.cc:515] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.071143 11112 raft_consensus.cc:2749] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:42.070264 10664 leader_election.cc:343] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.078004 11120 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.108179 11120 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.110467 11120 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.084666 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.112284 10622 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:42.086813 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:42.088687 10619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acce5611c5e4ad4be2fedf0b6f296ea" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:42.113472 10619 raft_consensus.cc:3060] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.115275 11139 leader_election.cc:290] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:42.102871 11115 raft_consensus.cc:493] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.102995 10550 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:42.116164 11115 raft_consensus.cc:515] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.119022 11115 leader_election.cc:290] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:42.119890 10744 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:42.120465 10744 ts_tablet_manager.cc:1403] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.156s	user 0.014s	sys 0.002s
I20260709 13:05:42.120215 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.124508 10441 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:42.129462 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:42.130597 11141 raft_consensus.cc:493] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.136557 10441 leader_election.cc:343] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.136790 10698 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:42.136814 11141 raft_consensus.cc:515] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.137552 11115 raft_consensus.cc:2804] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.138319 11115 raft_consensus.cc:697] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:42.138403 11123 raft_consensus.cc:493] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.138958 11123 raft_consensus.cc:515] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.139216 11115 consensus_queue.cc:237] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.140412 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.140982 11123 leader_election.cc:290] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.141081 10472 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:42.141999 11163 raft_consensus.cc:493] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.142562 11163 raft_consensus.cc:515] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.143086 10590 leader_election.cc:304] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.144130 11141 leader_election.cc:290] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.148729 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:42.150210 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 79974.3us
I20260709 13:05:42.150887 10472 raft_consensus.cc:2393] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:42.143899 11112 raft_consensus.cc:2804] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.152088 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6d5efd76a54d0892c288f4f50190ae" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:42.152468 11112 raft_consensus.cc:493] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.146484 10439 leader_election.cc:304] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.153961 11112 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.155360 11124 raft_consensus.cc:2804] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.155856 11124 raft_consensus.cc:493] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.156184 11163 leader_election.cc:290] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.159117 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.127195 10619 raft_consensus.cc:2468] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:42.133123 10543 raft_consensus.cc:2468] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:42.126009 10744 raft_consensus.cc:359] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.156227 11124 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.161293 10744 raft_consensus.cc:385] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.073932 10743 ts_tablet_manager.cc:1434] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.003s
I20260709 13:05:42.161653 10744 raft_consensus.cc:740] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.162349 10744 consensus_queue.cc:260] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.168254 11148 raft_consensus.cc:493] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.168820 11148 raft_consensus.cc:515] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.171027 11148 leader_election.cc:290] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.173794 11123 raft_consensus.cc:493] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.176060 11123 raft_consensus.cc:515] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.178537 11123 leader_election.cc:290] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:42.180162 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:42.180898 10695 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:42.181977 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:42.182638 10472 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.183208 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:42.185856 10441 leader_election.cc:343] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.187028 10547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.188328 10514 leader_election.cc:304] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:42.189601 11163 raft_consensus.cc:3060] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.190591 10472 raft_consensus.cc:2468] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:42.191538 10439 leader_election.cc:304] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.175542 10744 ts_tablet_manager.cc:1434] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:05:42.193557 11122 raft_consensus.cc:2804] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.194043 11122 raft_consensus.cc:493] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.194382 11122 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.193553 10744 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:42.201761 11189 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.202044 10664 leader_election.cc:304] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:42.202373 11189 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:42.204353 10441 leader_election.cc:343] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:42.204804 10664 leader_election.cc:343] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.205047 11139 raft_consensus.cc:2804] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.205612 11139 raft_consensus.cc:697] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:42.205664 11189 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.162729 10743 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:42.206527 11139 consensus_queue.cc:237] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.208222 11163 raft_consensus.cc:2749] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.143548 10624 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:42.199373 11112 raft_consensus.cc:515] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.206547 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:42.212874 10475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:42.214968 11122 raft_consensus.cc:515] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.220674 10664 leader_election.cc:304] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.225589 11122 leader_election.cc:290] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:42.225603 11112 leader_election.cc:290] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.227380 10743 tablet_bootstrap.cc:654] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.230023 10744 tablet_bootstrap.cc:654] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.231976 10664 leader_election.cc:343] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.230742 11148 raft_consensus.cc:2804] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.233618 11148 raft_consensus.cc:697] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:42.236687 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.238219 10745 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:42.234493 11148 consensus_queue.cc:237] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.239140 10745 ts_tablet_manager.cc:1403] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.311s	user 0.038s	sys 0.086s
I20260709 13:05:42.242905 10745 raft_consensus.cc:359] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.243824 10745 raft_consensus.cc:385] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.244170 10745 raft_consensus.cc:740] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.262181 10745 consensus_queue.cc:260] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.269693 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:42.275682 10693 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.280885 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
W20260709 13:05:42.282605 10441 leader_election.cc:343] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.293229 10745 ts_tablet_manager.cc:1434] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:05:42.294643 10745 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:42.283092 11124 raft_consensus.cc:515] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.298952 10743 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:42.305317 10743 ts_tablet_manager.cc:1403] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.143s	user 0.008s	sys 0.003s
W20260709 13:05:42.306471 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 55380.1us
I20260709 13:05:42.305444 10368 catalog_manager.cc:5697] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.321281 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.322165 10624 raft_consensus.cc:2393] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:42.325734 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:42.326524 10696 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:42.327961 10475 raft_consensus.cc:2393] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:42.328145 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:42.328826 10697 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.320235 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.332216 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.332935 10623 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 0.
I20260709 13:05:42.334256 11113 raft_consensus.cc:493] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.334820 11113 raft_consensus.cc:515] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.331647 10621 raft_consensus.cc:2393] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:42.337030 11113 leader_election.cc:290] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:42.338930 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:42.340406 10590 leader_election.cc:343] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.338987 10693 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:42.341849 10622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.342511 10622 raft_consensus.cc:2468] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:42.343034 10439 leader_election.cc:304] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.344767 11115 raft_consensus.cc:2804] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.344933 10693 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.345332 11115 raft_consensus.cc:697] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:42.351593 10746 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: No bootstrap required, opened a new log
I20260709 13:05:42.352144 10746 ts_tablet_manager.cc:1403] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.322s	user 0.058s	sys 0.094s
I20260709 13:05:42.350620 10743 raft_consensus.cc:359] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.355669 10743 raft_consensus.cc:385] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.356530 10743 raft_consensus.cc:740] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.346243 11115 consensus_queue.cc:237] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.361464 10743 consensus_queue.cc:260] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:42.366340 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 100134us
I20260709 13:05:42.368302 10663 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:42.370369 10743 ts_tablet_manager.cc:1434] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.001s
I20260709 13:05:42.384650 10743 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: Bootstrap starting.
I20260709 13:05:42.387565 10663 leader_election.cc:304] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.367347 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.394588 10697 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
W20260709 13:05:42.436636 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:42.445343 10588 leader_election.cc:304] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:42.429905 11139 raft_consensus.cc:2749] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.460424 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:42.455233 10440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 87690.8us
I20260709 13:05:42.467882 10745 tablet_bootstrap.cc:654] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.513845 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 87865.5us
I20260709 13:05:42.455651 11196 raft_consensus.cc:2804] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.464711 11113 raft_consensus.cc:2804] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.542032 10744 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:42.542814 10744 ts_tablet_manager.cc:1403] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.349s	user 0.017s	sys 0.009s
I20260709 13:05:42.547564 11201 consensus_queue.cc:1048] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.526280 11196 raft_consensus.cc:493] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.525209 11124 leader_election.cc:290] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:42.544209 11113 raft_consensus.cc:697] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:42.559231 11196 raft_consensus.cc:3060] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.586571 10746 raft_consensus.cc:359] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.587363 10746 raft_consensus.cc:385] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.588229 10746 raft_consensus.cc:740] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.592809 10746 consensus_queue.cc:260] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:42.607038 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 83912.7us
I20260709 13:05:42.618521 10746 ts_tablet_manager.cc:1434] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.266s	user 0.009s	sys 0.001s
I20260709 13:05:42.631739 11205 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.642804 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 197588us
I20260709 13:05:42.644591 10744 raft_consensus.cc:359] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.645350 10744 raft_consensus.cc:385] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.645586 10744 raft_consensus.cc:740] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.646095 10744 consensus_queue.cc:260] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.653759 10743 tablet_bootstrap.cc:654] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:42.653781 10588 leader_election.cc:343] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.654402 10588 leader_election.cc:304] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.661469 10744 ts_tablet_manager.cc:1434] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.118s	user 0.006s	sys 0.001s
I20260709 13:05:42.632356 11205 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.621104 11113 consensus_queue.cc:237] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.665091 10744 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: Bootstrap starting.
I20260709 13:05:42.678548 11196 raft_consensus.cc:515] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.665512 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:42.682237 10697 raft_consensus.cc:2393] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:42.666714 11112 raft_consensus.cc:493] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.685189 11112 raft_consensus.cc:515] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.689085 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.690032 10472 raft_consensus.cc:2468] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:42.696161 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:42.760039 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 63872.4us
I20260709 13:05:42.687657 11112 leader_election.cc:290] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:42.770381 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:42.771574 10472 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
I20260709 13:05:42.773424 10516 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:42.776325 10745 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:42.776814 10745 ts_tablet_manager.cc:1403] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.482s	user 0.012s	sys 0.016s
I20260709 13:05:42.760906 11210 raft_consensus.cc:2749] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.777444 11209 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.788648 11209 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.789224 11209 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.768656 11205 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:42.777530 10440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 77690.1us
I20260709 13:05:42.794130 10440 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.809508 10590 leader_election.cc:304] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.814715 11208 raft_consensus.cc:2804] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.815841 11208 raft_consensus.cc:493] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.816208 11208 raft_consensus.cc:3060] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.818385 11204 consensus_queue.cc:1048] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.819675 11211 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.820163 11211 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.824227 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:42.824921 10620 raft_consensus.cc:3060] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.822580 11196 leader_election.cc:290] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:42.831389 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:42.821372 11211 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.832230 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:42.836087 10745 raft_consensus.cc:359] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:42.836908 10745 raft_consensus.cc:385] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.837201 10745 raft_consensus.cc:740] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.837992 10745 consensus_queue.cc:260] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:42.838917 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 323999us
I20260709 13:05:42.841951 11139 consensus_queue.cc:1048] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.846239 10620 raft_consensus.cc:2468] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:42.846316 11208 raft_consensus.cc:515] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.832266 10617 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 0.
W20260709 13:05:42.866117 10588 leader_election.cc:343] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.872134 10663 leader_election.cc:304] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:42.876672 10745 ts_tablet_manager.cc:1434] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.100s	user 0.005s	sys 0.003s
I20260709 13:05:42.877434 10744 tablet_bootstrap.cc:654] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.860431 11208 leader_election.cc:290] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
W20260709 13:05:42.876036 11219 log.cc:927] Time spent T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:05:42.879956 11139 consensus_queue.cc:1048] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:05:42.879925 11137 raft_consensus.cc:2804] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.889698 11137 raft_consensus.cc:697] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:42.890651 11137 consensus_queue.cc:237] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:42.886999 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:42.893493 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:42.895082 10472 raft_consensus.cc:3060] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.885646 10550 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.900622 10743 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: No bootstrap required, opened a new log
I20260709 13:05:42.901180 10743 ts_tablet_manager.cc:1403] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: Time spent bootstrapping tablet: real 0.517s	user 0.016s	sys 0.004s
W20260709 13:05:42.894245 10440 leader_election.cc:343] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:42.882249 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 87730.4us
I20260709 13:05:42.895520 11211 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.918490 10745 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:42.923378 10472 raft_consensus.cc:2468] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:42.925009 11221 consensus_queue.cc:1048] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.908259 10440 leader_election.cc:304] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:42.926838 10590 leader_election.cc:304] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:42.939338 11113 raft_consensus.cc:2804] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.942076 11113 raft_consensus.cc:697] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:42.943040 11113 consensus_queue.cc:237] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:42.950620 10440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 64725.5us
I20260709 13:05:42.951992 10440 leader_election.cc:304] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:42.953486 11218 raft_consensus.cc:2749] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.953334 10373 catalog_manager.cc:5697] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.954200 11218 consensus_queue.cc:1048] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.948647 10743 raft_consensus.cc:359] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.956631 10743 raft_consensus.cc:385] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.956925 10743 raft_consensus.cc:740] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.957614 10743 consensus_queue.cc:260] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:42.959721 10743 ts_tablet_manager.cc:1434] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.000s
I20260709 13:05:42.959782 10368 catalog_manager.cc:5697] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.961516 10368 catalog_manager.cc:5697] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.944991 10744 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: No bootstrap required, opened a new log
I20260709 13:05:42.962888 10744 ts_tablet_manager.cc:1403] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: Time spent bootstrapping tablet: real 0.298s	user 0.020s	sys 0.011s
I20260709 13:05:42.965751 10744 raft_consensus.cc:359] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.966408 10744 raft_consensus.cc:385] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.966686 10744 raft_consensus.cc:740] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Initialized, Role: FOLLOWER
W20260709 13:05:42.977042 10588 leader_election.cc:343] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.985760 10744 consensus_queue.cc:260] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:42.991935 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a0a4ae847a4b94b968e58c01a524c2" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:42.996240 10744 ts_tablet_manager.cc:1434] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
I20260709 13:05:42.998013 11139 consensus_queue.cc:1048] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:43.003315 10664 leader_election.cc:343] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.010147 11218 consensus_queue.cc:1048] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:42.975805 11122 raft_consensus.cc:2749] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.022153 11201 consensus_queue.cc:1048] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:43.031229 10745 tablet_bootstrap.cc:654] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.037097 11211 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:43.057502 10441 consensus_peers.cc:597] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:43.102327 10663 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 48978.4us
I20260709 13:05:43.116024 11147 raft_consensus.cc:493] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.118821 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:43.103099 11199 consensus_queue.cc:1048] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.072s
W20260709 13:05:43.143949 11227 log.cc:927] Time spent T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:43.130439 11209 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:43.159305 11207 raft_consensus.cc:493] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.159940 11207 raft_consensus.cc:515] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:43.162734 10544 raft_consensus.cc:2393] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a4b3558672241d2969ebf2c86f6d84e in current term 1: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:43.135084 11147 raft_consensus.cc:515] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:43.293429 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 57691.2us
W20260709 13:05:43.315418 11240 log.cc:927] Time spent T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:05:43.317546 11147 leader_election.cc:290] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:43.327888 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:43.344983 10621 raft_consensus.cc:2468] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 0.
I20260709 13:05:43.345973 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:43.346593 10624 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.347268 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
W20260709 13:05:43.356832 11239 log.cc:927] Time spent T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 13:05:43.303578 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 71181us
I20260709 13:05:43.360271 10624 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 1.
I20260709 13:05:43.371663 11207 leader_election.cc:290] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:43.380199 11209 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:43.385864 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
W20260709 13:05:43.409587 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 192203us
I20260709 13:05:43.423214 10474 raft_consensus.cc:2393] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:43.408592 11197 consensus_queue.cc:1048] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:05:43.453723 10745 tablet_bootstrap.cc:492] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:43.454310 10745 ts_tablet_manager.cc:1403] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.536s	user 0.005s	sys 0.032s
I20260709 13:05:43.467193 10663 leader_election.cc:304] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:43.468626 11206 raft_consensus.cc:2804] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.469041 11206 raft_consensus.cc:493] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.469362 11206 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.479946 10440 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:43.481648 11221 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.482210 11221 raft_consensus.cc:697] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:43.483124 11221 consensus_queue.cc:237] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:43.496946 10369 catalog_manager.cc:5697] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.496621 11242 raft_consensus.cc:493] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.520195 10745 raft_consensus.cc:359] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:43.529210 11242 raft_consensus.cc:515] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:43.525457 10745 raft_consensus.cc:385] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.537436 10745 raft_consensus.cc:740] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
W20260709 13:05:43.541999 10664 leader_election.cc:343] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.538326 10745 consensus_queue.cc:260] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
W20260709 13:05:43.546593 10664 consensus_peers.cc:597] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.536198 10694 raft_consensus.cc:3060] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.556164 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:43.557507 10624 raft_consensus.cc:2393] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 6a4b3558672241d2969ebf2c86f6d84e in this term.
I20260709 13:05:43.568377 11242 leader_election.cc:290] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:43.581084 10992 raft_consensus.cc:493] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.581775 10992 raft_consensus.cc:515] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:43.583572 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:43.585860 11230 log.cc:927] Time spent T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.437s	user 0.010s	sys 0.004s
I20260709 13:05:43.603514 11194 consensus_queue.cc:1048] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.604743 10616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:43.605902 10992 leader_election.cc:290] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:43.607667 11206 raft_consensus.cc:515] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:43.614399 10745 ts_tablet_manager.cc:1434] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.160s	user 0.007s	sys 0.002s
I20260709 13:05:43.616667 10745 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:43.619607 10516 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:43.665244 11252 raft_consensus.cc:2749] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.675796 11115 consensus_queue.cc:1048] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.676563 10469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:43.677627 10469 raft_consensus.cc:2468] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 0.
I20260709 13:05:43.678553 10471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225f2db0ded7415b81e387822b6e3856" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:43.679644 10471 raft_consensus.cc:2393] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:43.680497 10468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:43.662525 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:43.727530 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:43.728016 11212 consensus_queue.cc:1048] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:43.736773 10664 leader_election.cc:343] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:43.745680 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:43.766007 10745 tablet_bootstrap.cc:654] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.705816 11211 consensus_queue.cc:1048] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 13:05:43.762234 11206 leader_election.cc:290] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:43.785961 10664 leader_election.cc:304] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:43.787644 10516 leader_election.cc:304] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:43.792475 11234 raft_consensus.cc:2749] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.798357 11163 raft_consensus.cc:2749] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:43.841195 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 208070us
W20260709 13:05:43.844065 10590 leader_election.cc:343] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:43.847731 10441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 153105us
I20260709 13:05:43.866780 10548 raft_consensus.cc:3060] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.870678 11265 consensus_queue.cc:1048] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.871109 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:43.871848 10617 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.844817 10439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 157900us
I20260709 13:05:43.900740 10590 leader_election.cc:304] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:43.930665 10745 tablet_bootstrap.cc:492] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:43.931265 10745 ts_tablet_manager.cc:1403] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.315s	user 0.017s	sys 0.007s
I20260709 13:05:43.934114 10745 raft_consensus.cc:359] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:43.934873 10745 raft_consensus.cc:385] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.935173 10745 raft_consensus.cc:740] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.934562 11274 consensus_queue.cc:1048] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:43.957965 10617 raft_consensus.cc:2468] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:43.930078 11265 raft_consensus.cc:2804] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.959272 11265 raft_consensus.cc:493] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.959679 11265 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.936489 10745 consensus_queue.cc:260] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:43.966924 10663 leader_election.cc:304] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:43.982321 10745 ts_tablet_manager.cc:1434] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.001s
I20260709 13:05:43.983952 10745 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:43.975594 11147 raft_consensus.cc:2804] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.005687 11147 raft_consensus.cc:697] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:44.006634 11147 consensus_queue.cc:237] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:43.971511 11194 consensus_queue.cc:1048] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:05:44.043221 10368 catalog_manager.cc:5697] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } }
I20260709 13:05:44.060436 10745 tablet_bootstrap.cc:654] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:44.075165 11279 log.cc:927] Time spent T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:44.089977 11252 raft_consensus.cc:493] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.089815 11265 raft_consensus.cc:515] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:44.117529 11252 raft_consensus.cc:515] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:44.159474 11252 leader_election.cc:290] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:44.168995 11265 leader_election.cc:290] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:44.178567 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:44.186529 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:44.187459 10617 raft_consensus.cc:2393] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:44.192356 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:44.202808 10745 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:44.203495 10745 ts_tablet_manager.cc:1403] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.220s	user 0.010s	sys 0.007s
I20260709 13:05:44.205874 10745 raft_consensus.cc:359] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:44.206590 10745 raft_consensus.cc:385] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.206837 10745 raft_consensus.cc:740] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.268035 10474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:44.270104 10474 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.276604 10515 leader_election.cc:304] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96
W20260709 13:05:44.285660 10590 leader_election.cc:343] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.207470 10745 consensus_queue.cc:260] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:44.331990 11197 consensus_queue.cc:1048] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.342228 11124 consensus_queue.cc:1048] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.343214 11252 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.364449 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 66692.1us
I20260709 13:05:44.365458 10369 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38488 (request call id 70) took 1444 ms. Trace:
W20260709 13:05:44.373616 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 58936.5us
I20260709 13:05:44.394047 11252 raft_consensus.cc:2749] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.365957 10369 rpcz_store.cc:276] 0709 13:05:42.928532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.928657 (+   125us) service_pool.cc:224] Handling call
0709 13:05:44.365198 (+1436541us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:43.851891 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:43.852377 (+   486us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:43.852406 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:05:43.852426 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:43.852779 (+   353us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:43.852856 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:43.852891 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:43.853077 (+   186us) tablet.cc:701] Row locks acquired
0709 13:05:43.853108 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:05:43.853264 (+   156us) write_op.cc:270] Start()
0709 13:05:43.853373 (+   109us) write_op.cc:276] Timestamp: P: 1783602343853216 usec, L: 0
0709 13:05:43.853399 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:05:43.853991 (+   592us) log.cc:844] Serialized 540 byte log entry
0709 13:05:43.898007 (+ 44016us) op_driver.cc:464] REPLICATION: finished
0709 13:05:44.107225 (+209218us) write_op.cc:301] APPLY: starting
0709 13:05:44.108830 (+  1605us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:44.122710 (+ 13880us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:44.142997 (+ 20287us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:44.147475 (+  4478us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:44.161770 (+ 14295us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:44.161824 (+    54us) write_op.cc:312] APPLY: finished
0709 13:05:44.162037 (+   213us) log.cc:844] Serialized 22 byte log entry
0709 13:05:44.162218 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.162296 (+    78us) write_op.cc:454] Released schema lock
0709 13:05:44.162335 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:44.162702 (+   367us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.162732 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":461654,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":191821,"apply.run_cpu_time_us":2267,"apply.run_wall_time_us":61970,"num_ops":1,"prepare.queue_time_us":116782,"prepare.run_cpu_time_us":2437,"prepare.run_wall_time_us":2438,"replication_time_us":44457,"thread_start_us":221241,"threads_started":2}]]}
I20260709 13:05:44.531466 11246 raft_consensus.cc:493] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.532148 11246 raft_consensus.cc:515] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:44.548743 10474 raft_consensus.cc:2468] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:44.535427 11246 leader_election.cc:290] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:44.586494 10590 leader_election.cc:304] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:44.589075 11217 raft_consensus.cc:2804] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.589768 11217 raft_consensus.cc:697] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:44.577499 10370 catalog_manager.cc:5697] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:44.593138 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 316932us
I20260709 13:05:44.591115 11217 consensus_queue.cc:237] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:44.620776 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 76132.3us
W20260709 13:05:44.635694 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 48790.7us
I20260709 13:05:44.647567 10471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:44.656797 11212 consensus_queue.cc:1048] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:44.712468 11295 log.cc:927] Time spent T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:44.732247 10745 ts_tablet_manager.cc:1434] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.528s	user 0.038s	sys 0.038s
W20260709 13:05:44.732740 10745 ts_tablet_manager.cc:1467] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Tablet startup took 528ms
W20260709 13:05:44.733119 10745 ts_tablet_manager.cc:1469] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: Trace:
0709 13:05:35.839055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.839219 (+   164us) service_pool.cc:224] Handling call
0709 13:05:35.872375 (+ 33156us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:35.872508 (+   133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:35.911031 (+ 38523us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:35.918019 (+  6988us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:35.918669 (+   650us) inbound_call.cc:177] Queueing success response
0709 13:05:43.983448 (+8064779us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:44.203804 (+220356us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:44.204379 (+   575us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:44.204785 (+   406us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":38789,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":291,"tablet-open.queue_time_us":8065042,"thread_start_us":294792,"threads_started":1}
I20260709 13:05:44.761929 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:44.771934 10373 catalog_manager.cc:5697] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } }
I20260709 13:05:44.775100 10548 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.780831 10745 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: Bootstrap starting.
I20260709 13:05:44.795993 10514 leader_election.cc:304] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:44.812619 11234 consensus_queue.cc:1048] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:44.815145 10745 tablet_bootstrap.cc:654] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.879177 10745 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: No bootstrap required, opened a new log
I20260709 13:05:44.886947 10745 ts_tablet_manager.cc:1403] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent bootstrapping tablet: real 0.099s	user 0.010s	sys 0.005s
I20260709 13:05:44.891423 10745 raft_consensus.cc:359] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:44.892479 10745 raft_consensus.cc:385] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.905609 10745 raft_consensus.cc:740] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.912963 11163 raft_consensus.cc:2749] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.916101 10745 consensus_queue.cc:260] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:44.918754 10745 ts_tablet_manager.cc:1434] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:05:44.928097 11163 raft_consensus.cc:493] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.928578 11163 raft_consensus.cc:515] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:44.932123 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:44.934051 10471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:44.934875 10471 raft_consensus.cc:2393] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 1: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:44.938853 11163 leader_election.cc:290] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:44.941244 10369 catalog_manager.cc:5697] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.944646 10516 leader_election.cc:304] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:44.946756 11163 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.982272 10420 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:44.999956 11163 raft_consensus.cc:2749] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.026628 10370 catalog_manager.cc:5697] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.140708 11274 consensus_queue.cc:1048] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:45.150141   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:45.153678 10590 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:45.172878 10590 consensus_peers.cc:597] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.173640 10590 consensus_peers.cc:597] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.172894 11217 consensus_queue.cc:1048] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:05:45.194226 10590 consensus_peers.cc:597] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.227186   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:45.228387   431 tablet_replica.cc:333] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.229336   431 raft_consensus.cc:2243] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.229849   431 raft_consensus.cc:2272] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.233180   431 tablet_replica.cc:333] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.234139   431 raft_consensus.cc:2243] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.235157   431 raft_consensus.cc:2272] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.238400   431 tablet_replica.cc:333] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.239639   431 raft_consensus.cc:2243] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:45.239928 10664 consensus_peers.cc:597] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.240350   431 raft_consensus.cc:2272] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.244257   431 tablet_replica.cc:333] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.245419   431 raft_consensus.cc:2243] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:05:45.245782 10664 consensus_peers.cc:597] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.246646   431 raft_consensus.cc:2272] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.249680 10664 consensus_peers.cc:597] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.249711   431 tablet_replica.cc:333] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.250540   431 raft_consensus.cc:2243] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.251390   431 raft_consensus.cc:2272] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.253710   431 tablet_replica.cc:333] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.254369   431 raft_consensus.cc:2243] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.254835   431 raft_consensus.cc:2272] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.257144   431 tablet_replica.cc:333] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.257906   431 raft_consensus.cc:2243] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.258742   431 raft_consensus.cc:2272] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.261240   431 tablet_replica.cc:333] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.261807   431 raft_consensus.cc:2243] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.262341   431 raft_consensus.cc:2272] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.265440   431 tablet_replica.cc:333] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.266037   431 raft_consensus.cc:2243] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.266847   431 raft_consensus.cc:2272] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.269024   431 tablet_replica.cc:333] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.269686   431 raft_consensus.cc:2243] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.270493   431 raft_consensus.cc:2272] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.272717   431 tablet_replica.cc:333] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.273478   431 raft_consensus.cc:2243] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.274549   431 raft_consensus.cc:2272] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.277833   431 tablet_replica.cc:333] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
W20260709 13:05:45.277794 10590 consensus_peers.cc:597] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.278748   431 raft_consensus.cc:2243] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.279210   431 raft_consensus.cc:2272] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.281492   431 tablet_replica.cc:333] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.282332   431 raft_consensus.cc:2243] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.283285   431 raft_consensus.cc:2272] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.285744 10590 consensus_peers.cc:597] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.286098   431 tablet_replica.cc:333] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.286934   431 raft_consensus.cc:2243] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.287400   431 raft_consensus.cc:2272] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.289948   431 tablet_replica.cc:333] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.290588   431 raft_consensus.cc:2243] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.291112   431 raft_consensus.cc:2272] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.293455   431 tablet_replica.cc:333] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.294031   431 raft_consensus.cc:2243] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.294831   431 raft_consensus.cc:2272] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.297032   431 tablet_replica.cc:333] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.297619   431 raft_consensus.cc:2243] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.298399   431 raft_consensus.cc:2272] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.301075   431 tablet_replica.cc:333] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.301828   431 raft_consensus.cc:2243] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.303098   431 raft_consensus.cc:2272] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.303583 10664 consensus_peers.cc:597] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.305691   431 tablet_replica.cc:333] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.306638   431 raft_consensus.cc:2243] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.307698   431 raft_consensus.cc:2272] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.308254 10664 consensus_peers.cc:597] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.309569 10664 consensus_peers.cc:597] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.310422   431 tablet_replica.cc:333] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.311101   431 raft_consensus.cc:2243] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.311523   431 raft_consensus.cc:2272] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.313666   431 tablet_replica.cc:333] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.314208   431 raft_consensus.cc:2243] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.314622   431 raft_consensus.cc:2272] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.316634   431 tablet_replica.cc:333] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.318671   431 raft_consensus.cc:2243] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.319253   431 raft_consensus.cc:2272] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.322331   431 tablet_replica.cc:333] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.323228   431 raft_consensus.cc:2243] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.324143   431 raft_consensus.cc:2272] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.326205   431 tablet_replica.cc:333] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.326864   431 raft_consensus.cc:2243] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.327803   431 raft_consensus.cc:2272] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.331795   431 tablet_replica.cc:333] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
W20260709 13:05:45.332159 10664 consensus_peers.cc:597] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.332372   431 raft_consensus.cc:2243] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.333364   431 raft_consensus.cc:2272] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.336038   431 tablet_replica.cc:333] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.336874   431 raft_consensus.cc:2243] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.337457   431 raft_consensus.cc:2272] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.339594   431 tablet_replica.cc:333] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.340173   431 raft_consensus.cc:2243] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.340603   431 raft_consensus.cc:2272] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.342957   431 tablet_replica.cc:333] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.343658   431 raft_consensus.cc:2243] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.344214   431 raft_consensus.cc:2272] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.347023   431 tablet_replica.cc:333] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.347922   431 raft_consensus.cc:2243] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.349054   431 raft_consensus.cc:2272] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.352116   431 tablet_replica.cc:333] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.352878   431 raft_consensus.cc:2243] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.353377   431 raft_consensus.cc:2272] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.356166   431 tablet_replica.cc:333] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.356900   431 raft_consensus.cc:2243] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.357926   431 raft_consensus.cc:2272] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.361073   431 tablet_replica.cc:333] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.361855   431 raft_consensus.cc:2243] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.362404   431 raft_consensus.cc:2272] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.365383   431 tablet_replica.cc:333] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
W20260709 13:05:45.365823 10664 consensus_peers.cc:597] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.366134   431 raft_consensus.cc:2243] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.366588   431 pending_rounds.cc:70] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Trying to abort 1 pending ops.
I20260709 13:05:45.366743   431 pending_rounds.cc:77] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635203673653248 op_type: NO_OP noop_request { }
I20260709 13:05:45.366976   431 raft_consensus.cc:2889] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:45.367198   431 raft_consensus.cc:2272] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.369719   431 tablet_replica.cc:333] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.370484   431 raft_consensus.cc:2243] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.370898   431 raft_consensus.cc:2272] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.372977   431 tablet_replica.cc:333] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.373440   431 raft_consensus.cc:2243] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.374262   431 raft_consensus.cc:2272] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.376451   431 tablet_replica.cc:333] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.376961   431 raft_consensus.cc:2243] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.377744   431 raft_consensus.cc:2272] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.380307   431 tablet_replica.cc:333] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.380926   431 raft_consensus.cc:2243] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.381361   431 raft_consensus.cc:2272] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.383275   431 tablet_replica.cc:333] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.383864   431 raft_consensus.cc:2243] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.384624   431 raft_consensus.cc:2272] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.386610   431 tablet_replica.cc:333] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.387156   431 raft_consensus.cc:2243] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.387671   431 raft_consensus.cc:2272] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.389662   431 tablet_replica.cc:333] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.390183   431 raft_consensus.cc:2243] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.390600   431 raft_consensus.cc:2272] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.392670   431 tablet_replica.cc:333] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.393275   431 raft_consensus.cc:2243] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.393702   431 raft_consensus.cc:2272] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.396209   431 tablet_replica.cc:333] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.396751   431 raft_consensus.cc:2243] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.397567   431 raft_consensus.cc:2272] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.400311   431 tablet_replica.cc:333] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.400859   431 raft_consensus.cc:2243] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.401693   431 raft_consensus.cc:2272] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.403891   431 tablet_replica.cc:333] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.404368   431 raft_consensus.cc:2243] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.405189   431 raft_consensus.cc:2272] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.407689   431 tablet_replica.cc:333] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: stopping tablet replica
I20260709 13:05:45.408267   431 raft_consensus.cc:2243] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.408797   431 raft_consensus.cc:2272] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.446944 10516 consensus_peers.cc:597] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.486299   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:05:45.492133 10516 consensus_peers.cc:597] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.496716 10590 consensus_peers.cc:597] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.509800 10664 consensus_peers.cc:597] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.574678   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:45.582504 11320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:45.582654 11319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:45.586115 11322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:45.590081   431 server_base.cc:1061] running on GCE node
I20260709 13:05:45.601373   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:45.601692   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:45.601918   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602345601900 us; error 0 us; skew 500 ppm
I20260709 13:05:45.602648   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:05:45.604233 10590 consensus_peers.cc:597] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.607095   431 webserver.cc:533] Webserver started at http://127.0.107.193:44317/ using document root <none> and password file <none>
I20260709 13:05:45.607967   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:45.608232   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:45.615325   431 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:05:45.623819 11327 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:45.624778   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:05:45.625175   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "6a4b3558672241d2969ebf2c86f6d84e"
format_stamp: "Formatted at 2026-07-09 13:05:32 on dist-test-slave-kgkf"
I20260709 13:05:45.625597   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:45.636729   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:45.638273   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:45.639643   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:45.644371 11334 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:05:45.654459 10590 consensus_peers.cc:597] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.662237 10590 consensus_peers.cc:597] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.668151 10664 consensus_peers.cc:597] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.747556 10664 consensus_peers.cc:597] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.896539   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:05:45.896937   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.254s	user 0.008s	sys 0.000s
I20260709 13:05:45.897236   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:05:45.901867 11334 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:45.915458 11334 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.916455 11334 tablet_bootstrap.cc:492] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:45.917070 11334 ts_tablet_manager.cc:1403] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:05:45.919674 11334 raft_consensus.cc:359] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:45.920439 11334 raft_consensus.cc:740] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.921615 11334 consensus_queue.cc:260] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:45.924063 11334 ts_tablet_manager.cc:1434] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:45.924878 11334 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:45.938357 11334 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.939241 11334 tablet_bootstrap.cc:492] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:45.939868 11334 ts_tablet_manager.cc:1403] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 13:05:45.942250 11334 raft_consensus.cc:359] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:45.942889 11334 raft_consensus.cc:740] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.943562 11334 consensus_queue.cc:260] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:45.945395 11334 ts_tablet_manager.cc:1434] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:45.946125 11334 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:45.959992 11334 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.960899 11334 tablet_bootstrap.cc:492] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:45.961516 11334 ts_tablet_manager.cc:1403] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:05:45.963876 11334 raft_consensus.cc:359] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:45.964524 11334 raft_consensus.cc:740] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.965132 11334 consensus_queue.cc:260] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:45.966866 11334 ts_tablet_manager.cc:1434] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:45.967576 11334 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:45.982856 11334 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.983637 11334 tablet_bootstrap.cc:492] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:45.984190 11334 ts_tablet_manager.cc:1403] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:05:45.986259 11334 raft_consensus.cc:359] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:45.986792 11334 raft_consensus.cc:740] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.987375 11334 consensus_queue.cc:260] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:45.989051 11334 ts_tablet_manager.cc:1434] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:45.989683 11334 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.002359 11334 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.003633 11334 tablet_bootstrap.cc:492] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.004364 11334 ts_tablet_manager.cc:1403] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:05:46.008158 11334 raft_consensus.cc:359] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.008901 11334 raft_consensus.cc:740] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.009578 11334 consensus_queue.cc:260] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.011490 11334 ts_tablet_manager.cc:1434] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:05:46.012262 11334 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.027164 11334 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.028221 11334 tablet_bootstrap.cc:492] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.028837 11334 ts_tablet_manager.cc:1403] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 13:05:46.031200 11334 raft_consensus.cc:359] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.031958 11334 raft_consensus.cc:740] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.032622 11334 consensus_queue.cc:260] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.034479 11334 ts_tablet_manager.cc:1434] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 13:05:46.035339 11334 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.066427 11334 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.067365 11334 tablet_bootstrap.cc:492] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.068005 11334 ts_tablet_manager.cc:1403] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.015s
I20260709 13:05:46.070290 11334 raft_consensus.cc:359] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.070962 11334 raft_consensus.cc:740] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.072228 11334 consensus_queue.cc:260] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.074230 11334 ts_tablet_manager.cc:1434] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:05:46.074965 11334 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.092128 11334 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.093089 11334 tablet_bootstrap.cc:492] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.093696 11334 ts_tablet_manager.cc:1403] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 13:05:46.096174 11334 raft_consensus.cc:359] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.096861 11334 raft_consensus.cc:740] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.097544 11334 consensus_queue.cc:260] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.099135 11334 ts_tablet_manager.cc:1434] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:46.099828 11334 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.114828 11334 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.115641 11334 tablet_bootstrap.cc:492] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.116292 11334 ts_tablet_manager.cc:1403] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:05:46.118712 11334 raft_consensus.cc:359] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.119313 11334 raft_consensus.cc:740] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.120103 11334 consensus_queue.cc:260] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.121960 11334 ts_tablet_manager.cc:1434] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:46.122807 11334 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.162452 11334 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.163403 11334 tablet_bootstrap.cc:492] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.164072 11334 ts_tablet_manager.cc:1403] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.041s	user 0.027s	sys 0.012s
I20260709 13:05:46.166615 11334 raft_consensus.cc:359] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.167311 11334 raft_consensus.cc:740] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.168099 11334 consensus_queue.cc:260] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.169847 11334 ts_tablet_manager.cc:1434] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:46.170508 11334 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.177811   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:05:46.178194   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.281s	user 0.218s	sys 0.009s
I20260709 13:05:46.181381 11334 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.182134 11334 tablet_bootstrap.cc:492] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.182659 11334 ts_tablet_manager.cc:1403] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:05:46.185034 11334 raft_consensus.cc:359] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.185693 11334 raft_consensus.cc:740] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.186439 11334 consensus_queue.cc:260] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.186719 11253 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.188416 11334 ts_tablet_manager.cc:1434] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:05:46.189375 11334 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.187517 11253 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.192181 11253 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.194233 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
W20260709 13:05:46.204689 10516 leader_election.cc:336] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111)
I20260709 13:05:46.205327 10516 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.206883 11163 raft_consensus.cc:2749] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.222363 11334 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.223438 11334 tablet_bootstrap.cc:492] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.224154 11334 ts_tablet_manager.cc:1403] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.035s	user 0.026s	sys 0.008s
I20260709 13:05:46.226800 11334 raft_consensus.cc:359] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.227813 11334 raft_consensus.cc:740] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.228649 11334 consensus_queue.cc:260] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.230832 11334 ts_tablet_manager.cc:1434] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:46.231662 11334 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.250070 11334 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.251250 11334 tablet_bootstrap.cc:492] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.252116 11334 ts_tablet_manager.cc:1403] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:05:46.255177 11334 raft_consensus.cc:359] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.256112 11334 raft_consensus.cc:740] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.256982 11334 consensus_queue.cc:260] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.259477 11334 ts_tablet_manager.cc:1434] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:46.260386 11334 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.278735 11334 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.279731 11334 tablet_bootstrap.cc:492] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.280598 11334 ts_tablet_manager.cc:1403] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.011s
I20260709 13:05:46.282788   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42377
I20260709 13:05:46.283114 11406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42377 every 8 connection(s)
I20260709 13:05:46.283577 11334 raft_consensus.cc:359] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.284341 11334 raft_consensus.cc:740] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.285213 11334 consensus_queue.cc:260] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.287645 11334 ts_tablet_manager.cc:1434] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:46.288704 11334 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.297640 11206 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.298290 11206 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.300837 11206 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.301035 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.310632 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:46.313700 10664 leader_election.cc:343] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.314131 10664 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.314855 11206 raft_consensus.cc:2749] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.326715 11334 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.327977 11334 tablet_bootstrap.cc:492] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.328682 11334 ts_tablet_manager.cc:1403] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.040s	user 0.024s	sys 0.000s
I20260709 13:05:46.331420 11408 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36559
I20260709 13:05:46.331295 11334 raft_consensus.cc:359] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.331952 11408 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:46.331952 11334 raft_consensus.cc:740] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.332810 11408 heartbeater.cc:507] Master 127.0.107.254:36559 requested a full tablet report, sending...
I20260709 13:05:46.332787 11334 consensus_queue.cc:260] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.334499 11334 ts_tablet_manager.cc:1434] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:46.335157 11334 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.346024 10370 ts_manager.cc:194] Re-registered known tserver with Master: 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.346355 11334 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.347139 11334 tablet_bootstrap.cc:492] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.347729 11334 ts_tablet_manager.cc:1403] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:46.349190 11335 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.349707 11335 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.350294 11334 raft_consensus.cc:359] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.350980 11334 raft_consensus.cc:740] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.351809 11335 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.351871 11334 consensus_queue.cc:260] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.354010 11334 ts_tablet_manager.cc:1434] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:46.354496 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.354825 11334 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.356722 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.357414 11379 raft_consensus.cc:2468] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.358494 10664 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.359614 11335 raft_consensus.cc:2804] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.360026 11335 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.360374 11335 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.368731 11334 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.368542 11335 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.369593 11334 tablet_bootstrap.cc:492] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.370204 11334 ts_tablet_manager.cc:1403] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:05:46.370671 11335 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.372017 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.372118 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.372779 11379 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.372825 11334 raft_consensus.cc:359] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.373724 11334 raft_consensus.cc:740] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.374375 11334 consensus_queue.cc:260] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.376387 11334 ts_tablet_manager.cc:1434] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:46.377178 11334 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.381618 11379 raft_consensus.cc:2468] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:46.382903 10664 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.383706 11335 raft_consensus.cc:2804] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:46.384168 11274 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.384835 11335 raft_consensus.cc:697] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:46.384753 11274 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.387005 11274 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.387418 11201 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.388161 11201 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.390379 11201 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.386049 11335 consensus_queue.cc:237] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.392232 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.394618 10370 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43094
I20260709 13:05:46.395913 11201 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.396538 11201 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.399959 11274 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.401984 11163 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.404222 11408 heartbeater.cc:499] Master 127.0.107.254:36559 was elected leader, sending a full tablet report...
I20260709 13:05:46.404605 11274 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.402892 11253 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.404685 11163 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.402627 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.405642 11253 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.406610 10544 raft_consensus.cc:2468] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.408262 11334 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.409370 11334 tablet_bootstrap.cc:492] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.410084 11334 ts_tablet_manager.cc:1403] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.004s
I20260709 13:05:46.410823 11265 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.411403 11265 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.408264 11163 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.413640 11265 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.414345 11334 raft_consensus.cc:359] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.415186 11334 raft_consensus.cc:740] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.415975 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.416085 11334 consensus_queue.cc:260] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.416813 10621 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.417640 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.418283 11378 raft_consensus.cc:2468] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.418766 11334 ts_tablet_manager.cc:1434] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:05:46.419683 11334 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.408109 11253 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.416013 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.420523 10515 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.421475 11253 raft_consensus.cc:2804] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.421830 11253 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.422204 11253 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.422665 11201 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.423930 10695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.430816 11163 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.431399 11163 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.431984 11201 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.432520 11201 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.433938 11163 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.434937 11380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.436271 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.436957 11381 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.435621 11380 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.438786 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.439747 11378 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.440789 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.442564 11274 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.424000 10664 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.443805 11206 raft_consensus.cc:2804] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.444208 11206 raft_consensus.cc:493] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.444600 11206 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.449404 10590 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.450569 11217 raft_consensus.cc:2804] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.451115 10590 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:46.451390 11217 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.464365 11217 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.458496 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.469010 11265 raft_consensus.cc:493] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.469817 11265 raft_consensus.cc:515] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.467819 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.453604 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.475389 11206 raft_consensus.cc:515] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.460254 11163 raft_consensus.cc:493] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.492427 11274 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.492702 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.493937 11206 leader_election.cc:290] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.494606 11201 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.491906 11163 raft_consensus.cc:515] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.476893 11265 leader_election.cc:290] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.477927 10590 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.501188 10664 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:46.493029 11274 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
W20260709 13:05:46.505184 10664 leader_election.cc:343] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.512457 11163 leader_election.cc:290] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.512827 11424 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.513345 11163 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.513980 11424 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.504594 11274 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.506798 11206 raft_consensus.cc:2804] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.506683 11265 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.514106 11163 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.529384 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.529970 10548 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.523384 11206 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.532343 11163 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.533783 11423 raft_consensus.cc:2804] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.533911 11426 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.534310 11423 raft_consensus.cc:493] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.534792 11423 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.534595 11426 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.536890 11426 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.539693 11426 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.540259 11426 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.528177 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.542203 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64cae8b9e6b645c59124d6728f4ecb65" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.525113 11265 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.547910 11206 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.542286 11422 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.553236 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.555563 11422 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.558413 11422 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.559617 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.561297 11265 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.559638 11334 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.563724 11334 tablet_bootstrap.cc:492] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:46.564358 11380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.564651 11334 ts_tablet_manager.cc:1403] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.145s	user 0.064s	sys 0.039s
I20260709 13:05:46.565021 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.565734 11377 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.567541 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.568405 11378 raft_consensus.cc:2468] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.568640 11334 raft_consensus.cc:359] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.569550 11163 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.569773 11334 raft_consensus.cc:740] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.570142 11163 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.570940 11334 consensus_queue.cc:260] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.572494 11163 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.573578 11334 ts_tablet_manager.cc:1434] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:46.574447 11334 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:46.577554 11424 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.579142 11163 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
W20260709 13:05:46.579366 10516 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.579905 10516 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.579831 11163 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.581383 10516 leader_election.cc:304] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.569813 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.582756 11163 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.583124 11377 raft_consensus.cc:2393] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:46.561769 11217 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.584726 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.586529 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.593187 10617 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.587147 11425 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.594040 11217 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.588505 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.594120 11425 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.594480 11274 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.595039 11378 raft_consensus.cc:2468] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.596335 11425 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.597026 11274 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.597289 11422 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.597855 11422 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.589517 10548 raft_consensus.cc:2468] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:46.600556 11422 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.591883 11265 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.600858 11431 raft_consensus.cc:2749] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.601064 10515 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.600006 11379 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.601325 11265 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.590119 11253 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.601590 11431 raft_consensus.cc:2804] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.608352 11431 raft_consensus.cc:493] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.608765 11431 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.609740 11253 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.610571 11253 raft_consensus.cc:2804] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.610788 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:46.611027 11253 raft_consensus.cc:493] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.612731 11265 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.611545 11253 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.615721 11206 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.618366 11206 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.618229 11431 raft_consensus.cc:515] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.620612 11426 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.603956 11163 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.621477 11420 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.622014 11420 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.623962 11433 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.624645 11433 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.636258 11433 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.621510 11163 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.638474 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.638757 11163 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.639175 11378 raft_consensus.cc:2468] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.639864 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.640770 10696 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.642234 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.642599 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.643159 10692 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.643694 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.644783 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.635458 11420 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.646164 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.646745 11378 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.646752 11422 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.647375 11422 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.649572 11422 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
W20260709 13:05:46.654353 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 73036.9us
I20260709 13:05:46.630000 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:46.655269 10664 leader_election.cc:343] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.655160 10516 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96
W20260709 13:05:46.655936 10664 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.651525 11274 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.656363 11201 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.656333 10664 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.656744 11274 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.651834 11426 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.657080 11201 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.657835 11426 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.658298 10664 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:46.658797 11274 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.659260 11201 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.659813 11201 raft_consensus.cc:2804] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.660161 11201 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.657445 11419 raft_consensus.cc:2749] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.625689 11423 raft_consensus.cc:515] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.650054 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.669914 10370 catalog_manager.cc:5697] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.661721 10588 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:46.664937 10516 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.662290 11201 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.664005 11163 raft_consensus.cc:2804] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.663856 11420 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.683745 11424 raft_consensus.cc:2804] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.684428 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.686165 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.686970 10617 raft_consensus.cc:2393] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:46.697721 11375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.698477 11375 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.687640 11163 raft_consensus.cc:493] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.699163 11163 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.662694 11426 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.701593 11374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.668673 11433 raft_consensus.cc:493] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.691007 11424 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.707268 11424 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.707129 11433 raft_consensus.cc:515] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.693236 10694 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.693905 11217 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.694188 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.682452 11425 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.687299 11431 leader_election.cc:290] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.678901 11381 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.702392 11374 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.657475 11253 raft_consensus.cc:515] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.691612 11420 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.710628 11334 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:46.710968 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 46085.8us
I20260709 13:05:46.725445 11253 leader_election.cc:290] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:46.722292 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 63214.5us
I20260709 13:05:46.726343 11334 tablet_bootstrap.cc:492] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
W20260709 13:05:46.726617 10664 leader_election.cc:343] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.727170 11334 ts_tablet_manager.cc:1403] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.153s	user 0.019s	sys 0.000s
I20260709 13:05:46.725466 11420 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.730651 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.730986 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.732928 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.733280 10542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.733685 10548 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.731612 11334 raft_consensus.cc:359] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.736461 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.736928 10543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.738265 11379 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.738299 10543 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.738588 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.739441 10546 raft_consensus.cc:2468] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.740017 10664 leader_election.cc:304] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.741572 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.742010 11421 raft_consensus.cc:2804] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:46.742463 11421 raft_consensus.cc:697] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:46.743151 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.745223 10550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.745973 10550 raft_consensus.cc:2393] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:46.740044 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.747612 10549 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.748248 10543 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:46.748474 10542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.749177 10542 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.736804 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.751104 10544 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.737514 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.752097 10548 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.743294 11421 consensus_queue.cc:237] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.755947 11334 raft_consensus.cc:740] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.756692 11334 consensus_queue.cc:260] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:46.721302 10590 leader_election.cc:343] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.759569 10590 leader_election.cc:304] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.759881 11334 ts_tablet_manager.cc:1434] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.003s
I20260709 13:05:46.706040 11380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.761475 11334 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:46.765164 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 43912.2us
W20260709 13:05:46.766572 10590 leader_election.cc:343] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:46.767369 10590 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.710312 10618 raft_consensus.cc:2468] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.705981 11376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.722426 11206 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.721081 11217 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.759395 11431 raft_consensus.cc:2804] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.779937 11428 raft_consensus.cc:2749] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.782006 10549 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:46.785444 10588 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:46.787741 11426 raft_consensus.cc:2804] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.788127 11426 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.788458 11426 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.792713 11430 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.793310 11430 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.794305 11436 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.794821 11436 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.795812 11430 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.796875 11436 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.798910 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.799913 10623 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.776453 11433 leader_election.cc:290] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.821465 11423 leader_election.cc:290] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.769124 11253 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.832587 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.834182 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.835306 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.835645 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.836221 11381 raft_consensus.cc:3060] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.836514 11379 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.837770 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.838932 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.840231 11373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.840356 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.841270 11373 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.719229 11425 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.842684 11372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.843556 11372 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.845309 11425 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.846446 11431 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.847107 11431 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.839665 11379 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.852326 11206 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:46.853749 11372 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:46.855283 11206 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.857286 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.858944 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc7b153bdd64ea6823d2ca146d5ab61" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.861716 10616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.865311 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:46.768435 11380 raft_consensus.cc:3060] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.866693 11376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.869642 11217 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:46.870641 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:46.870879 11433 raft_consensus.cc:493] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.871598 11420 raft_consensus.cc:493] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.875450 10624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:46.876214 11427 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
W20260709 13:05:46.879732 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 139804us
W20260709 13:05:46.881201 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 108117us
I20260709 13:05:46.880786 10618 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:46.811281 10515 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.829643 11253 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.870684 11374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869dce63600848149f571b6bd169206a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.891064 10624 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:46.896287 10694 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.891343 10617 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.886157 10616 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
W20260709 13:05:46.910482 10516 leader_election.cc:343] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.915408 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:46.916375 10548 raft_consensus.cc:2393] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:46.905491 10621 raft_consensus.cc:2393] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:46.895061 11376 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.918504 10590 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.925781 10590 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:46.926728 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.926905 10590 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.927227 11432 raft_consensus.cc:2804] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.917393 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.927666 10663 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:46.928093 11274 raft_consensus.cc:2804] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:46.929167 10590 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.925236 10588 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:46.927589 11432 raft_consensus.cc:493] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.930831 10590 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.935276 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:46.917610 11372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:46.936280 10544 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:46.937348 11438 raft_consensus.cc:2804] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.937973 11438 raft_consensus.cc:493] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.938438 11438 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.897018 11420 raft_consensus.cc:515] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:46.909184 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 98041.4us
I20260709 13:05:46.943192 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c96c4f1f68468a8302df29e4fa399f" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:46.943825 11428 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.944294 11425 raft_consensus.cc:2804] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.897177 11433 raft_consensus.cc:515] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.940138 11432 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.947065 10516 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.901633 11427 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.946924 11423 raft_consensus.cc:2804] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:46.955978 10516 leader_election.cc:343] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.958227 11437 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.958496 11440 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.958804 11437 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.959031 11440 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.959486 11440 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.960553 11253 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:46.959403 11437 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.964447 11201 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.966549 11201 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.964181 11427 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:46.968313 11375 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
W20260709 13:05:46.970258 10516 leader_election.cc:343] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.952907 11428 raft_consensus.cc:493] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.953115 11425 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
W20260709 13:05:46.983073 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 52302.8us
I20260709 13:05:46.986249 11428 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.971274 11163 raft_consensus.cc:515] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:46.986258 10590 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.988242 10590 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.957726 11423 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:46.956730 10515 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:46.990350 11425 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.990978 11274 raft_consensus.cc:697] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:46.992483 11378 raft_consensus.cc:2468] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:46.994390 11380 raft_consensus.cc:2468] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.009084 11443 raft_consensus.cc:2804] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:47.009796 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 91356.8us
I20260709 13:05:47.010993 10664 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:46.995316 11253 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.009261 11274 consensus_queue.cc:237] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:46.996613 11423 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.016726 10541 raft_consensus.cc:2393] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.023394 11334 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:47.019169 10590 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:47.014549 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.013792 11443 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
W20260709 13:05:47.015547 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 62297.8us
I20260709 13:05:47.017987 11436 raft_consensus.cc:2804] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.028443 11334 tablet_bootstrap.cc:492] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:47.012329 11424 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.031577 11334 ts_tablet_manager.cc:1403] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.270s	user 0.036s	sys 0.034s
I20260709 13:05:47.026103 11433 leader_election.cc:290] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.042754 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a164c68f31403ab1ce21952d49ed2e" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.043347 10698 raft_consensus.cc:3060] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.043936 11435 raft_consensus.cc:2804] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.044384 11435 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.044766 11435 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.036378 11420 leader_election.cc:290] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.028079 11445 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:47.037572 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 54194.4us
I20260709 13:05:47.049440 10516 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:47.051752 11373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.053385 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.053644 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:47.054296 11378 raft_consensus.cc:2393] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.032241 11380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.030333 11377 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 1.
I20260709 13:05:47.055114 11380 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:47.024853 11253 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.056653 11376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.053787 11372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.057386 11253 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.030952 11436 raft_consensus.cc:493] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.058014 11436 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.028158 11443 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.056777 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.059316 11379 raft_consensus.cc:3060] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.038913 11426 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.061867 11163 leader_election.cc:290] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.041445 10514 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.063674 11424 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.065465 11423 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.066685 11426 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.068535 11425 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.070158 10516 leader_election.cc:304] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:47.077380 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.073997 11420 raft_consensus.cc:2804] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.078066 10690 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.078136 11420 raft_consensus.cc:493] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.078657 11420 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.079366 11425 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.081044 10691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.081759 10691 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.074179 11445 raft_consensus.cc:697] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.084930 11424 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.085359 11424 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.085729 11424 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:47.089422 10516 leader_election.cc:343] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.091238 10516 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
W20260709 13:05:47.093164 10516 leader_election.cc:343] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.093314 10698 raft_consensus.cc:2468] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
W20260709 13:05:47.086233 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 44907.7us
I20260709 13:05:47.096369 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:47.097111 10692 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:47.097240 11431 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.094751 11253 raft_consensus.cc:2804] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.098815 11163 raft_consensus.cc:2804] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.103071 11163 raft_consensus.cc:493] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.103550 11163 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.088923 11430 raft_consensus.cc:3060] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.122545 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:47.123281 10548 raft_consensus.cc:2393] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
W20260709 13:05:47.122972 10664 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.123788 10664 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.125442 10542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.126251 10542 raft_consensus.cc:2393] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.129024 11421 raft_consensus.cc:2749] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.132441 11381 raft_consensus.cc:2468] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:47.137356 10516 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:47.087662 11428 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.065721 11437 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.084702 11445 consensus_queue.cc:237] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.168584 11334 raft_consensus.cc:359] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.112958 11163 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.171697 11334 raft_consensus.cc:740] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.155079 11431 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.154381 11372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.156936 11253 raft_consensus.cc:697] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.176191 11334 consensus_queue.cc:260] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.157074 11423 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.190622 11334 ts_tablet_manager.cc:1434] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.158s	user 0.006s	sys 0.000s
I20260709 13:05:47.179524 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:47.187906 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 101884us
I20260709 13:05:47.209017 11437 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.209877 11381 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:47.191213 11431 raft_consensus.cc:2804] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.218587 11420 raft_consensus.cc:515] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.221715 11420 leader_election.cc:290] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.205138 11440 raft_consensus.cc:515] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.224882 11428 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.225834 11440 leader_election.cc:290] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:47.222726 10515 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.225687 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.226055 11163 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.182835 11438 raft_consensus.cc:515] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.250698 11431 raft_consensus.cc:697] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
W20260709 13:05:47.369028 10516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 231795us
I20260709 13:05:47.368340 11412 raft_consensus.cc:493] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.372941 11412 raft_consensus.cc:515] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:47.373397 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 228305us
I20260709 13:05:47.374176 10664 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.374658 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
I20260709 13:05:47.375422 11381 raft_consensus.cc:2468] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 1.
I20260709 13:05:47.376471 10692 raft_consensus.cc:2393] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.376716 10549 raft_consensus.cc:2393] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.371183 11163 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:47.377477 10690 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.377617 11163 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.378051 11163 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.380698 11431 consensus_queue.cc:237] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.381299 11430 raft_consensus.cc:2749] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.382243 10544 raft_consensus.cc:1275] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.362654 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.384311 10623 raft_consensus.cc:2393] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:47.375459 11412 leader_election.cc:290] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.386962 11438 leader_election.cc:290] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.375326 11379 raft_consensus.cc:2468] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:47.244889 11443 raft_consensus.cc:515] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.387531 11163 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.392488 10664 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:47.393227 10516 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.391225 10588 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:47.397109 11443 leader_election.cc:290] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.392210 11253 consensus_queue.cc:237] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.398495 11421 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.398983 11421 raft_consensus.cc:493] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.397953 11443 raft_consensus.cc:2804] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.243476 11432 raft_consensus.cc:515] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.360682 10691 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.399381 11421 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.410384 11335 raft_consensus.cc:2804] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.430243 10588 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:47.430343 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.432184 11163 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:47.432847 10617 consensus_meta.cc:287] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: Time spent flushing consensus metadata: real 0.510s	user 0.004s	sys 0.006s
I20260709 13:05:47.432992 11381 raft_consensus.cc:2393] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.433288 10617 raft_consensus.cc:2468] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:47.434741 11445 raft_consensus.cc:2804] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.435326 11445 raft_consensus.cc:697] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.435918 11335 raft_consensus.cc:697] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:47.437047 11335 consensus_queue.cc:237] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.387421 11435 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.440131 10694 raft_consensus.cc:2393] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.440382 11435 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.436003 11265 raft_consensus.cc:2749] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.442934 11445 consensus_queue.cc:237] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.444243 11424 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.445840 10664 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:47.446460 11424 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.448553 11432 leader_election.cc:290] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.449831 11436 raft_consensus.cc:515] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.451601 11419 raft_consensus.cc:2804] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.452056 11419 raft_consensus.cc:493] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.452406 11419 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:47.454493 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 435360us
I20260709 13:05:47.457587 10664 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.420461 11443 raft_consensus.cc:697] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
W20260709 13:05:47.460305 10590 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:47.405319 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 182664us
I20260709 13:05:47.460945 11440 raft_consensus.cc:2749] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.465471 11334 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:47.468400 10590 leader_election.cc:343] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.469553 10590 leader_election.cc:304] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.473423 10516 leader_election.cc:304] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
W20260709 13:05:47.476018 10590 leader_election.cc:343] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.467950 11436 leader_election.cc:290] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.474159 10515 leader_election.cc:304] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:47.477312 10590 leader_election.cc:343] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.482076 11377 raft_consensus.cc:1275] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.475616 11437 raft_consensus.cc:2804] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.486263 10515 leader_election.cc:304] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.487214 11453 raft_consensus.cc:2749] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.472231 11440 consensus_queue.cc:1048] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:05:47.473794 11432 raft_consensus.cc:2749] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.492923 11443 consensus_queue.cc:237] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.498642 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.499444 10546 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.500012 10542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.500921 10542 raft_consensus.cc:2393] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.503260 10544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.504158 10544 raft_consensus.cc:2393] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.510464 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.511121 10548 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.487064 11437 raft_consensus.cc:697] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.515386 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.510288 11436 raft_consensus.cc:2804] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.516845 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.517588 11377 raft_consensus.cc:3060] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.517006 11436 raft_consensus.cc:697] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.520545 11436 consensus_queue.cc:237] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.521452 11373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.522141 11373 raft_consensus.cc:3060] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.519331 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.516535 10541 raft_consensus.cc:2393] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13c7cc72a0274ec3859962b46603bb12 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.521524 11380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.526093 10590 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.524793 11380 raft_consensus.cc:3060] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.546345 11437 consensus_queue.cc:237] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.534798 11428 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.556703 11421 raft_consensus.cc:515] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.559391 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.564013 10621 raft_consensus.cc:2393] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:47.560297 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:47.565503 10617 raft_consensus.cc:2393] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:47.561425 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:47.569237 10623 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:47.572803 11421 leader_election.cc:290] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.580760 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.585675 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.586767 10618 raft_consensus.cc:2393] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
W20260709 13:05:47.589812 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 123258us
I20260709 13:05:47.595577 10698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:47.595822 10691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.596297 10698 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 1.
I20260709 13:05:47.595566 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e79f3f50a72438383af31b858f986f5" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:47.596802 10697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.597594 10697 raft_consensus.cc:2393] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.598639 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.602227 11376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.602970 11376 raft_consensus.cc:3060] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.603667 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.603813 11372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.604382 10549 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.604573 11372 raft_consensus.cc:3060] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.605705 11380 raft_consensus.cc:2468] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.599361 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01b6c33d7014320a6ee51d8f1c731cb" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:47.607031 10690 raft_consensus.cc:2393] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.607316 10514 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.610229 10545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.610497 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffc7391985ae44a38ba1f2fc35104eca" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.612737 10545 raft_consensus.cc:2393] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.624374 10548 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.625617 10373 catalog_manager.cc:5697] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.634966 11375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63cd6402a88541f794b90126ed48b800" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.639034 11374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.621501 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd44cb78e12145edbcd38d9ead51f9b9" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.646332 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fcc014601624e27bddc57bc6aadb67d" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.660087 11375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.660764 11375 raft_consensus.cc:3060] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.630374 11419 raft_consensus.cc:515] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.663971 10549 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:47.676249 11431 raft_consensus.cc:2749] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.665450 11419 leader_election.cc:290] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:47.665299 11374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.666146 10691 raft_consensus.cc:2393] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13ee73323c814e939932f8e5db34fc96 in this term.
I20260709 13:05:47.666407 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8fb2ab1f70442786c46c19d8323095" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.668488 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.694202 11378 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.669703 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.702736 11428 raft_consensus.cc:2749] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.672873 11373 raft_consensus.cc:2468] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.658042 11379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04a676a3431425a91e63cd315e899fe" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.649530 10546 raft_consensus.cc:2468] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.674431 10623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:47.675028 10515 leader_election.cc:304] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.705742 10623 raft_consensus.cc:2393] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:47.709686 11376 raft_consensus.cc:2468] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
W20260709 13:05:47.686270 10590 leader_election.cc:343] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:47.716620 10590 leader_election.cc:343] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.717298 10590 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.693372 11381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbad48f22c2344388089817bc703b04a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.709825 11437 raft_consensus.cc:2804] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.682307 10516 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.722325 10590 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.679340 11377 raft_consensus.cc:2468] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
W20260709 13:05:47.724155 10516 leader_election.cc:343] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:47.728520 10590 leader_election.cc:343] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:47.729312 10590 leader_election.cc:343] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.733752 10621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:47.734898 11381 raft_consensus.cc:2393] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e730c6ad44924c0ab2a30ccec893dcd3 in current term 2: Already voted for candidate 13c7cc72a0274ec3859962b46603bb12 in this term.
I20260709 13:05:47.737993 11334 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:47.738457 11425 raft_consensus.cc:2749] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.738999 10618 raft_consensus.cc:2468] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:47.731707 11378 raft_consensus.cc:2468] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 3.
I20260709 13:05:47.724511 11375 raft_consensus.cc:2468] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:47.740792 11377 raft_consensus.cc:1240] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Rejecting Update request from peer 13c7cc72a0274ec3859962b46603bb12 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:47.727610 11423 raft_consensus.cc:2804] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.741585 11423 raft_consensus.cc:697] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.739029 11334 tablet_bootstrap.cc:492] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:47.743897 11334 ts_tablet_manager.cc:1403] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.279s	user 0.023s	sys 0.028s
I20260709 13:05:47.724478 11253 raft_consensus.cc:2749] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.742489 11423 consensus_queue.cc:237] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.722934 11437 raft_consensus.cc:493] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.747016 11437 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.721930 11372 raft_consensus.cc:2468] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:47.749691 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09591a419b7a4edf9163e27b3861dda6" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:47.727382 10516 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.753708 11436 raft_consensus.cc:2749] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.755682 10516 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:47.756807 10516 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:47.757551 11436 raft_consensus.cc:2804] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.757900 10516 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:47.758263 11431 raft_consensus.cc:2804] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:47.730871 10590 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.758854 11453 raft_consensus.cc:2804] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.759084 11436 raft_consensus.cc:697] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.759377 11453 raft_consensus.cc:697] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.762773 11453 consensus_queue.cc:237] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.763679 11331 leader_election.cc:304] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.763829 11436 consensus_queue.cc:237] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.764601 11412 raft_consensus.cc:2749] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.758872 11431 raft_consensus.cc:697] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.765548 10515 leader_election.cc:304] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.766399 11253 raft_consensus.cc:2749] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.766345 11431 consensus_queue.cc:237] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.760460 11424 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.768611 11424 raft_consensus.cc:697] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.769610 11424 consensus_queue.cc:237] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.772926 11334 raft_consensus.cc:359] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.773602 11334 raft_consensus.cc:740] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.760963 10590 leader_election.cc:304] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
W20260709 13:05:47.773890 10664 leader_election.cc:343] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.775967 10664 leader_election.cc:304] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:47.780146 11334 consensus_queue.cc:260] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.783180 10590 leader_election.cc:304] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.796432 10590 leader_election.cc:304] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.802491 11334 ts_tablet_manager.cc:1434] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 13:05:47.803495 11334 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:47.784202 11427 raft_consensus.cc:2804] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.823096 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:47.826644 10545 raft_consensus.cc:1275] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.827256 11443 raft_consensus.cc:2804] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.779374 11421 consensus_queue.cc:1048] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:05:47.802374 11448 raft_consensus.cc:2804] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.832297 11335 raft_consensus.cc:493] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e730c6ad44924c0ab2a30ccec893dcd3)
I20260709 13:05:47.803617 11428 raft_consensus.cc:2804] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.767656 10515 leader_election.cc:304] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.837353 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bf69fec47094c99a317be303e0def71" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.803033 11265 consensus_queue.cc:1059] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:05:47.842619 10514 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.843106 10690 raft_consensus.cc:1275] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:47.852762 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 51158.6us
I20260709 13:05:47.867301 10664 leader_election.cc:304] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.865782 11436 raft_consensus.cc:2804] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.869653 11436 raft_consensus.cc:697] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:47.875706 10664 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.858538 11443 raft_consensus.cc:697] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.848829 11437 raft_consensus.cc:515] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.859441 11427 raft_consensus.cc:697] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:47.859670 11448 raft_consensus.cc:697] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.889375 11465 raft_consensus.cc:2804] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:47.889967 11465 raft_consensus.cc:697] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:47.889598 11448 consensus_queue.cc:237] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.890904 11465 consensus_queue.cc:237] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:47.885753 11455 raft_consensus.cc:2804] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.891942 11455 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:05:47.892376 11455 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.860066 11335 raft_consensus.cc:515] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:47.884627 11377 raft_consensus.cc:2393] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13ee73323c814e939932f8e5db34fc96 in current term 2: Already voted for candidate e730c6ad44924c0ab2a30ccec893dcd3 in this term.
I20260709 13:05:47.883826 11428 raft_consensus.cc:697] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.888481 11473 raft_consensus.cc:2749] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.892827 11440 consensus_queue.cc:1048] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:47.874846 11423 raft_consensus.cc:3055] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:05:47.877269 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 109703us
I20260709 13:05:47.894168 11428 consensus_queue.cc:237] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:47.892208 10514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 49705us
I20260709 13:05:47.894546 11423 raft_consensus.cc:740] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:47.896170 10514 leader_election.cc:304] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:47.887594 11443 consensus_queue.cc:237] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.896370 11423 consensus_queue.cc:260] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.897154 11253 raft_consensus.cc:3060] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.897605 11423 raft_consensus.cc:3060] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.897660 11427 consensus_queue.cc:237] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.902334 10623 raft_consensus.cc:1275] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.907368 11335 leader_election.cc:290] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:47.912751 10664 consensus_peers.cc:597] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.908027 11436 consensus_queue.cc:237] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:47.916986 10664 leader_election.cc:304] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.885694 11425 consensus_queue.cc:1048] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:05:47.924039 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:47.937696 10620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7b9860a1dbf4923a357199faf592424" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:47.939335 11437 leader_election.cc:290] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:47.927300 11440 raft_consensus.cc:2749] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.929543 10546 raft_consensus.cc:1240] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 LEADER]: Rejecting Update request from peer 13c7cc72a0274ec3859962b46603bb12 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:47.927659 11377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:47.942296 10663 leader_election.cc:304] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:47.946233 11334 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:47.917999 11438 consensus_queue.cc:1048] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:05:47.948170 11335 raft_consensus.cc:2749] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.948951 10694 raft_consensus.cc:1275] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:47.949885 11475 log.cc:927] Time spent T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.055s	user 0.000s	sys 0.004s
I20260709 13:05:47.950227 11334 tablet_bootstrap.cc:492] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:47.936911 10690 raft_consensus.cc:1275] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:47.957278 11334 ts_tablet_manager.cc:1403] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.154s	user 0.020s	sys 0.007s
I20260709 13:05:47.954522 10369 catalog_manager.cc:5697] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.955801 10370 catalog_manager.cc:5697] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.960019 10369 catalog_manager.cc:5697] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.965188 10370 catalog_manager.cc:5697] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.992865 11334 raft_consensus.cc:359] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:47.994416 11334 raft_consensus.cc:740] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
W20260709 13:05:47.995096 10516 leader_election.cc:343] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.962481 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "611126921768423c97934867e0e9cb44" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:48.002830 11334 consensus_queue.cc:260] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
W20260709 13:05:48.006081 10590 consensus_peers.cc:597] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:48.008740 11334 ts_tablet_manager.cc:1434] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:05:48.011647 10618 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.017360 11412 raft_consensus.cc:493] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.051136 11375 raft_consensus.cc:1275] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.059623 11334 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:48.047188 11412 raft_consensus.cc:515] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:48.114415 11412 leader_election.cc:290] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:48.117889 11378 raft_consensus.cc:1275] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.122781 10590 consensus_peers.cc:597] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:48.124361 11443 consensus_queue.cc:1048] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:48.132097 10590 consensus_peers.cc:597] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:48.146337 11481 log.cc:927] Time spent T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:05:48.178601 10516 consensus_peers.cc:597] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:48.112442 11485 consensus_queue.cc:1048] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:05:48.193696 10618 raft_consensus.cc:2468] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:48.208556 10621 raft_consensus.cc:1275] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.209880 11377 raft_consensus.cc:1275] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.182199 11253 raft_consensus.cc:2749] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:48.221041 11483 consensus_queue.cc:1048] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:48.180377 11455 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
W20260709 13:05:48.224391 10590 consensus_peers.cc:597] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:48.225438 11377 raft_consensus.cc:1275] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.225956 10694 raft_consensus.cc:1275] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.226689 10692 raft_consensus.cc:1275] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.181509 11443 consensus_queue.cc:1048] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:05:48.238240 10548 raft_consensus.cc:1275] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.227306 11453 consensus_queue.cc:1048] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:05:48.273547 11372 raft_consensus.cc:1275] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.280514 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 47028.3us
I20260709 13:05:48.299961 11455 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:48.305668 11201 consensus_queue.cc:1048] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:48.306226 11499 consensus_queue.cc:1048] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:48.313650 10691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:48.314335 10691 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.317787 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:48.319092 11453 consensus_queue.cc:1048] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:48.322263 10515 leader_election.cc:304] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:48.324100 11438 raft_consensus.cc:2804] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:48.324783 11431 consensus_queue.cc:1048] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:48.325501 11438 raft_consensus.cc:697] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:48.326190 11378 raft_consensus.cc:1275] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.326994 11438 consensus_queue.cc:237] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
W20260709 13:05:48.333316 10590 consensus_peers.cc:597] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:48.341398 11436 consensus_queue.cc:1048] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:05:48.343075 10698 raft_consensus.cc:1275] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.346259 10548 raft_consensus.cc:1275] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.348423 11378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
I20260709 13:05:48.352039 10548 raft_consensus.cc:1275] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.361042 10516 leader_election.cc:343] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.362816 11334 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.364881 10541 raft_consensus.cc:1275] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.363554 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2352e31e46a8437ab6897bfbe52d7f9b" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:48.368992 11372 raft_consensus.cc:1275] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.383350 10549 raft_consensus.cc:1275] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.385460 10620 raft_consensus.cc:1275] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:05:48.386184 10620 pending_rounds.cc:85] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Aborting all ops after (but not including) 1
I20260709 13:05:48.386530 10620 pending_rounds.cc:107] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:48.386862 10620 raft_consensus.cc:2889] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.388337 11374 raft_consensus.cc:3060] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.395133 11274 consensus_queue.cc:1048] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:48.382720 11436 consensus_queue.cc:1048] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:05:48.398602 11331 leader_election.cc:304] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:48.399873 11501 raft_consensus.cc:2749] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:48.399875 11497 consensus_queue.cc:1048] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:05:48.382930 11448 consensus_queue.cc:1048] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:05:48.383121 11499 consensus_queue.cc:1048] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:48.417259 11440 consensus_queue.cc:1048] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:48.420881 11375 raft_consensus.cc:1275] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.428553 11334 tablet_bootstrap.cc:492] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:48.430405 11334 ts_tablet_manager.cc:1403] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.371s	user 0.029s	sys 0.000s
W20260709 13:05:48.446329 10664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46713, user_credentials={real_user=slave}} blocked reactor thread for 48139.6us
I20260709 13:05:48.391542 11432 consensus_queue.cc:1048] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:05:48.433670 11443 consensus_queue.cc:1048] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:05:48.448868 11455 consensus_queue.cc:1048] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:05:48.460590 11436 consensus_queue.cc:1048] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:05:48.519681 10663 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 47540.3us
I20260709 13:05:48.528458 11334 raft_consensus.cc:359] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:48.529295 11334 raft_consensus.cc:740] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.531178 11334 consensus_queue.cc:260] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:48.516371 10541 raft_consensus.cc:1275] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.535241 10368 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43094 (request call id 2) took 1623 ms. Trace:
I20260709 13:05:48.535648 10368 rpcz_store.cc:276] 0709 13:05:46.801716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.801954 (+   238us) service_pool.cc:224] Handling call
0709 13:05:48.425336 (+1623382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40975,"reported_tablets":45}
I20260709 13:05:48.517601 10691 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:48.571337 11334 ts_tablet_manager.cc:1434] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.141s	user 0.014s	sys 0.003s
I20260709 13:05:48.572645 11334 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:48.574653 11508 log.cc:927] Time spent T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:05:48.587503 10514 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:48.593570 10590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42377, user_credentials={real_user=slave}} blocked reactor thread for 82694.5us
I20260709 13:05:48.580991 11375 raft_consensus.cc:1275] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.597229 11505 log.cc:927] Time spent T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 13:05:48.641939 10590 consensus_peers.cc:597] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:48.642936 10590 consensus_peers.cc:597] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:48.593798 11485 raft_consensus.cc:2804] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:48.645761 11517 log.cc:927] Time spent T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:05:48.646154 11373 raft_consensus.cc:1275] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.642985 11428 consensus_queue.cc:1048] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:05:48.659173 11485 raft_consensus.cc:697] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
W20260709 13:05:48.679631 11514 log.cc:927] Time spent T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:05:48.682561 11515 log.cc:927] Time spent T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:05:48.688095 11509 log.cc:927] Time spent T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:05:48.687122 11485 consensus_queue.cc:237] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:48.683421 11374 raft_consensus.cc:1275] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.816418 11373 raft_consensus.cc:1275] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:48.821561 10621 raft_consensus.cc:1275] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.841189 11519 log.cc:927] Time spent T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:05:48.845367 11522 log.cc:927] Time spent T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:05:48.883694 11507 log.cc:927] Time spent T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.414s	user 0.003s	sys 0.000s
I20260709 13:05:48.889434 10541 raft_consensus.cc:1275] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.918497 10617 raft_consensus.cc:1275] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.938946 11526 log.cc:927] Time spent T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:05:48.959692 11334 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.960776 11334 tablet_bootstrap.cc:492] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:48.961539 11334 ts_tablet_manager.cc:1403] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.389s	user 0.029s	sys 0.034s
I20260709 13:05:48.983021 11373 raft_consensus.cc:1275] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:48.971702 11253 consensus_queue.cc:1048] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
W20260709 13:05:49.044368 10515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 106675us
W20260709 13:05:49.058768 10664 consensus_peers.cc:597] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:49.079347 11443 consensus_queue.cc:1048] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:49.094271 10590 consensus_peers.cc:597] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.124677 11525 consensus_queue.cc:1048] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:49.137595 11543 consensus_queue.cc:1048] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.131441 11334 raft_consensus.cc:359] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.138880 11334 raft_consensus.cc:740] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
W20260709 13:05:49.140623 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 46002.7us
I20260709 13:05:49.133167 11482 consensus_queue.cc:1048] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:05:49.071059 11421 consensus_queue.cc:1048] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:05:49.144591 11334 consensus_queue.cc:260] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.167443 11334 ts_tablet_manager.cc:1434] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.206s	user 0.006s	sys 0.000s
I20260709 13:05:49.168447 11334 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:49.190033 10588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40563, user_credentials={real_user=slave}} blocked reactor thread for 47096.4us
I20260709 13:05:49.209936 11485 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.210599 11485 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:49.216750 11485 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:05:49.217422 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13ee73323c814e939932f8e5db34fc96" is_pre_election: true
I20260709 13:05:49.219450 11374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e" is_pre_election: true
W20260709 13:05:49.227109 11548 log.cc:927] Time spent T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:05:49.237390 11543 consensus_queue.cc:1048] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:49.238337 10516 consensus_peers.cc:597] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.239073 10516 consensus_peers.cc:597] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.238899 11543 consensus_queue.cc:1048] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:49.242951 10516 leader_election.cc:343] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:49.251860 10590 consensus_peers.cc:597] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:49.259634 10590 consensus_peers.cc:597] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:49.266594 11552 raft_consensus.cc:493] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.268903 11552 raft_consensus.cc:515] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.272778 11552 leader_election.cc:290] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
W20260709 13:05:49.277209 11551 log.cc:927] Time spent T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:05:49.277396 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "13c7cc72a0274ec3859962b46603bb12" is_pre_election: true
I20260709 13:05:49.282480 11334 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.283552 11334 tablet_bootstrap.cc:492] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.284243 11334 ts_tablet_manager.cc:1403] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.002s
I20260709 13:05:49.286803 11334 raft_consensus.cc:359] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.287495 11334 raft_consensus.cc:740] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.304724 10541 raft_consensus.cc:1275] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.308264 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8ad15cc9f46928e9304f151ccc0af" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" is_pre_election: true
I20260709 13:05:49.308640 10618 raft_consensus.cc:1275] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.309844 11331 leader_election.cc:304] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e; no voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:49.311121 11552 raft_consensus.cc:2749] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.307477 11334 consensus_queue.cc:260] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.313359 11334 ts_tablet_manager.cc:1434] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:05:49.342303 10514 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e730c6ad44924c0ab2a30ccec893dcd3; no voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e
W20260709 13:05:49.345603 10664 consensus_peers.cc:597] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:49.353659 10664 consensus_peers.cc:597] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.354719 10664 consensus_peers.cc:597] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:49.357895 11335 consensus_queue.cc:1048] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.365284 11334 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
W20260709 13:05:49.366474 10664 consensus_peers.cc:597] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:49.369447 11436 raft_consensus.cc:2749] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:49.378213 10664 consensus_peers.cc:597] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:49.391448 11464 consensus_queue.cc:1048] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:49.393939 10516 consensus_peers.cc:597] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.405069 10368 catalog_manager.cc:5697] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 3, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } }
I20260709 13:05:49.425572 10368 catalog_manager.cc:5697] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.429656 10368 catalog_manager.cc:5697] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.431950 10368 catalog_manager.cc:5697] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.433820 10368 catalog_manager.cc:5697] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.435722 10368 catalog_manager.cc:5697] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.439321 10368 catalog_manager.cc:5697] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.441184 10368 catalog_manager.cc:5697] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.533435 10696 raft_consensus.cc:1275] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.553186 11455 consensus_queue.cc:1048] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.557281 11334 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.570176 11334 tablet_bootstrap.cc:492] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.571097 11334 ts_tablet_manager.cc:1403] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.206s	user 0.024s	sys 0.004s
I20260709 13:05:49.603355 11334 raft_consensus.cc:359] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:49.604255 11334 raft_consensus.cc:740] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.605167 11334 consensus_queue.cc:260] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:49.608875 11334 ts_tablet_manager.cc:1434] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
I20260709 13:05:49.613523 11334 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.640237 11334 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.641319 11334 tablet_bootstrap.cc:492] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.642616 11334 ts_tablet_manager.cc:1403] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.000s
I20260709 13:05:49.642359 10369 catalog_manager.cc:5697] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.644156 10369 catalog_manager.cc:5697] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.645779 10369 catalog_manager.cc:5697] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.647645 11334 raft_consensus.cc:359] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.648412 11334 raft_consensus.cc:740] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.649156 11334 consensus_queue.cc:260] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.652611 11334 ts_tablet_manager.cc:1434] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:49.653523 11334 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.658352 10369 catalog_manager.cc:5697] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.660151 10369 catalog_manager.cc:5697] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.661931 10369 catalog_manager.cc:5697] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.663349 10369 catalog_manager.cc:5697] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.676909 11334 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:05:49.678098 11334 tablet_bootstrap.cc:492] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.678915 11334 ts_tablet_manager.cc:1403] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.005s
I20260709 13:05:49.681885 11334 raft_consensus.cc:359] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.682916 11334 raft_consensus.cc:740] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.683789 11334 consensus_queue.cc:260] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.686251 11334 ts_tablet_manager.cc:1434] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:49.687207 11334 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.709740 11334 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.710870 11334 tablet_bootstrap.cc:492] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.711627 11334 ts_tablet_manager.cc:1403] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:05:49.714622 11334 raft_consensus.cc:359] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.715372 11334 raft_consensus.cc:740] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.716136 11334 consensus_queue.cc:260] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.717986 11334 ts_tablet_manager.cc:1434] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:49.718827 11334 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.738863 10370 catalog_manager.cc:5697] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.740849 10370 catalog_manager.cc:5697] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.743829 11334 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.745007 11334 tablet_bootstrap.cc:492] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.745720 11334 ts_tablet_manager.cc:1403] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.000s
I20260709 13:05:49.748456 11334 raft_consensus.cc:359] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.749102 11334 raft_consensus.cc:740] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.749861 11334 consensus_queue.cc:260] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:49.752147 11334 ts_tablet_manager.cc:1434] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:05:49.752950 11334 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.852665 11334 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.854818 11375 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.855175 11334 tablet_bootstrap.cc:492] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.856123 11334 ts_tablet_manager.cc:1403] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.103s	user 0.048s	sys 0.022s
I20260709 13:05:49.859314 11334 raft_consensus.cc:359] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.860108 11334 raft_consensus.cc:740] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.861001 11334 consensus_queue.cc:260] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.863499 11334 ts_tablet_manager.cc:1434] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:49.867005 11334 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.937604 11334 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.938692 11334 tablet_bootstrap.cc:492] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.939399 11334 ts_tablet_manager.cc:1403] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.013s
I20260709 13:05:49.942018 11334 raft_consensus.cc:359] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.942946 11334 raft_consensus.cc:740] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.943662 11334 consensus_queue.cc:260] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:49.946741 11334 ts_tablet_manager.cc:1434] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:49.947525 11334 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:49.955543 11375 raft_consensus.cc:3060] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.988667 11334 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.989670 11334 tablet_bootstrap.cc:492] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:49.990329 11334 ts_tablet_manager.cc:1403] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.043s	user 0.025s	sys 0.007s
I20260709 13:05:49.993012 11334 raft_consensus.cc:359] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:49.993757 11334 raft_consensus.cc:740] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.994575 11334 consensus_queue.cc:260] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:49.996814 11334 ts_tablet_manager.cc:1434] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:49.997601 11334 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.015148 11334 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.015997 11334 tablet_bootstrap.cc:492] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.016642 11334 ts_tablet_manager.cc:1403] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:05:50.018584 11334 raft_consensus.cc:359] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.019048 11334 raft_consensus.cc:740] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.019634 11334 consensus_queue.cc:260] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.021136 11334 ts_tablet_manager.cc:1434] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:05:50.021773 11334 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.050757 11334 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.051841 11334 tablet_bootstrap.cc:492] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.052590 11334 ts_tablet_manager.cc:1403] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.015s
I20260709 13:05:50.054582 11334 raft_consensus.cc:359] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:50.055150 11334 raft_consensus.cc:740] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.055716 11334 consensus_queue.cc:260] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:50.057682 11334 ts_tablet_manager.cc:1434] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:50.058531 11334 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.080237 11334 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.081087 11334 tablet_bootstrap.cc:492] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.081728 11334 ts_tablet_manager.cc:1403] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 13:05:50.084508 11334 raft_consensus.cc:359] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.085232 11334 raft_consensus.cc:740] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.085857 11334 consensus_queue.cc:260] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.087235 11334 ts_tablet_manager.cc:1434] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:50.087931 11334 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.101114 11334 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.102074 11334 tablet_bootstrap.cc:492] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.102741 11334 ts_tablet_manager.cc:1403] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:05:50.104825 11334 raft_consensus.cc:359] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.105355 11334 raft_consensus.cc:740] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.105942 11334 consensus_queue.cc:260] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.107781 11334 ts_tablet_manager.cc:1434] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:50.109000 11334 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.128575 11334 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.129606 11334 tablet_bootstrap.cc:492] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.130319 11334 ts_tablet_manager.cc:1403] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.005s
I20260709 13:05:50.133600 11334 raft_consensus.cc:359] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.134281 11334 raft_consensus.cc:740] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.135251 11334 consensus_queue.cc:260] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.147330 11334 ts_tablet_manager.cc:1434] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:50.148244 11334 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.163401 11334 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.164361 11334 tablet_bootstrap.cc:492] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.165047 11334 ts_tablet_manager.cc:1403] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:05:50.167235 11334 raft_consensus.cc:359] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.168455 11334 raft_consensus.cc:740] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.169335 11334 consensus_queue.cc:260] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.171293 11334 ts_tablet_manager.cc:1434] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:05:50.172610 11334 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.185074 11379 raft_consensus.cc:3060] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.185303 11334 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.186344 11334 tablet_bootstrap.cc:492] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.186995 11334 ts_tablet_manager.cc:1403] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:05:50.190263 11334 raft_consensus.cc:359] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.190970 11334 raft_consensus.cc:740] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.191628 11334 consensus_queue.cc:260] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.196703 11334 ts_tablet_manager.cc:1434] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:50.197533 11334 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.214334 11334 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.215073 11334 tablet_bootstrap.cc:492] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.215672 11334 ts_tablet_manager.cc:1403] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 13:05:50.218180 11334 raft_consensus.cc:359] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.218850 11334 raft_consensus.cc:740] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.219537 11334 consensus_queue.cc:260] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.221664 11334 ts_tablet_manager.cc:1434] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:50.222453 11334 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.226379 11379 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.244623 11334 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.245723 11334 tablet_bootstrap.cc:492] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.246824 11334 ts_tablet_manager.cc:1403] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.008s
I20260709 13:05:50.251648 11334 raft_consensus.cc:359] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.252359 11334 raft_consensus.cc:740] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.253033 11334 consensus_queue.cc:260] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:50.254258 11334 ts_tablet_manager.cc:1434] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:50.255102 11334 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.270805 11375 raft_consensus.cc:3060] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.275962 11334 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.277009 11334 tablet_bootstrap.cc:492] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.277671 11334 ts_tablet_manager.cc:1403] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:05:50.280418 11334 raft_consensus.cc:359] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:50.281090 11334 raft_consensus.cc:740] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.281801 11334 consensus_queue.cc:260] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:50.284409 11334 ts_tablet_manager.cc:1434] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:50.286675 11334 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap starting.
I20260709 13:05:50.302160 11334 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.303069 11334 tablet_bootstrap.cc:492] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Bootstrap complete.
I20260709 13:05:50.303664 11334 ts_tablet_manager.cc:1403] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:05:50.305608 11334 raft_consensus.cc:359] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.306217 11334 raft_consensus.cc:740] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.306715 11334 consensus_queue.cc:260] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:50.308143 11334 ts_tablet_manager.cc:1434] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:50.341910 11379 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.526400 11375 raft_consensus.cc:3060] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.620394 11375 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.746138 11375 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:54.478638 10663 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36903, user_credentials={real_user=slave}} blocked reactor thread for 54640.3us
I20260709 13:05:56.305486   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:56.353508 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:56.354355 10617 raft_consensus.cc:606] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:56.366442 10548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa"
dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
mode: GRACEFUL
new_leader_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:56570
I20260709 13:05:56.367067 10548 raft_consensus.cc:606] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Received request to transfer leadership to TS 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:56.369895 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:56.370514 10617 raft_consensus.cc:606] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:56.383205 10692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:56.383792 10692 raft_consensus.cc:606] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:56.386134 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:56.386683 10617 raft_consensus.cc:606] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:05:56.388989 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:56.389480 10617 raft_consensus.cc:606] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:56.391669 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:56.392249 10617 raft_consensus.cc:606] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:56.393257   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:05:56.394534   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:56.440958 11624 raft_consensus.cc:993] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.441679 11630 raft_consensus.cc:1081] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.441870 11631 raft_consensus.cc:993] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: : Instructing follower 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:56.442606 11622 raft_consensus.cc:1081] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Signalling peer 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:56.443693 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:05:56.445478 11379 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.445499 10617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:58666
I20260709 13:05:56.445981 11379 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.446190 10617 raft_consensus.cc:493] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.446614 10617 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.454406 10617 raft_consensus.cc:515] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.454614 11379 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:56.456722 11379 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:56.456717 10617 leader_election.cc:290] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:56.457966 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:56.458647 10548 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.464411 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:56.465085 10541 raft_consensus.cc:3055] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.465174 10548 raft_consensus.cc:2468] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.465440 10541 raft_consensus.cc:740] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:56.466336 10541 consensus_queue.cc:260] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.466652 11331 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:56.467474 10541 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.468642 11637 raft_consensus.cc:2804] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.469777 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43aaaf38cfe7411aa64c01c2519884fa" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:56.470371 10696 raft_consensus.cc:3060] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.475206 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdbdcb35eeb430e824a441d12445254" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:56.476121 10692 raft_consensus.cc:3055] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.476480 10692 raft_consensus.cc:740] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:56.477397 10692 consensus_queue.cc:260] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:56.478152 10696 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:56.478847 10692 raft_consensus.cc:3060] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.482931 10588 leader_election.cc:304] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:56.485055 11632 raft_consensus.cc:2804] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.476480 10541 raft_consensus.cc:2468] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:56.495364 10692 raft_consensus.cc:2468] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.499053 11632 raft_consensus.cc:697] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.500268 11632 consensus_queue.cc:237] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.517727 11640 raft_consensus.cc:993] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.518242 11639 raft_consensus.cc:993] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.520212 11644 raft_consensus.cc:1081] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.521186 11644 raft_consensus.cc:1081] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.521957 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:56.522746 11379 raft_consensus.cc:493] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.523181 11379 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.531203 11375 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:56.531862 11375 raft_consensus.cc:493] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.532186 11375 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.535614 10368 catalog_manager.cc:5697] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.542879 11379 raft_consensus.cc:515] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.547629 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:56.548226 10541 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.552552 11375 raft_consensus.cc:515] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.556282 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828be85a64dc4ceaaa111481681d3550" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:56.558264 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:56.558492 10617 raft_consensus.cc:3055] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:56.559090 10617 raft_consensus.cc:740] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.558887 10549 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.560788 10541 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:56.560410 10617 consensus_queue.cc:260] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.561961 10617 raft_consensus.cc:3060] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.562549 11637 raft_consensus.cc:697] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:56.563563 11637 consensus_queue.cc:237] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:56.564371 11331 leader_election.cc:304] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:56.566851 11375 leader_election.cc:290] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:56.567886 10549 raft_consensus.cc:2468] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.568357 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dd264c867c4296a9a2f7fd53ac51e4" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:56.567934 11638 raft_consensus.cc:2804] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:56.569131 10618 raft_consensus.cc:3055] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.569330 11331 leader_election.cc:304] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:56.569569 10618 raft_consensus.cc:740] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.570194 11379 leader_election.cc:290] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:56.573724 11637 raft_consensus.cc:2804] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.575665 10617 raft_consensus.cc:2468] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:56.574314 11637 raft_consensus.cc:697] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:56.574170 10618 consensus_queue.cc:260] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.577481 11637 consensus_queue.cc:237] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.579468 10618 raft_consensus.cc:3060] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.569561 11638 raft_consensus.cc:697] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:56.590718 11638 consensus_queue.cc:237] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:56.596669 10618 raft_consensus.cc:2468] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.597847 10369 catalog_manager.cc:5697] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.628854 10369 catalog_manager.cc:5697] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 2 to 3, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.630218 10369 catalog_manager.cc:5697] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.659919 11642 raft_consensus.cc:993] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower e730c6ad44924c0ab2a30ccec893dcd3 to start an election
I20260709 13:05:56.660387 11632 raft_consensus.cc:1081] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer e730c6ad44924c0ab2a30ccec893dcd3 to start an election
I20260709 13:05:56.661852 10549 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97"
dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
 from {username='slave'} at 127.0.0.1:57488
I20260709 13:05:56.662321 10549 raft_consensus.cc:493] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.662626 10549 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.668632 10549 raft_consensus.cc:515] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.670887 10549 leader_election.cc:290] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:56.671587 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:56.671367 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acd69d2652a64ddba95c78f88d6e9a97" candidate_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:56.672242 10693 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.672587 10618 raft_consensus.cc:3055] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.675998 10618 raft_consensus.cc:740] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.688962 10618 consensus_queue.cc:260] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.686120 10693 raft_consensus.cc:2468] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:56.725518 10618 raft_consensus.cc:3060] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.726703 10514 leader_election.cc:304] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:56.731323 11655 raft_consensus.cc:2804] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.732203 11655 raft_consensus.cc:697] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Leader. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:56.733117 11655 consensus_queue.cc:237] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:56.738633 10618 raft_consensus.cc:2468] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e730c6ad44924c0ab2a30ccec893dcd3 in term 2.
I20260709 13:05:56.741233 11632 raft_consensus.cc:993] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.742579 11620 raft_consensus.cc:1081] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.744915 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:56.745489 11379 raft_consensus.cc:493] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.745880 11379 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.758052 11379 raft_consensus.cc:515] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.758147 10369 catalog_manager.cc:5697] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.760339 11379 leader_election.cc:290] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:56.760560 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:56.760864 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98054391f4644c9a7c7ccf6a6e2db80" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:56.761216 10618 raft_consensus.cc:3055] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.761432 10692 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.761596 10618 raft_consensus.cc:740] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.762428 10618 consensus_queue.cc:260] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.763645 10618 raft_consensus.cc:3060] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.768636 10692 raft_consensus.cc:2468] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.770457 11329 leader_election.cc:304] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:56.771215 10618 raft_consensus.cc:2468] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:56.771222 11638 raft_consensus.cc:2804] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.772058 11638 raft_consensus.cc:697] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:56.773066 11638 consensus_queue.cc:237] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.785313 11642 raft_consensus.cc:993] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.785809 11644 raft_consensus.cc:1081] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:56.785979 10369 catalog_manager.cc:5697] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.787266 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:56.787849 11379 raft_consensus.cc:493] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.788229 11379 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.795352 11379 raft_consensus.cc:515] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.797014 11379 leader_election.cc:290] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:56.797516 10618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:56.798173 10618 raft_consensus.cc:3055] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:56.798098 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d4f443226d4b09b41d1b414448b007" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:56.798563 10618 raft_consensus.cc:740] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:56.798739 10692 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.799324 10618 consensus_queue.cc:260] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.800253 10618 raft_consensus.cc:3060] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:56.803297 10692 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:56.804308 11329 leader_election.cc:304] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:56.804667 10618 raft_consensus.cc:2468] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:56.805018 11638 raft_consensus.cc:2804] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:56.805557 11638 raft_consensus.cc:697] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:56.806452 11638 consensus_queue.cc:237] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:56.822669 10369 catalog_manager.cc:5697] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 2 to 3, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.956212 10692 raft_consensus.cc:1275] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.957553 11638 consensus_queue.cc:1048] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.984382 10541 raft_consensus.cc:1275] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.985651 11648 consensus_queue.cc:1048] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.001022 10693 raft_consensus.cc:1275] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.002216 11644 consensus_queue.cc:1048] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.026657 10618 raft_consensus.cc:1275] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.028456 11648 consensus_queue.cc:1048] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.033890 10541 raft_consensus.cc:1275] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.037171 11642 consensus_queue.cc:1048] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.044889 10549 raft_consensus.cc:1275] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.046293 11648 consensus_queue.cc:1048] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.068665 10618 raft_consensus.cc:1275] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.070567 11648 consensus_queue.cc:1048] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.093817 10541 raft_consensus.cc:1275] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.097311 11648 consensus_queue.cc:1048] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.129815 10693 raft_consensus.cc:1275] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.133409 11622 consensus_queue.cc:1048] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.171185 10618 raft_consensus.cc:1275] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer e730c6ad44924c0ab2a30ccec893dcd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.173214 11631 consensus_queue.cc:1048] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.186162 10696 raft_consensus.cc:1275] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.187698 11648 consensus_queue.cc:1048] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.226594 10618 raft_consensus.cc:1275] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.227981 11637 consensus_queue.cc:1048] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:57.262777 10696 raft_consensus.cc:1275] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.268188 11637 consensus_queue.cc:1048] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.298755 10616 raft_consensus.cc:1275] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.301800 11637 consensus_queue.cc:1048] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.395517   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.431416   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.470037 10693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:57.470768 10693 raft_consensus.cc:606] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:57.473480 10616 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:57.474056 10616 raft_consensus.cc:606] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:57.476787 10693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:57.477520 10693 raft_consensus.cc:606] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:57.480293 10696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:57.480911 10696 raft_consensus.cc:606] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:57.483578 10546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda"
dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:56570
I20260709 13:05:57.484215 10546 raft_consensus.cc:606] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:57.486706 10616 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:57.487222 10616 raft_consensus.cc:606] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:57.488354   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:57.489823   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:57.639791 11645 raft_consensus.cc:993] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.640357 11645 raft_consensus.cc:1081] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.642097 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:05:57.642633 11379 raft_consensus.cc:493] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.642994 11379 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.649612 11379 raft_consensus.cc:515] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.651944 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:57.652098 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e9d448f8854ecd81d429b12890f07a" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:57.652328 11379 leader_election.cc:290] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:57.652647 10696 raft_consensus.cc:3055] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.652765 10549 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.653055 10696 raft_consensus.cc:740] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:57.653990 10696 consensus_queue.cc:260] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.655196 10696 raft_consensus.cc:3060] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.659382 10549 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:57.660168 10696 raft_consensus.cc:2468] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:57.660642 11331 leader_election.cc:304] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:57.661441 11648 raft_consensus.cc:2804] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.661926 11648 raft_consensus.cc:697] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:57.662695 11648 consensus_queue.cc:237] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.673295 10370 catalog_manager.cc:5697] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 2 to 3, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.691717 11645 raft_consensus.cc:993] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.692263 11630 raft_consensus.cc:1081] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.693631 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:05:57.694116 11379 raft_consensus.cc:493] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.694401 11379 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.699296 11379 raft_consensus.cc:515] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.700819 11379 leader_election.cc:290] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:57.701336 10616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:57.701781 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "708dee8c708a4b6e91bafafed7d61c2e" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:57.701964 10616 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.702410 10696 raft_consensus.cc:3055] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.702723 10696 raft_consensus.cc:740] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:57.703481 10696 consensus_queue.cc:260] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.704694 10696 raft_consensus.cc:3060] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.708873 10616 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.710726 11330 leader_election.cc:304] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:57.711550 11648 raft_consensus.cc:2804] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.712035 11648 raft_consensus.cc:697] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:57.712836 11648 consensus_queue.cc:237] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.717047 10696 raft_consensus.cc:2468] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.725106 10370 catalog_manager.cc:5697] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.726521 11620 raft_consensus.cc:993] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.727085 11644 raft_consensus.cc:1081] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.728742 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:57.729211 11379 raft_consensus.cc:493] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.729544 11379 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.734694 11379 raft_consensus.cc:515] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.738219 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:57.738744 10617 raft_consensus.cc:3055] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.739020 10617 raft_consensus.cc:740] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:57.739689 10617 consensus_queue.cc:260] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.740665 10617 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.752277 11379 leader_election.cc:290] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:57.753633 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559deb5d3dd44d6f8cd2e98f479b9168" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:57.754362 10693 raft_consensus.cc:3060] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.754873 10617 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.756567 11330 leader_election.cc:304] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:57.757323 11648 raft_consensus.cc:2804] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.757791 11648 raft_consensus.cc:697] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:57.758658 11648 consensus_queue.cc:237] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:57.761400 10693 raft_consensus.cc:2468] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.772408 10370 catalog_manager.cc:5697] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.829195 11645 raft_consensus.cc:993] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:57.829769 11630 raft_consensus.cc:1081] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:57.831483 10616 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:58680
I20260709 13:05:57.831995 10616 raft_consensus.cc:493] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.832265 10616 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.838409 10616 raft_consensus.cc:515] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:57.838810 11676 raft_consensus.cc:993] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.839430 11644 raft_consensus.cc:1081] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.840232 10616 leader_election.cc:290] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:57.841104 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:57.841413 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3353aa19c6e4404fbcbd2f5e810074c5" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:57.841635 10693 raft_consensus.cc:3055] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.841958 10549 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.842001 10693 raft_consensus.cc:740] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:57.842845 10693 consensus_queue.cc:260] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:57.843858 10693 raft_consensus.cc:3060] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.844329 11374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:57.844887 11374 raft_consensus.cc:493] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.845209 11374 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.853724 10693 raft_consensus.cc:2468] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:57.854851 10588 leader_election.cc:304] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 13ee73323c814e939932f8e5db34fc96; no voters: 
I20260709 13:05:57.855647 11644 raft_consensus.cc:2804] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.857894 11644 raft_consensus.cc:697] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:57.858705 11644 consensus_queue.cc:237] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:57.858824 10549 raft_consensus.cc:2468] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:57.863348 11374 raft_consensus.cc:515] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:57.865720 11374 leader_election.cc:290] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:57.866711 10549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:57.867233 10549 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.866662 10616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08b155b1b294e4ea4a4b29755b98651" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:57.868371 10616 raft_consensus.cc:3055] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.868645 10616 raft_consensus.cc:740] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:57.869467 10616 consensus_queue.cc:260] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:57.873725 10549 raft_consensus.cc:2468] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.874730 11331 leader_election.cc:304] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:57.875615 11684 raft_consensus.cc:2804] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.876214 11684 raft_consensus.cc:697] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:57.877029 11684 consensus_queue.cc:237] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } }
I20260709 13:05:57.877395 10370 catalog_manager.cc:5697] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.875639 10616 raft_consensus.cc:3060] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.886668 10616 raft_consensus.cc:2468] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:57.889992 10369 catalog_manager.cc:5697] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.911167 11656 raft_consensus.cc:993] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.911654 11655 raft_consensus.cc:1081] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:57.913272 11374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53058
I20260709 13:05:57.913730 11374 raft_consensus.cc:493] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.914069 11374 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.918956 11374 raft_consensus.cc:515] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.921717 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:57.921917 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc077c5bba0428d9350f538bd2aeeda" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:57.922351 10693 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.922480 10548 raft_consensus.cc:3055] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.922510 11374 leader_election.cc:290] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:57.922904 10548 raft_consensus.cc:740] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e730c6ad44924c0ab2a30ccec893dcd3, State: Running, Role: LEADER
I20260709 13:05:57.923837 10548 consensus_queue.cc:260] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.925073 10548 raft_consensus.cc:3060] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.929265 10693 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:57.930320 11329 leader_election.cc:304] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:57.931221 11684 raft_consensus.cc:2804] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.931676 10548 raft_consensus.cc:2468] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:57.931711 11684 raft_consensus.cc:697] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:57.932837 11684 consensus_queue.cc:237] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:57.942636 10369 catalog_manager.cc:5697] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 2 to 3, leader changed from e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.118225 10693 raft_consensus.cc:1275] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.119611 11684 consensus_queue.cc:1048] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.134328 10616 raft_consensus.cc:1275] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.136193 11684 consensus_queue.cc:1048] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.156296 10616 raft_consensus.cc:1275] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.157754 11684 consensus_queue.cc:1048] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.169598 10693 raft_consensus.cc:1275] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.171978 11684 consensus_queue.cc:1048] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.201812 10549 raft_consensus.cc:1275] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.203053 11684 consensus_queue.cc:1048] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:58.216003 10696 raft_consensus.cc:1275] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.218978 11638 consensus_queue.cc:1048] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:58.274909 10693 raft_consensus.cc:1275] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.276136 11676 consensus_queue.cc:1048] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.291498 10549 raft_consensus.cc:1275] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.292799 11701 consensus_queue.cc:1048] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.316038 10616 raft_consensus.cc:1275] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.317327 11637 consensus_queue.cc:1048] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.328706 10549 raft_consensus.cc:1275] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.330264 11648 consensus_queue.cc:1048] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.388289 10549 raft_consensus.cc:1275] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.389539 11648 consensus_queue.cc:1048] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:58.398801 10693 raft_consensus.cc:1275] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.400703 11707 consensus_queue.cc:1048] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:58.490660   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:58.526132   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:58.562045 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:58.562601 10617 raft_consensus.cc:606] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:58.564811 10693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:58.565274 10693 raft_consensus.cc:606] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:58.567431 10693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:58.567942 10693 raft_consensus.cc:606] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:05:58.570905 10696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4849594fca843a982888e29f557c3f4"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:05:58.571408 10696 raft_consensus.cc:606] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 13c7cc72a0274ec3859962b46603bb12
I20260709 13:05:58.572551   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:58.573911   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:58.629557 11630 raft_consensus.cc:993] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:58.630172 11630 raft_consensus.cc:1081] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 13c7cc72a0274ec3859962b46603bb12 to start an election
I20260709 13:05:58.632133 10617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4849594fca843a982888e29f557c3f4"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
 from {username='slave'} at 127.0.0.1:58680
I20260709 13:05:58.632649 10617 raft_consensus.cc:493] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.632956 10617 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.639554 10617 raft_consensus.cc:515] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:58.640204 11676 raft_consensus.cc:993] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.640793 11644 raft_consensus.cc:1081] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.641359 10617 leader_election.cc:290] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:58.642225 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:05:58.642776 11379 raft_consensus.cc:493] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.643132 11379 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.644194 10696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:58.644797 10696 raft_consensus.cc:3055] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.645076 10696 raft_consensus.cc:740] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:58.645874 10696 consensus_queue.cc:260] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:58.646685 10696 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.648880 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4849594fca843a982888e29f557c3f4" candidate_uuid: "13c7cc72a0274ec3859962b46603bb12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:58.649502 10546 raft_consensus.cc:3060] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.654161 11379 raft_consensus.cc:515] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.657037 10546 raft_consensus.cc:2468] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:58.657385 11379 leader_election.cc:290] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:58.657693 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:58.658425 10693 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.656774 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f0aa6cf57a4d228d8214c7dca07866" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:58.659001 10590 leader_election.cc:304] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:58.659507 10617 raft_consensus.cc:3055] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:58.659865 10617 raft_consensus.cc:740] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:58.659957 11676 raft_consensus.cc:2804] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.660696 10617 consensus_queue.cc:260] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.661140 11676 raft_consensus.cc:697] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Becoming Leader. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:58.662189 11676 consensus_queue.cc:237] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:58.657923 10696 raft_consensus.cc:2468] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13c7cc72a0274ec3859962b46603bb12 in term 2.
I20260709 13:05:58.661938 10617 raft_consensus.cc:3060] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.667824 10693 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:58.671824 10617 raft_consensus.cc:2468] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 3.
I20260709 13:05:58.672811 11330 leader_election.cc:304] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:58.674024 11648 raft_consensus.cc:2804] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:58.674494 11648 raft_consensus.cc:697] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:58.675318 11648 consensus_queue.cc:237] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.679471 10369 catalog_manager.cc:5697] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13c7cc72a0274ec3859962b46603bb12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.691226 10373 catalog_manager.cc:5697] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 2 to 3, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.847368 11645 raft_consensus.cc:993] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.848067 11709 raft_consensus.cc:1081] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.849845 11379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:05:58.850370 11379 raft_consensus.cc:493] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.850790 11379 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.858091 11379 raft_consensus.cc:515] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.859714 11379 leader_election.cc:290] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:05:58.860698 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:58.861208 10693 raft_consensus.cc:3055] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.861403 10693 raft_consensus.cc:740] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:58.861919 10693 consensus_queue.cc:260] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.862771 10693 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.864796 10616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71b4eb28f74c3db730751a1aaca656" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:58.865401 10616 raft_consensus.cc:3060] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.869946 10693 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:58.871327 11329 leader_election.cc:304] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:05:58.872107 10616 raft_consensus.cc:2468] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:58.872499 11638 raft_consensus.cc:2804] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.873107 11638 raft_consensus.cc:697] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:58.874048 11638 consensus_queue.cc:237] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:05:58.885910 10373 catalog_manager.cc:5697] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.922230 11645 raft_consensus.cc:993] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.922766 11709 raft_consensus.cc:1081] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:05:58.924485 11374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:05:58.924949 11374 raft_consensus.cc:493] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.925282 11374 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.931248 11374 raft_consensus.cc:515] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:58.933171 11374 leader_election.cc:290] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:05:58.933823 10693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:05:58.934068 10546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf7697bdcf9f49ed9129a8fd785b67be" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:58.934571 10546 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.934453 10693 raft_consensus.cc:3055] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.938081 10693 raft_consensus.cc:740] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:58.938918 10693 consensus_queue.cc:260] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:58.940136 10693 raft_consensus.cc:3060] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.941697 10546 raft_consensus.cc:2468] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:58.942919 11331 leader_election.cc:304] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:58.943668 11684 raft_consensus.cc:2804] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.944171 11684 raft_consensus.cc:697] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:05:58.945041 11684 consensus_queue.cc:237] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:05:58.946666 10693 raft_consensus.cc:2468] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:05:58.955381 10373 catalog_manager.cc:5697] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.169718 10696 raft_consensus.cc:1275] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.170953 11676 consensus_queue.cc:1048] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.186815 10548 raft_consensus.cc:1275] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13c7cc72a0274ec3859962b46603bb12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.188306 11676 consensus_queue.cc:1048] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.269369 10696 raft_consensus.cc:1275] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:59.270816 11684 consensus_queue.cc:1048] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:59.284446 10616 raft_consensus.cc:1275] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:59.289716 11684 consensus_queue.cc:1048] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:59.413641 10617 raft_consensus.cc:1275] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.414855 11684 consensus_queue.cc:1048] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.435822 10696 raft_consensus.cc:1275] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.437287 11707 consensus_queue.cc:1048] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.488446 10546 raft_consensus.cc:1275] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.489926 11707 consensus_queue.cc:1048] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.499691 10696 raft_consensus.cc:1275] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.501420 11684 consensus_queue.cc:1048] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.574810   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.618495   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.653996 10617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72"
dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
mode: GRACEFUL
new_leader_uuid: "13ee73323c814e939932f8e5db34fc96"
 from {username='slave'} at 127.0.0.1:37238
I20260709 13:05:59.654616 10617 raft_consensus.cc:606] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Received request to transfer leadership to TS 13ee73323c814e939932f8e5db34fc96
I20260709 13:05:59.655915   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:05:59.657059   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:59.845816 11644 raft_consensus.cc:993] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: : Instructing follower 13ee73323c814e939932f8e5db34fc96 to start an election
I20260709 13:05:59.846366 11676 raft_consensus.cc:1081] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Signalling peer 13ee73323c814e939932f8e5db34fc96 to start an election
I20260709 13:05:59.847949 10692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
 from {username='slave'} at 127.0.0.1:34664
I20260709 13:05:59.848490 10692 raft_consensus.cc:493] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.848783 10692 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.853669 10692 raft_consensus.cc:515] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:59.855690 10692 leader_election.cc:290] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Requested vote from peers e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713), 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903)
I20260709 13:05:59.856321 10548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e730c6ad44924c0ab2a30ccec893dcd3"
I20260709 13:05:59.856812 10548 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.857390 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1733523e994002a8df5fa4d29d8a72" candidate_uuid: "13ee73323c814e939932f8e5db34fc96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:05:59.857961 10617 raft_consensus.cc:3055] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.858251 10617 raft_consensus.cc:740] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13c7cc72a0274ec3859962b46603bb12, State: Running, Role: LEADER
I20260709 13:05:59.858832 10617 consensus_queue.cc:260] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:59.859795 10617 raft_consensus.cc:3060] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.861945 10548 raft_consensus.cc:2468] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:59.862995 10664 leader_election.cc:304] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96, e730c6ad44924c0ab2a30ccec893dcd3; no voters: 
I20260709 13:05:59.863647 11645 raft_consensus.cc:2804] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.864163 11645 raft_consensus.cc:697] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Becoming Leader. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:05:59.864897 11645 consensus_queue.cc:237] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } }
I20260709 13:05:59.865361 10617 raft_consensus.cc:2468] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ee73323c814e939932f8e5db34fc96 in term 2.
I20260709 13:05:59.874569 10373 catalog_manager.cc:5697] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 reported cstate change: term changed from 1 to 2, leader changed from 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195) to 13ee73323c814e939932f8e5db34fc96 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "13ee73323c814e939932f8e5db34fc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.416567 10546 raft_consensus.cc:1275] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.417682 11709 consensus_queue.cc:1048] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:00.428748 10617 raft_consensus.cc:1275] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 13ee73323c814e939932f8e5db34fc96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.431267 11709 consensus_queue.cc:1048] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:06:00.658006   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:00.699862   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:00.740093 10693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135"
dest_uuid: "13ee73323c814e939932f8e5db34fc96"
mode: GRACEFUL
new_leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:50968
I20260709 13:06:00.740758 10693 raft_consensus.cc:606] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Received request to transfer leadership to TS 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:06:00.742130   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:06:00.743677   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:00.896575 11738 raft_consensus.cc:993] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: : Instructing follower 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:06:00.897104 11739 raft_consensus.cc:1081] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Signalling peer 6a4b3558672241d2969ebf2c86f6d84e to start an election
I20260709 13:06:00.898479 11374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135"
dest_uuid: "6a4b3558672241d2969ebf2c86f6d84e"
 from {username='slave'} at 127.0.0.1:53034
I20260709 13:06:00.898881 11374 raft_consensus.cc:493] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:00.899111 11374 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:00.903736 11374 raft_consensus.cc:515] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:06:00.905367 11374 leader_election.cc:290] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Requested vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 13ee73323c814e939932f8e5db34fc96 (127.0.107.196:40563)
I20260709 13:06:00.905907 10617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13c7cc72a0274ec3859962b46603bb12"
I20260709 13:06:00.906353 10692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c33cae280ab4407bf13087a4521d135" candidate_uuid: "6a4b3558672241d2969ebf2c86f6d84e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13ee73323c814e939932f8e5db34fc96"
I20260709 13:06:00.906584 10617 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:00.906966 10692 raft_consensus.cc:3055] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:00.907287 10692 raft_consensus.cc:740] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13ee73323c814e939932f8e5db34fc96, State: Running, Role: LEADER
I20260709 13:06:00.908088 10692 consensus_queue.cc:260] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:06:00.909338 10692 raft_consensus.cc:3060] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:00.913113 10617 raft_consensus.cc:2468] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:06:00.914232 11330 leader_election.cc:304] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e; no voters: 
I20260709 13:06:00.914984 11684 raft_consensus.cc:2804] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:00.915405 11684 raft_consensus.cc:697] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Becoming Leader. State: Replica: 6a4b3558672241d2969ebf2c86f6d84e, State: Running, Role: LEADER
I20260709 13:06:00.915741 10692 raft_consensus.cc:2468] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4b3558672241d2969ebf2c86f6d84e in term 2.
I20260709 13:06:00.916265 11684 consensus_queue.cc:237] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:06:00.925796 10373 catalog_manager.cc:5697] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e reported cstate change: term changed from 1 to 2, leader changed from 13ee73323c814e939932f8e5db34fc96 (127.0.107.196) to 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6a4b3558672241d2969ebf2c86f6d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.366459 10693 raft_consensus.cc:1275] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.368134 11638 consensus_queue.cc:1048] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:01.379428 10616 raft_consensus.cc:1275] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Refusing update from remote peer 6a4b3558672241d2969ebf2c86f6d84e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.382376 11638 consensus_queue.cc:1048] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:01.744596   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:01.782123   431 tablet_server.cc:179] TabletServer@127.0.107.193:42377 shutting down...
W20260709 13:06:01.787729 10516 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 49 similar messages]
W20260709 13:06:01.792300 10516 consensus_peers.cc:597] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.800732 10664 consensus_peers.cc:597] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.825706 10590 consensus_peers.cc:597] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.829385 10590 consensus_peers.cc:597] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.834574 10664 consensus_peers.cc:597] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.838152 10664 consensus_peers.cc:597] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.841380   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:01.842593   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.843858   431 raft_consensus.cc:2243] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:01.843914 10516 consensus_peers.cc:597] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.845090   431 raft_consensus.cc:2272] T 1c33cae280ab4407bf13087a4521d135 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.848627   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.849622   431 raft_consensus.cc:2243] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.850570   431 raft_consensus.cc:2272] T 68e9d448f8854ecd81d429b12890f07a P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.853168   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.853682   431 raft_consensus.cc:2243] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.854327   431 raft_consensus.cc:2272] T 00f0aa6cf57a4d228d8214c7dca07866 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.856312   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.856928   431 raft_consensus.cc:2243] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.857367   431 raft_consensus.cc:2272] T 9f8fb2ab1f70442786c46c19d8323095 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.859633   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.860198   431 raft_consensus.cc:2243] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.860602   431 raft_consensus.cc:2272] T 519a41f028694f6693c6c85038c54db0 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.862816   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.863487   431 raft_consensus.cc:2243] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.864418   431 raft_consensus.cc:2272] T 42d4f443226d4b09b41d1b414448b007 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.867312   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.868232   431 raft_consensus.cc:2243] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.868767   431 raft_consensus.cc:2272] T 64cae8b9e6b645c59124d6728f4ecb65 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.871385   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.872032   431 raft_consensus.cc:2243] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.872761   431 raft_consensus.cc:2272] T 8cdbdcb35eeb430e824a441d12445254 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.874989   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.875694   431 raft_consensus.cc:2243] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.876339   431 raft_consensus.cc:2272] T 7fcc014601624e27bddc57bc6aadb67d P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:01.879063 10664 consensus_peers.cc:597] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.879307   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.880074   431 raft_consensus.cc:2243] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.880486   431 raft_consensus.cc:2272] T 225f2db0ded7415b81e387822b6e3856 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.882524   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.883088   431 raft_consensus.cc:2243] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.883464   431 raft_consensus.cc:2272] T 08fdd13c2ece4a958ada223d6e631c9f P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.885475   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.886008   431 raft_consensus.cc:2243] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.886389   431 raft_consensus.cc:2272] T 027997962a9c49a19a76790e5f7ead90 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.888680   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.889389   431 raft_consensus.cc:2243] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.890067   431 raft_consensus.cc:2272] T 828be85a64dc4ceaaa111481681d3550 P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.891927   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.892495   431 raft_consensus.cc:2243] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.892918   431 raft_consensus.cc:2272] T 265c12d44bf04deb9f52c9bdb82cc88d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.894801   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.895325   431 raft_consensus.cc:2243] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.896102   431 raft_consensus.cc:2272] T 559deb5d3dd44d6f8cd2e98f479b9168 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.898239   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.898880   431 raft_consensus.cc:2243] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.899356   431 raft_consensus.cc:2272] T 1bf22c286c6747518bafda752540a96c P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:01.900578 10664 consensus_peers.cc:597] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.901804   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.902443   431 raft_consensus.cc:2243] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.902909   431 raft_consensus.cc:2272] T 869dce63600848149f571b6bd169206a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.904960   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.905721   431 raft_consensus.cc:2243] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.906154   431 raft_consensus.cc:2272] T 8f11079f7def4cef8f18ca580808c5d5 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.908457   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:01.908905 10516 consensus_peers.cc:597] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.909228   431 raft_consensus.cc:2243] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.909746   431 raft_consensus.cc:2272] T a04a676a3431425a91e63cd315e899fe P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.911727   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.912351   431 raft_consensus.cc:2243] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.913079   431 raft_consensus.cc:2272] T cf7697bdcf9f49ed9129a8fd785b67be P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.914872   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.915416   431 raft_consensus.cc:2243] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.915876   431 raft_consensus.cc:2272] T 1ebe12bfeac04b27ba5070e402b0307a P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.917726   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.918280   431 raft_consensus.cc:2243] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.918659   431 raft_consensus.cc:2272] T 6d59609def584fcca66488c4e7073630 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.920783   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.921340   431 raft_consensus.cc:2243] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.922060   431 raft_consensus.cc:2272] T e98054391f4644c9a7c7ccf6a6e2db80 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.924062   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.924638   431 raft_consensus.cc:2243] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.925045   431 raft_consensus.cc:2272] T 7e79f3f50a72438383af31b858f986f5 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.927067   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.927641   431 raft_consensus.cc:2243] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.928143   431 raft_consensus.cc:2272] T 44c96c4f1f68468a8302df29e4fa399f P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.930146   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.930711   431 raft_consensus.cc:2243] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.931216   431 raft_consensus.cc:2272] T e9a9e5b696054a01b8aec158182b544d P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.933295   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.933835   431 raft_consensus.cc:2243] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.934259   431 raft_consensus.cc:2272] T ffc7391985ae44a38ba1f2fc35104eca P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.936640   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.937170   431 raft_consensus.cc:2243] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.937736   431 raft_consensus.cc:2272] T 02a164c68f31403ab1ce21952d49ed2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.940794   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.941550   431 raft_consensus.cc:2243] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.942070   431 raft_consensus.cc:2272] T 4e6d5efd76a54d0892c288f4f50190ae P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.945071   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.945887   431 raft_consensus.cc:2243] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:01.946137 10590 consensus_peers.cc:597] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.946439   431 raft_consensus.cc:2272] T 159d5a82c8664fc79597d7f1715bf80e P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:01.948521 10590 consensus_peers.cc:597] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.949311   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.949936   431 raft_consensus.cc:2243] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.950615   431 raft_consensus.cc:2272] T b08b155b1b294e4ea4a4b29755b98651 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.952625   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.953171   431 raft_consensus.cc:2243] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.953567   431 raft_consensus.cc:2272] T 09591a419b7a4edf9163e27b3861dda6 P 6a4b3558672241d2969ebf2c86f6d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.955528   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.956115   431 raft_consensus.cc:2243] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.956549   431 raft_consensus.cc:2272] T 611126921768423c97934867e0e9cb44 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.958448   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.959038   431 raft_consensus.cc:2243] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.959494   431 raft_consensus.cc:2272] T 9bf69fec47094c99a317be303e0def71 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.962299   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.963128   431 raft_consensus.cc:2243] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.963654   431 raft_consensus.cc:2272] T 63cd6402a88541f794b90126ed48b800 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.966418   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:01.966483 10516 consensus_peers.cc:597] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.967293   431 raft_consensus.cc:2243] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:01.967954 10664 consensus_peers.cc:597] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.968423   431 raft_consensus.cc:2272] T b9dd264c867c4296a9a2f7fd53ac51e4 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.970654   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.971300   431 raft_consensus.cc:2243] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.972079   431 raft_consensus.cc:2272] T 8d71b4eb28f74c3db730751a1aaca656 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.974216   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.974694   431 raft_consensus.cc:2243] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.975067   431 raft_consensus.cc:2272] T e01b6c33d7014320a6ee51d8f1c731cb P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.977186   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.977891   431 raft_consensus.cc:2243] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.978786   431 raft_consensus.cc:2272] T 708dee8c708a4b6e91bafafed7d61c2e P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:01.979707 10516 consensus_peers.cc:597] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.981575   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:01.982095 10516 consensus_peers.cc:597] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:01.982229   431 raft_consensus.cc:2243] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:01.983295   431 raft_consensus.cc:2272] T 7dc077c5bba0428d9350f538bd2aeeda P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.985582   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.986130   431 raft_consensus.cc:2243] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.986575   431 raft_consensus.cc:2272] T bbad48f22c2344388089817bc703b04a P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.988956   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.989528   431 raft_consensus.cc:2243] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.990046   431 raft_consensus.cc:2272] T fd44cb78e12145edbcd38d9ead51f9b9 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.992489   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.992985   431 raft_consensus.cc:2243] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.993397   431 raft_consensus.cc:2272] T 2352e31e46a8437ab6897bfbe52d7f9b P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.995365   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.995939   431 raft_consensus.cc:2243] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.996312   431 raft_consensus.cc:2272] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 6a4b3558672241d2969ebf2c86f6d84e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.998067   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:01.998569   431 raft_consensus.cc:2243] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.998986   431 raft_consensus.cc:2272] T 28c8ad15cc9f46928e9304f151ccc0af P 6a4b3558672241d2969ebf2c86f6d84e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.002362 10664 consensus_peers.cc:597] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.033949 10664 consensus_peers.cc:597] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.035254 10664 consensus_peers.cc:597] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.048012 10516 consensus_peers.cc:597] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.050379 10590 consensus_peers.cc:597] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.063871 10590 consensus_peers.cc:597] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.072708 10590 consensus_peers.cc:597] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.074101   431 tablet_server.cc:196] TabletServer@127.0.107.193:42377 shutdown complete.
W20260709 13:06:02.104993 10590 consensus_peers.cc:597] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.148324 10516 consensus_peers.cc:597] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.148486 10590 consensus_peers.cc:597] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.161312   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:02.194823 10516 consensus_peers.cc:597] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.203146 10516 consensus_peers.cc:597] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.203598 10590 consensus_peers.cc:597] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.215301 10664 consensus_peers.cc:597] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.218652   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:02.219589   431 tablet_replica.cc:333] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.220359   431 raft_consensus.cc:2243] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.221202   431 raft_consensus.cc:2272] T 02a164c68f31403ab1ce21952d49ed2e P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.223434   431 tablet_replica.cc:333] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.224112   431 raft_consensus.cc:2243] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.224565   431 raft_consensus.cc:2272] T cf7697bdcf9f49ed9129a8fd785b67be P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.227185   431 tablet_replica.cc:333] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.228134   431 raft_consensus.cc:2243] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.229830   431 raft_consensus.cc:2272] T c6a0a4ae847a4b94b968e58c01a524c2 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.231851 10590 consensus_peers.cc:597] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.232414   431 tablet_replica.cc:333] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.233098   431 raft_consensus.cc:2243] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.233616   431 raft_consensus.cc:2272] T 0acce5611c5e4ad4be2fedf0b6f296ea P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.235962   431 tablet_replica.cc:333] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.236613   431 raft_consensus.cc:2243] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.237367   431 raft_consensus.cc:2272] T 869dce63600848149f571b6bd169206a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.239256   431 tablet_replica.cc:333] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.239944   431 raft_consensus.cc:2243] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.240339   431 raft_consensus.cc:2272] T 3353aa19c6e4404fbcbd2f5e810074c5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.242153   431 tablet_replica.cc:333] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.242713   431 raft_consensus.cc:2243] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.243197   431 raft_consensus.cc:2272] T 4e6d5efd76a54d0892c288f4f50190ae P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.245642   431 tablet_replica.cc:333] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.246201   431 raft_consensus.cc:2243] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.247237   431 raft_consensus.cc:2272] T 28c8ad15cc9f46928e9304f151ccc0af P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.250085   431 tablet_replica.cc:333] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.250932   431 raft_consensus.cc:2243] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.251478   431 raft_consensus.cc:2272] T 9f8fb2ab1f70442786c46c19d8323095 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.254091   431 tablet_replica.cc:333] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.254739   431 raft_consensus.cc:2243] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.255749   431 raft_consensus.cc:2272] T ffc7391985ae44a38ba1f2fc35104eca P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.257844   431 tablet_replica.cc:333] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.258343   431 raft_consensus.cc:2243] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.258695   431 raft_consensus.cc:2272] T 6d59609def584fcca66488c4e7073630 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.260579   431 tablet_replica.cc:333] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.261063   431 raft_consensus.cc:2243] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.261510   431 raft_consensus.cc:2272] T a04a676a3431425a91e63cd315e899fe P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.263226   431 tablet_replica.cc:333] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.263906   431 raft_consensus.cc:2243] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.264386   431 raft_consensus.cc:2272] T 3dff2cbddb694bf8a854bf2a1960649d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.266677   431 tablet_replica.cc:333] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
W20260709 13:06:02.267168 10590 consensus_peers.cc:597] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.267436   431 raft_consensus.cc:2243] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.268338   431 raft_consensus.cc:2272] T 7fcc014601624e27bddc57bc6aadb67d P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.270226   431 tablet_replica.cc:333] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.270830   431 raft_consensus.cc:2243] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.271242   431 raft_consensus.cc:2272] T 43aaaf38cfe7411aa64c01c2519884fa P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.273442   431 tablet_replica.cc:333] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.273996   431 raft_consensus.cc:2243] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.274410   431 raft_consensus.cc:2272] T 7dc077c5bba0428d9350f538bd2aeeda P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.276296   431 tablet_replica.cc:333] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.276866   431 raft_consensus.cc:2243] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.277330   431 raft_consensus.cc:2272] T bbad48f22c2344388089817bc703b04a P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.279170   431 tablet_replica.cc:333] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.279703   431 raft_consensus.cc:2243] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.280134   431 raft_consensus.cc:2272] T 828be85a64dc4ceaaa111481681d3550 P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.281860   431 tablet_replica.cc:333] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.282406   431 raft_consensus.cc:2243] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.282866   431 raft_consensus.cc:2272] T 265c12d44bf04deb9f52c9bdb82cc88d P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.284770   431 tablet_replica.cc:333] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.285326   431 raft_consensus.cc:2243] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.285763   431 raft_consensus.cc:2272] T 1c1733523e994002a8df5fa4d29d8a72 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.287518   431 tablet_replica.cc:333] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.288136   431 raft_consensus.cc:2243] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.288579   431 raft_consensus.cc:2272] T 2352e31e46a8437ab6897bfbe52d7f9b P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.290506   431 tablet_replica.cc:333] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.291045   431 raft_consensus.cc:2243] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.291484   431 raft_consensus.cc:2272] T 5fc7b153bdd64ea6823d2ca146d5ab61 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.293522   431 tablet_replica.cc:333] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.294075   431 raft_consensus.cc:2243] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.294495   431 raft_consensus.cc:2272] T 64cae8b9e6b645c59124d6728f4ecb65 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.296667   431 tablet_replica.cc:333] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.297202   431 raft_consensus.cc:2243] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.297624   431 raft_consensus.cc:2272] T 68e9d448f8854ecd81d429b12890f07a P e730c6ad44924c0ab2a30ccec893dcd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.299387   431 tablet_replica.cc:333] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.299948   431 raft_consensus.cc:2243] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.300825   431 raft_consensus.cc:2272] T 4d38aaddc970432bb924dc77ac8bcfa7 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.302858   431 tablet_replica.cc:333] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.303370   431 raft_consensus.cc:2243] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.303910   431 raft_consensus.cc:2272] T 63cd6402a88541f794b90126ed48b800 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.305768 10664 consensus_peers.cc:597] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 -> Peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Couldn't send request to peer 6a4b3558672241d2969ebf2c86f6d84e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.305912   431 tablet_replica.cc:333] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.306553   431 raft_consensus.cc:2243] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.307315   431 raft_consensus.cc:2272] T 027997962a9c49a19a76790e5f7ead90 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.309337   431 tablet_replica.cc:333] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.310045   431 raft_consensus.cc:2243] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.310614   431 raft_consensus.cc:2272] T 7e79f3f50a72438383af31b858f986f5 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.313351   431 tablet_replica.cc:333] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.314121   431 raft_consensus.cc:2243] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.315088   431 raft_consensus.cc:2272] T 9bf69fec47094c99a317be303e0def71 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.316757 10664 consensus_peers.cc:597] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.317754   431 tablet_replica.cc:333] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.318851   431 raft_consensus.cc:2243] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.320211   431 raft_consensus.cc:2272] T 611126921768423c97934867e0e9cb44 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.323678   431 tablet_replica.cc:333] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.324749   431 raft_consensus.cc:2243] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.326079   431 raft_consensus.cc:2272] T b87f245147a44fc0a03f124aa5040d22 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.329396   431 tablet_replica.cc:333] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.330415   431 raft_consensus.cc:2243] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.331782   431 raft_consensus.cc:2272] T 44c96c4f1f68468a8302df29e4fa399f P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.335294   431 tablet_replica.cc:333] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.336071   431 raft_consensus.cc:2243] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:02.336410 10590 consensus_peers.cc:597] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.337204   431 raft_consensus.cc:2272] T 1ebe12bfeac04b27ba5070e402b0307a P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.340657   431 tablet_replica.cc:333] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.341441   431 raft_consensus.cc:2243] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.341976   431 raft_consensus.cc:2272] T 8cdbdcb35eeb430e824a441d12445254 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.344851   431 tablet_replica.cc:333] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.345512   431 raft_consensus.cc:2243] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.346081   431 raft_consensus.cc:2272] T b4b3541d8e1e4c3faefc6d3350706237 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.348263   431 tablet_replica.cc:333] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.348997   431 raft_consensus.cc:2243] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.349485   431 raft_consensus.cc:2272] T 6eefd49b51434d4786968ac53f50899c P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.352205   431 tablet_replica.cc:333] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.352870   431 raft_consensus.cc:2243] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:02.353601 10590 consensus_peers.cc:597] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.353917   431 raft_consensus.cc:2272] T e01b6c33d7014320a6ee51d8f1c731cb P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.356132   431 tablet_replica.cc:333] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.356688   431 raft_consensus.cc:2243] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.357097   431 raft_consensus.cc:2272] T b9dd264c867c4296a9a2f7fd53ac51e4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.359597   431 tablet_replica.cc:333] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.360244   431 raft_consensus.cc:2243] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.361115   431 raft_consensus.cc:2272] T acd69d2652a64ddba95c78f88d6e9a97 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.363413   431 tablet_replica.cc:333] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.364058   431 raft_consensus.cc:2243] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.364502   431 raft_consensus.cc:2272] T d4849594fca843a982888e29f557c3f4 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.366547   431 tablet_replica.cc:333] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.367112   431 raft_consensus.cc:2243] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.367952   431 raft_consensus.cc:2272] T e7b9860a1dbf4923a357199faf592424 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.369735   431 tablet_replica.cc:333] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.370157   431 raft_consensus.cc:2243] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.370507   431 raft_consensus.cc:2272] T e7a759b0b5de444fa1c1486147b76cfa P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.372359   431 tablet_replica.cc:333] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
I20260709 13:06:02.372949   431 raft_consensus.cc:2243] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.373339   431 raft_consensus.cc:2272] T b08b155b1b294e4ea4a4b29755b98651 P e730c6ad44924c0ab2a30ccec893dcd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.376091   431 tablet_replica.cc:333] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3: stopping tablet replica
W20260709 13:06:02.376372 10590 consensus_peers.cc:597] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.376617   431 raft_consensus.cc:2243] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.377163   431 raft_consensus.cc:2272] T 09591a419b7a4edf9163e27b3861dda6 P e730c6ad44924c0ab2a30ccec893dcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.401940 10664 consensus_peers.cc:597] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.413549 10590 consensus_peers.cc:597] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.432341 10664 consensus_peers.cc:597] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.433708 10590 consensus_peers.cc:597] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.459687 10590 consensus_peers.cc:597] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.462631   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:02.501868 10664 consensus_peers.cc:597] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.517022 10664 consensus_peers.cc:597] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.520391 10664 consensus_peers.cc:597] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.534840 10664 consensus_peers.cc:597] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.540439 10664 consensus_peers.cc:597] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.540940 10664 consensus_peers.cc:597] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.552484 10590 consensus_peers.cc:597] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.562186   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:02.570465 10663 consensus_peers.cc:597] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.603317 10663 consensus_peers.cc:597] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.605916   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:02.606678   431 tablet_replica.cc:333] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.607394   431 raft_consensus.cc:2243] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.608479   431 raft_consensus.cc:2272] T 225f2db0ded7415b81e387822b6e3856 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.610467   431 tablet_replica.cc:333] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.611035   431 raft_consensus.cc:2243] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.611747   431 raft_consensus.cc:2272] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.613977   431 tablet_replica.cc:333] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.614795   431 raft_consensus.cc:2243] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.615268   431 raft_consensus.cc:2272] T 42d4f443226d4b09b41d1b414448b007 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.617723   431 tablet_replica.cc:333] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.618441   431 raft_consensus.cc:2243] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.618930   431 raft_consensus.cc:2272] T 28c8ad15cc9f46928e9304f151ccc0af P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.621155   431 tablet_replica.cc:333] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.621852   431 raft_consensus.cc:2243] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.622313   431 raft_consensus.cc:2272] T fd44cb78e12145edbcd38d9ead51f9b9 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.624713   431 tablet_replica.cc:333] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.625243   431 raft_consensus.cc:2243] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.625631   431 raft_consensus.cc:2272] T 00f0aa6cf57a4d228d8214c7dca07866 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.627312   431 tablet_replica.cc:333] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.628130   431 raft_consensus.cc:2243] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.628904   431 raft_consensus.cc:2272] T 9f8fb2ab1f70442786c46c19d8323095 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.630791   431 tablet_replica.cc:333] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.631351   431 raft_consensus.cc:2243] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.631750   431 raft_consensus.cc:2272] T 1c33cae280ab4407bf13087a4521d135 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.633589   431 tablet_replica.cc:333] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.634125   431 raft_consensus.cc:2243] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.635248   431 raft_consensus.cc:2272] T a04a676a3431425a91e63cd315e899fe P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.637063   431 tablet_replica.cc:333] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.637604   431 raft_consensus.cc:2243] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.638084   431 raft_consensus.cc:2272] T 3dff2cbddb694bf8a854bf2a1960649d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.640076   431 tablet_replica.cc:333] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.640743   431 raft_consensus.cc:2243] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.641220   431 raft_consensus.cc:2272] T 7fcc014601624e27bddc57bc6aadb67d P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.645975   431 tablet_replica.cc:333] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.646667   431 raft_consensus.cc:2243] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.647135   431 raft_consensus.cc:2272] T 828be85a64dc4ceaaa111481681d3550 P 13c7cc72a0274ec3859962b46603bb12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.649626   431 tablet_replica.cc:333] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.650276   431 raft_consensus.cc:2243] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.650683   431 raft_consensus.cc:2272] T 1c1733523e994002a8df5fa4d29d8a72 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.652479   431 tablet_replica.cc:333] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.652990   431 raft_consensus.cc:2243] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.653398   431 raft_consensus.cc:2272] T 708dee8c708a4b6e91bafafed7d61c2e P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.655062   431 tablet_replica.cc:333] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.655586   431 raft_consensus.cc:2243] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.656317   431 raft_consensus.cc:2272] T bbad48f22c2344388089817bc703b04a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.658087   431 tablet_replica.cc:333] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.658613   431 raft_consensus.cc:2243] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.659333   431 raft_consensus.cc:2272] T 5fc7b153bdd64ea6823d2ca146d5ab61 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.661252   431 tablet_replica.cc:333] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.661799   431 raft_consensus.cc:2243] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.662539   431 raft_consensus.cc:2272] T 08fdd13c2ece4a958ada223d6e631c9f P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.664448   431 tablet_replica.cc:333] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.665136   431 raft_consensus.cc:2243] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.665570   431 raft_consensus.cc:2272] T 027997962a9c49a19a76790e5f7ead90 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.667367   431 tablet_replica.cc:333] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.667985   431 raft_consensus.cc:2243] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.668808   431 raft_consensus.cc:2272] T 8f11079f7def4cef8f18ca580808c5d5 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.671123   431 tablet_replica.cc:333] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.671818   431 raft_consensus.cc:2243] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.672587   431 raft_consensus.cc:2272] T 43aaaf38cfe7411aa64c01c2519884fa P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.672906 10590 consensus_peers.cc:597] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.674523   431 tablet_replica.cc:333] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.675050   431 raft_consensus.cc:2243] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.675403   431 raft_consensus.cc:2272] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.677242   431 tablet_replica.cc:333] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.677733   431 raft_consensus.cc:2243] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.678071   431 raft_consensus.cc:2272] T c6a0a4ae847a4b94b968e58c01a524c2 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.679862   431 tablet_replica.cc:333] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.680316   431 raft_consensus.cc:2243] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.680707   431 raft_consensus.cc:2272] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.682475   431 tablet_replica.cc:333] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.682946   431 raft_consensus.cc:2243] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.683324   431 raft_consensus.cc:2272] T 159d5a82c8664fc79597d7f1715bf80e P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.685289   431 tablet_replica.cc:333] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.685827   431 raft_consensus.cc:2243] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.686548   431 raft_consensus.cc:2272] T 63cd6402a88541f794b90126ed48b800 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.688535   431 tablet_replica.cc:333] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.689061   431 raft_consensus.cc:2243] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.689443   431 raft_consensus.cc:2272] T b9dd264c867c4296a9a2f7fd53ac51e4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.691305   431 tablet_replica.cc:333] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.692018   431 raft_consensus.cc:2243] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.692451   431 raft_consensus.cc:2272] T 8d71b4eb28f74c3db730751a1aaca656 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.694667   431 tablet_replica.cc:333] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.695571   431 raft_consensus.cc:2243] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.695991   431 raft_consensus.cc:2272] T b08b155b1b294e4ea4a4b29755b98651 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.698028   431 tablet_replica.cc:333] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.698606   431 raft_consensus.cc:2243] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.699357   431 raft_consensus.cc:2272] T 09591a419b7a4edf9163e27b3861dda6 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.700651 10664 consensus_peers.cc:597] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 -> Peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Couldn't send request to peer e730c6ad44924c0ab2a30ccec893dcd3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.701596   431 tablet_replica.cc:333] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.702209   431 raft_consensus.cc:2243] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.702929   431 raft_consensus.cc:2272] T e9a9e5b696054a01b8aec158182b544d P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.704864   431 tablet_replica.cc:333] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.705375   431 raft_consensus.cc:2243] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.705761   431 raft_consensus.cc:2272] T e98054391f4644c9a7c7ccf6a6e2db80 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.707386   431 tablet_replica.cc:333] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.707921   431 raft_consensus.cc:2243] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.708582   431 raft_consensus.cc:2272] T b4b3541d8e1e4c3faefc6d3350706237 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.711035   431 tablet_replica.cc:333] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.711544   431 raft_consensus.cc:2243] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.712309   431 raft_consensus.cc:2272] T 6eefd49b51434d4786968ac53f50899c P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.714042   431 tablet_replica.cc:333] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.714493   431 raft_consensus.cc:2243] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.714855   431 raft_consensus.cc:2272] T b87f245147a44fc0a03f124aa5040d22 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.716646   431 tablet_replica.cc:333] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.717092   431 raft_consensus.cc:2243] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.717516   431 raft_consensus.cc:2272] T 611126921768423c97934867e0e9cb44 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.719266   431 tablet_replica.cc:333] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.719717   431 raft_consensus.cc:2243] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.720193   431 raft_consensus.cc:2272] T acd69d2652a64ddba95c78f88d6e9a97 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.722290   431 tablet_replica.cc:333] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.722828   431 raft_consensus.cc:2243] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.723639   431 raft_consensus.cc:2272] T d4849594fca843a982888e29f557c3f4 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.726014   431 tablet_replica.cc:333] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.726573   431 raft_consensus.cc:2243] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.727001   431 raft_consensus.cc:2272] T e7b9860a1dbf4923a357199faf592424 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.727459 10663 consensus_peers.cc:597] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.729019   431 tablet_replica.cc:333] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.729656   431 raft_consensus.cc:2243] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.730058   431 raft_consensus.cc:2272] T e7a759b0b5de444fa1c1486147b76cfa P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.732095   431 tablet_replica.cc:333] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.732594   431 raft_consensus.cc:2243] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.732983   431 raft_consensus.cc:2272] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.734951   431 tablet_replica.cc:333] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.735438   431 raft_consensus.cc:2243] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.735918   431 raft_consensus.cc:2272] T 869dce63600848149f571b6bd169206a P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.737702   431 tablet_replica.cc:333] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.738173   431 raft_consensus.cc:2243] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.738560   431 raft_consensus.cc:2272] T 1bf22c286c6747518bafda752540a96c P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.740684   431 tablet_replica.cc:333] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.741151   431 raft_consensus.cc:2243] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.741580   431 raft_consensus.cc:2272] T 44c96c4f1f68468a8302df29e4fa399f P 13c7cc72a0274ec3859962b46603bb12 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.743006 10663 consensus_peers.cc:597] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.743582   431 tablet_replica.cc:333] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12: stopping tablet replica
I20260709 13:06:02.744168   431 raft_consensus.cc:2243] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.744593   431 raft_consensus.cc:2272] T 519a41f028694f6693c6c85038c54db0 P 13c7cc72a0274ec3859962b46603bb12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.828380 10663 consensus_peers.cc:597] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:36903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.830662   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:02.877352 11767 raft_consensus.cc:493] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:06:02.878154 11767 raft_consensus.cc:515] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:06:02.880542 11767 leader_election.cc:290] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
W20260709 13:06:02.884392 10663 leader_election.cc:336] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Network error: Client connection negotiation failed: client connection to 127.0.107.195:36903: connect: Connection refused (error 111)
W20260709 13:06:02.885260 10664 leader_election.cc:336] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111)
I20260709 13:06:02.885620 10664 leader_election.cc:304] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:06:02.886240 11767 raft_consensus.cc:2749] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.901077   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:06:02.939697   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:02.940446   431 tablet_replica.cc:333] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.941085   431 raft_consensus.cc:2243] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.941584   431 raft_consensus.cc:2272] T 225f2db0ded7415b81e387822b6e3856 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.943671   431 tablet_replica.cc:333] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.944320   431 raft_consensus.cc:2243] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.944792   431 raft_consensus.cc:2272] T 02a164c68f31403ab1ce21952d49ed2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.946767   431 tablet_replica.cc:333] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.947335   431 raft_consensus.cc:2243] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.947803   431 raft_consensus.cc:2272] T cf7697bdcf9f49ed9129a8fd785b67be P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.949872   431 tablet_replica.cc:333] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.950486   431 raft_consensus.cc:2243] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.950896   431 raft_consensus.cc:2272] T 42d4f443226d4b09b41d1b414448b007 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.952757   431 tablet_replica.cc:333] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.953285   431 raft_consensus.cc:2243] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.953683   431 raft_consensus.cc:2272] T 3353aa19c6e4404fbcbd2f5e810074c5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.955739   431 tablet_replica.cc:333] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.956393   431 raft_consensus.cc:2243] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.957317   431 raft_consensus.cc:2272] T 4e6d5efd76a54d0892c288f4f50190ae P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.957976 11767 raft_consensus.cc:493] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:06:02.958509 11767 raft_consensus.cc:515] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c7cc72a0274ec3859962b46603bb12" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36903 } } peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } }
I20260709 13:06:02.960060   431 tablet_replica.cc:333] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.960641 11767 leader_election.cc:290] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903), 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377)
I20260709 13:06:02.960959   431 raft_consensus.cc:2243] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.961580   431 raft_consensus.cc:2272] T 00f0aa6cf57a4d228d8214c7dca07866 P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:02.963339 10663 consensus_peers.cc:597] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 -> Peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Couldn't send request to peer 13c7cc72a0274ec3859962b46603bb12. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:36903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:02.963817 10663 leader_election.cc:336] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 13c7cc72a0274ec3859962b46603bb12 (127.0.107.195:36903): Network error: Client connection negotiation failed: client connection to 127.0.107.195:36903: connect: Connection refused (error 111)
I20260709 13:06:02.964202   431 tablet_replica.cc:333] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
W20260709 13:06:02.964713 10664 leader_election.cc:336] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111)
I20260709 13:06:02.965022   431 raft_consensus.cc:2243] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.965117 10664 leader_election.cc:304] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 13c7cc72a0274ec3859962b46603bb12, 6a4b3558672241d2969ebf2c86f6d84e
I20260709 13:06:02.965806 11767 raft_consensus.cc:2749] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.966171   431 raft_consensus.cc:2272] T fd44cb78e12145edbcd38d9ead51f9b9 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.968369   431 tablet_replica.cc:333] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.968873   431 raft_consensus.cc:2243] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.969224   431 raft_consensus.cc:2272] T 1c33cae280ab4407bf13087a4521d135 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.970993   431 tablet_replica.cc:333] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.971486   431 raft_consensus.cc:2243] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.971920   431 raft_consensus.cc:2272] T 43aaaf38cfe7411aa64c01c2519884fa P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.973641   431 tablet_replica.cc:333] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.974119   431 raft_consensus.cc:2243] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.974818   431 raft_consensus.cc:2272] T 3dff2cbddb694bf8a854bf2a1960649d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.977768   431 tablet_replica.cc:333] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.978255   431 raft_consensus.cc:2243] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.978927   431 raft_consensus.cc:2272] T 6d59609def584fcca66488c4e7073630 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.981178   431 tablet_replica.cc:333] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.981981   431 raft_consensus.cc:2243] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.982913   431 raft_consensus.cc:2272] T 265c12d44bf04deb9f52c9bdb82cc88d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.985309   431 tablet_replica.cc:333] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.985841   431 raft_consensus.cc:2243] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:02.986567   431 raft_consensus.cc:2272] T 1c1733523e994002a8df5fa4d29d8a72 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.988447   431 tablet_replica.cc:333] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.989009   431 raft_consensus.cc:2243] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.989408   431 raft_consensus.cc:2272] T 8f11079f7def4cef8f18ca580808c5d5 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.991119   431 tablet_replica.cc:333] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.991638   431 raft_consensus.cc:2243] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.992089   431 raft_consensus.cc:2272] T 68e9d448f8854ecd81d429b12890f07a P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.993813   431 tablet_replica.cc:333] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.994320   431 raft_consensus.cc:2243] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.994716   431 raft_consensus.cc:2272] T 4d38aaddc970432bb924dc77ac8bcfa7 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.996613   431 tablet_replica.cc:333] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.997131   431 raft_consensus.cc:2243] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:02.997536   431 raft_consensus.cc:2272] T 6eefd49b51434d4786968ac53f50899c P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:02.999281   431 tablet_replica.cc:333] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:02.999850   431 raft_consensus.cc:2243] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.000579   431 raft_consensus.cc:2272] T 2352e31e46a8437ab6897bfbe52d7f9b P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.002300   431 tablet_replica.cc:333] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.002804   431 raft_consensus.cc:2243] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.003193   431 raft_consensus.cc:2272] T 08fdd13c2ece4a958ada223d6e631c9f P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.004948   431 tablet_replica.cc:333] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.005461   431 raft_consensus.cc:2243] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.006157   431 raft_consensus.cc:2272] T 64cae8b9e6b645c59124d6728f4ecb65 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.007962   431 tablet_replica.cc:333] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.008478   431 raft_consensus.cc:2243] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.009219   431 raft_consensus.cc:2272] T c6a0a4ae847a4b94b968e58c01a524c2 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.010982   431 tablet_replica.cc:333] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.011448   431 raft_consensus.cc:2243] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.012178   431 raft_consensus.cc:2272] T 0acce5611c5e4ad4be2fedf0b6f296ea P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.013988   431 tablet_replica.cc:333] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.014470   431 raft_consensus.cc:2243] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.015128   431 raft_consensus.cc:2272] T 159d5a82c8664fc79597d7f1715bf80e P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.016952   431 tablet_replica.cc:333] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.017417   431 raft_consensus.cc:2243] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.017774   431 raft_consensus.cc:2272] T e01b6c33d7014320a6ee51d8f1c731cb P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.019492   431 tablet_replica.cc:333] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.020031   431 raft_consensus.cc:2243] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.020408   431 raft_consensus.cc:2272] T 8d71b4eb28f74c3db730751a1aaca656 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.022125   431 tablet_replica.cc:333] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.022739   431 raft_consensus.cc:2243] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.023128   431 raft_consensus.cc:2272] T acd69d2652a64ddba95c78f88d6e9a97 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.023377 11767 raft_consensus.cc:493] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a4b3558672241d2969ebf2c86f6d84e)
I20260709 13:06:03.023954 11767 raft_consensus.cc:515] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ee73323c814e939932f8e5db34fc96" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40563 } } peers { permanent_uuid: "6a4b3558672241d2969ebf2c86f6d84e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42377 } } peers { permanent_uuid: "e730c6ad44924c0ab2a30ccec893dcd3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46713 } }
I20260709 13:06:03.025460   431 tablet_replica.cc:333] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.026048 11767 leader_election.cc:290] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377), e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713)
I20260709 13:06:03.026305   431 raft_consensus.cc:2243] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.026832   431 raft_consensus.cc:2272] T e7b9860a1dbf4923a357199faf592424 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.029554   431 tablet_replica.cc:333] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.030367   431 raft_consensus.cc:2243] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:03.030470 10664 leader_election.cc:336] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6a4b3558672241d2969ebf2c86f6d84e (127.0.107.193:42377): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42377: connect: Connection refused (error 111)
I20260709 13:06:03.030925   431 raft_consensus.cc:2272] T d4849594fca843a982888e29f557c3f4 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:03.031266 10664 leader_election.cc:336] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e730c6ad44924c0ab2a30ccec893dcd3 (127.0.107.194:46713): Network error: Client connection negotiation failed: client connection to 127.0.107.194:46713: connect: Connection refused (error 111)
I20260709 13:06:03.031677 10664 leader_election.cc:304] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13ee73323c814e939932f8e5db34fc96; no voters: 6a4b3558672241d2969ebf2c86f6d84e, e730c6ad44924c0ab2a30ccec893dcd3
I20260709 13:06:03.032382 11767 raft_consensus.cc:2749] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:03.033414   431 tablet_replica.cc:333] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.034037   431 raft_consensus.cc:2243] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.034838   431 raft_consensus.cc:2272] T e7a759b0b5de444fa1c1486147b76cfa P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.036717   431 tablet_replica.cc:333] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.037256   431 raft_consensus.cc:2243] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.037683   431 raft_consensus.cc:2272] T 559deb5d3dd44d6f8cd2e98f479b9168 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.039342   431 tablet_replica.cc:333] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.039914   431 raft_consensus.cc:2243] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.040665   431 raft_consensus.cc:2272] T 1bf22c286c6747518bafda752540a96c P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.042488   431 tablet_replica.cc:333] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.043025   431 raft_consensus.cc:2243] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.043751   431 raft_consensus.cc:2272] T 7e79f3f50a72438383af31b858f986f5 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.045647   431 tablet_replica.cc:333] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.046144   431 raft_consensus.cc:2243] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.046532   431 raft_consensus.cc:2272] T b4b3541d8e1e4c3faefc6d3350706237 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.048372   431 tablet_replica.cc:333] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.048857   431 raft_consensus.cc:2243] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.049250   431 raft_consensus.cc:2272] T e98054391f4644c9a7c7ccf6a6e2db80 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.050956   431 tablet_replica.cc:333] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.051426   431 raft_consensus.cc:2243] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.051872   431 raft_consensus.cc:2272] T 9bf69fec47094c99a317be303e0def71 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.053679   431 tablet_replica.cc:333] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.054165   431 raft_consensus.cc:2243] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.054549   431 raft_consensus.cc:2272] T b87f245147a44fc0a03f124aa5040d22 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.056305   431 tablet_replica.cc:333] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.056789   431 raft_consensus.cc:2243] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.057479   431 raft_consensus.cc:2272] T 519a41f028694f6693c6c85038c54db0 P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.059198   431 tablet_replica.cc:333] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.059703   431 raft_consensus.cc:2243] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.060132   431 raft_consensus.cc:2272] T 8cdbdcb35eeb430e824a441d12445254 P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.061815   431 tablet_replica.cc:333] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.062295   431 raft_consensus.cc:2243] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.062691   431 raft_consensus.cc:2272] T 1ebe12bfeac04b27ba5070e402b0307a P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.064416   431 tablet_replica.cc:333] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.064886   431 raft_consensus.cc:2243] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.065297   431 raft_consensus.cc:2272] T ffc7391985ae44a38ba1f2fc35104eca P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.067126   431 tablet_replica.cc:333] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.067598   431 raft_consensus.cc:2243] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.068019   431 raft_consensus.cc:2272] T e9a9e5b696054a01b8aec158182b544d P 13ee73323c814e939932f8e5db34fc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.069778   431 tablet_replica.cc:333] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.070240   431 raft_consensus.cc:2243] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.070633   431 raft_consensus.cc:2272] T 708dee8c708a4b6e91bafafed7d61c2e P 13ee73323c814e939932f8e5db34fc96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.072460   431 tablet_replica.cc:333] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96: stopping tablet replica
I20260709 13:06:03.072913   431 raft_consensus.cc:2243] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.073307   431 raft_consensus.cc:2272] T 7dc077c5bba0428d9350f538bd2aeeda P 13ee73323c814e939932f8e5db34fc96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.137907   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:06:03.200639   431 master.cc:562] Master@127.0.107.254:36559 shutting down...
I20260709 13:06:03.227880   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.228464   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.228871   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 254dc4ec51764d408dae2397b5a8bb02: stopping tablet replica
I20260709 13:06:03.252660   431 master.cc:584] Master@127.0.107.254:36559 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31445 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:06:03.341203   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:40129
I20260709 13:06:03.342422   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.348403 11773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.348512 11772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.349588 11775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.351941   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.353210   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.353396   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.353606   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363353564 us; error 0 us; skew 500 ppm
I20260709 13:06:03.354138   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.357147   431 webserver.cc:533] Webserver started at http://127.0.107.254:43815/ using document root <none> and password file <none>
I20260709 13:06:03.357654   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.357836   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.358126   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.359266   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "b55d41aff28c4895a9f2412bd11d573c"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.364054   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:03.368045 11780 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.368932   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:03.369277   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "b55d41aff28c4895a9f2412bd11d573c"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.369591   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.382575   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.383942   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.422798   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:40129
I20260709 13:06:03.422912 11831 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:40129 every 8 connection(s)
I20260709 13:06:03.426887 11832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.437955 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c: Bootstrap starting.
I20260709 13:06:03.443029 11832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:03.447651 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c: No bootstrap required, opened a new log
I20260709 13:06:03.449990 11832 raft_consensus.cc:359] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER }
I20260709 13:06:03.450397 11832 raft_consensus.cc:385] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:03.450670 11832 raft_consensus.cc:740] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b55d41aff28c4895a9f2412bd11d573c, State: Initialized, Role: FOLLOWER
I20260709 13:06:03.451293 11832 consensus_queue.cc:260] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER }
I20260709 13:06:03.451745 11832 raft_consensus.cc:399] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:03.452003 11832 raft_consensus.cc:493] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:03.452275 11832 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.457423 11832 raft_consensus.cc:515] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER }
I20260709 13:06:03.458051 11832 leader_election.cc:304] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b55d41aff28c4895a9f2412bd11d573c; no voters: 
I20260709 13:06:03.459344 11832 leader_election.cc:290] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:03.459836 11835 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:03.461349 11835 raft_consensus.cc:697] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 1 LEADER]: Becoming Leader. State: Replica: b55d41aff28c4895a9f2412bd11d573c, State: Running, Role: LEADER
I20260709 13:06:03.462080 11835 consensus_queue.cc:237] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER }
I20260709 13:06:03.462390 11832 sys_catalog.cc:565] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:03.467054 11836 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b55d41aff28c4895a9f2412bd11d573c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER } }
I20260709 13:06:03.468006 11836 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:03.468101 11837 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b55d41aff28c4895a9f2412bd11d573c. Latest consensus state: current_term: 1 leader_uuid: "b55d41aff28c4895a9f2412bd11d573c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55d41aff28c4895a9f2412bd11d573c" member_type: VOTER } }
I20260709 13:06:03.469087 11837 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:03.472759 11842 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:03.481637 11842 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:03.485898   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:03.492833 11842 catalog_manager.cc:1383] Generated new cluster ID: 0376edfb059b402fa3e91ef174c70a41
I20260709 13:06:03.493147 11842 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:03.517283 11842 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:03.518666 11842 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:03.532339 11842 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c: Generated new TSK 0
I20260709 13:06:03.532959 11842 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:03.552812   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.559062 11853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.560354 11854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.563476 11856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.563584   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.565011   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.565197   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.565315   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363565304 us; error 0 us; skew 500 ppm
I20260709 13:06:03.565815   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.568809   431 webserver.cc:533] Webserver started at http://127.0.107.193:33797/ using document root <none> and password file <none>
I20260709 13:06:03.569262   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.569422   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.569639   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.570779   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.575493   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:06:03.578807 11861 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.579665   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:03.580054   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.580348   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.605168   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.606415   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.608078   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.610467   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.610707   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.610965   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.611162   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.653101   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43167
I20260709 13:06:03.653196 11923 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43167 every 8 connection(s)
I20260709 13:06:03.658144   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.666131 11928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.666712 11929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.669617 11931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.669756   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.672465   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.672708   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.672923   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363672893 us; error 0 us; skew 500 ppm
I20260709 13:06:03.673594   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.675046 11924 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40129
I20260709 13:06:03.675414 11924 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:03.676198 11924 heartbeater.cc:507] Master 127.0.107.254:40129 requested a full tablet report, sending...
I20260709 13:06:03.676694   431 webserver.cc:533] Webserver started at http://127.0.107.194:43791/ using document root <none> and password file <none>
I20260709 13:06:03.677389   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.677615   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.677865   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.678372 11797 ts_manager.cc:194] Registered new tserver with Master: d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167)
I20260709 13:06:03.678992   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "e01f3b6608004bd881304eecb689b308"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.680279 11797 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46806
I20260709 13:06:03.684075   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:06:03.687534 11936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.688436   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:03.688705   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "e01f3b6608004bd881304eecb689b308"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.688982   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.715149   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.716377   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.717789   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.720063   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.720283   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.720525   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.720695   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.760152   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:35569
I20260709 13:06:03.760244 11998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:35569 every 8 connection(s)
I20260709 13:06:03.765164   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.772254 12002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.773756 12003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.776178   431 server_base.cc:1061] running on GCE node
W20260709 13:06:03.776257 12005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.777474   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:03.777711 11999 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40129
W20260709 13:06:03.777773   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.778175   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363778155 us; error 0 us; skew 500 ppm
I20260709 13:06:03.778209 11999 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:03.778818   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.779079 11999 heartbeater.cc:507] Master 127.0.107.254:40129 requested a full tablet report, sending...
I20260709 13:06:03.781428   431 webserver.cc:533] Webserver started at http://127.0.107.195:43621/ using document root <none> and password file <none>
I20260709 13:06:03.781585 11797 ts_manager.cc:194] Registered new tserver with Master: e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:03.782285   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.782560   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.782878   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.783442 11797 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46814
I20260709 13:06:03.784461   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "3afb90c914b64383b7fa6d5ec19950a7"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.789402   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:03.792704 12010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.793459   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:03.793711   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "3afb90c914b64383b7fa6d5ec19950a7"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:03.793965   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.812592   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.813719   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.815138   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.817512   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.817698   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.817997   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.818163   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.858212   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:42237
I20260709 13:06:03.858311 12072 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:42237 every 8 connection(s)
I20260709 13:06:03.872792 12073 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40129
I20260709 13:06:03.873217 12073 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:03.873965 12073 heartbeater.cc:507] Master 127.0.107.254:40129 requested a full tablet report, sending...
I20260709 13:06:03.875895 11797 ts_manager.cc:194] Registered new tserver with Master: 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:03.875994   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014790131s
I20260709 13:06:03.877310 11797 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46830
I20260709 13:06:03.901064 11797 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46832:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:03.903914 11797 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:03.977133 11964 tablet_service.cc:1511] Processing CreateTablet for tablet 461f7e8beb6d4f3d9587184658c8e8fb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:03.978679 11964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461f7e8beb6d4f3d9587184658c8e8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.979298 11963 tablet_service.cc:1511] Processing CreateTablet for tablet 290f48e0551d4a6fa4f08dede4d79ec7 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:03.980880 11963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290f48e0551d4a6fa4f08dede4d79ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.983400 11962 tablet_service.cc:1511] Processing CreateTablet for tablet e3cd2cdce4c040dab6503b6000324821 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:03.984949 11962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cd2cdce4c040dab6503b6000324821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.994099 11961 tablet_service.cc:1511] Processing CreateTablet for tablet 36215ec218514de9b8449e9461a16c49 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:03.995589 11961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36215ec218514de9b8449e9461a16c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.997110 11959 tablet_service.cc:1511] Processing CreateTablet for tablet 0dad18b855bc43529c37d812ced3ffc8 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:03.998533 11959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dad18b855bc43529c37d812ced3ffc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.000893 11960 tablet_service.cc:1511] Processing CreateTablet for tablet 6c417277bdea4b6ba9c42a3f1fcada34 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:04.001209 11958 tablet_service.cc:1511] Processing CreateTablet for tablet 77fcc90cbf664798be4314a8076dbf1e (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:04.002332 11960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c417277bdea4b6ba9c42a3f1fcada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.003329 11958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fcc90cbf664798be4314a8076dbf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.009498 11957 tablet_service.cc:1511] Processing CreateTablet for tablet e60f33c8f84f4db890e1d5c402de57f3 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:04.011140 11957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60f33c8f84f4db890e1d5c402de57f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.016005 11956 tablet_service.cc:1511] Processing CreateTablet for tablet b5c0dae7a44148d5abbfb963fa6eb387 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:04.017526 11956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c0dae7a44148d5abbfb963fa6eb387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.020972 11955 tablet_service.cc:1511] Processing CreateTablet for tablet 49301cd9cec046a69c616dc8792466bb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:04.022419 11955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49301cd9cec046a69c616dc8792466bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.030673 11888 tablet_service.cc:1511] Processing CreateTablet for tablet 290f48e0551d4a6fa4f08dede4d79ec7 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:04.032295 11883 tablet_service.cc:1511] Processing CreateTablet for tablet 77fcc90cbf664798be4314a8076dbf1e (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:04.033723 11883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fcc90cbf664798be4314a8076dbf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.034615 11886 tablet_service.cc:1511] Processing CreateTablet for tablet 36215ec218514de9b8449e9461a16c49 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:04.032837 11889 tablet_service.cc:1511] Processing CreateTablet for tablet 461f7e8beb6d4f3d9587184658c8e8fb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:04.037544 11885 tablet_service.cc:1511] Processing CreateTablet for tablet 6c417277bdea4b6ba9c42a3f1fcada34 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:04.038166 11884 tablet_service.cc:1511] Processing CreateTablet for tablet 0dad18b855bc43529c37d812ced3ffc8 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:04.038573 11886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36215ec218514de9b8449e9461a16c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.039747 11887 tablet_service.cc:1511] Processing CreateTablet for tablet e3cd2cdce4c040dab6503b6000324821 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:04.040113 11882 tablet_service.cc:1511] Processing CreateTablet for tablet e60f33c8f84f4db890e1d5c402de57f3 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:04.041047 11887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cd2cdce4c040dab6503b6000324821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.041651 11889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461f7e8beb6d4f3d9587184658c8e8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.042256 11885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c417277bdea4b6ba9c42a3f1fcada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.042932 11882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60f33c8f84f4db890e1d5c402de57f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.044070 11888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290f48e0551d4a6fa4f08dede4d79ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.046319 11884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dad18b855bc43529c37d812ced3ffc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.065418 11881 tablet_service.cc:1511] Processing CreateTablet for tablet b5c0dae7a44148d5abbfb963fa6eb387 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:04.066870 11881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c0dae7a44148d5abbfb963fa6eb387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.077435 11886 tablet_service.cc:1511] Processing CreateTablet for tablet 49301cd9cec046a69c616dc8792466bb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:04.078883 11886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49301cd9cec046a69c616dc8792466bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.078836 12038 tablet_service.cc:1511] Processing CreateTablet for tablet 461f7e8beb6d4f3d9587184658c8e8fb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:04.082479 12038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461f7e8beb6d4f3d9587184658c8e8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.090104 12037 tablet_service.cc:1511] Processing CreateTablet for tablet 290f48e0551d4a6fa4f08dede4d79ec7 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:04.093362 12036 tablet_service.cc:1511] Processing CreateTablet for tablet e3cd2cdce4c040dab6503b6000324821 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:04.096264 12035 tablet_service.cc:1511] Processing CreateTablet for tablet 36215ec218514de9b8449e9461a16c49 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:04.098775 12034 tablet_service.cc:1511] Processing CreateTablet for tablet 6c417277bdea4b6ba9c42a3f1fcada34 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:04.097934 12031 tablet_service.cc:1511] Processing CreateTablet for tablet e60f33c8f84f4db890e1d5c402de57f3 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:04.091626 12037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290f48e0551d4a6fa4f08dede4d79ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.101334 12033 tablet_service.cc:1511] Processing CreateTablet for tablet 0dad18b855bc43529c37d812ced3ffc8 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:04.102767 12033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dad18b855bc43529c37d812ced3ffc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.104110 12031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60f33c8f84f4db890e1d5c402de57f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.105051 12032 tablet_service.cc:1511] Processing CreateTablet for tablet 77fcc90cbf664798be4314a8076dbf1e (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:04.105933 12036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cd2cdce4c040dab6503b6000324821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.106755 12035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36215ec218514de9b8449e9461a16c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.108827 12030 tablet_service.cc:1511] Processing CreateTablet for tablet b5c0dae7a44148d5abbfb963fa6eb387 (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:04.110301 12030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c0dae7a44148d5abbfb963fa6eb387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.112411 12032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fcc90cbf664798be4314a8076dbf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.113623 12034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c417277bdea4b6ba9c42a3f1fcada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.156637 12087 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.164556 12088 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.166844 12029 tablet_service.cc:1511] Processing CreateTablet for tablet 49301cd9cec046a69c616dc8792466bb (DEFAULT_TABLE table=test-workload [id=4efab9db3e784049a020a815472e1cad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:04.168584 12029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49301cd9cec046a69c616dc8792466bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.189364 12088 tablet_bootstrap.cc:654] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.163627 12087 tablet_bootstrap.cc:654] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.230089 12089 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.233455 12088 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.234079 12088 ts_tablet_manager.cc:1403] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.070s	user 0.004s	sys 0.018s
I20260709 13:06:04.236764 12087 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.237349 12087 ts_tablet_manager.cc:1403] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.010s
I20260709 13:06:04.241158 12088 raft_consensus.cc:359] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.242020 12088 raft_consensus.cc:385] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.242326 12088 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.242254 12087 raft_consensus.cc:359] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.243198 12087 raft_consensus.cc:385] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.243525 12087 raft_consensus.cc:740] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.243312 12088 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.244011 12089 tablet_bootstrap.cc:654] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.244297 12087 consensus_queue.cc:260] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.245776 11924 heartbeater.cc:499] Master 127.0.107.254:40129 was elected leader, sending a full tablet report...
I20260709 13:06:04.246379 12088 ts_tablet_manager.cc:1434] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.005s
I20260709 13:06:04.248008 12088 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.257143 12088 tablet_bootstrap.cc:654] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.270404 11999 heartbeater.cc:499] Master 127.0.107.254:40129 was elected leader, sending a full tablet report...
I20260709 13:06:04.271356 12087 ts_tablet_manager.cc:1434] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.021s
I20260709 13:06:04.272285 12089 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.272737 12092 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.272825 12087 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.272862 12089 ts_tablet_manager.cc:1403] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.023s
I20260709 13:06:04.273322 12092 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.270414 12088 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.276194 12088 ts_tablet_manager.cc:1403] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.009s
I20260709 13:06:04.277287 12089 raft_consensus.cc:359] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.278115 12089 raft_consensus.cc:385] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.278394 12089 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.279184 12089 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.279542 12087 tablet_bootstrap.cc:654] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.279732 12088 raft_consensus.cc:359] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.280666 12088 raft_consensus.cc:385] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.280977 12088 raft_consensus.cc:740] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.281749 12088 consensus_queue.cc:260] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.283471 12092 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.334089 12093 raft_consensus.cc:493] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.334971 12093 raft_consensus.cc:515] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.338469 12093 leader_election.cc:290] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237), d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167)
I20260709 13:06:04.344990 12092 raft_consensus.cc:493] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.345759 12092 raft_consensus.cc:515] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.348728 12092 leader_election.cc:290] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.382367 12089 ts_tablet_manager.cc:1434] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.109s	user 0.041s	sys 0.066s
I20260709 13:06:04.382900 12073 heartbeater.cc:499] Master 127.0.107.254:40129 was elected leader, sending a full tablet report...
I20260709 13:06:04.383632 12089 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.387692 12088 ts_tablet_manager.cc:1434] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.111s	user 0.032s	sys 0.069s
I20260709 13:06:04.390548 12088 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.392375 12089 tablet_bootstrap.cc:654] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.397058 11973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.397684 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
W20260709 13:06:04.399065 11865 leader_election.cc:343] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:04.399780 11865 leader_election.cc:343] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.400969 12087 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.401561 12087 ts_tablet_manager.cc:1403] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.053s
I20260709 13:06:04.401921 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:04.403570 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:04.404839 12087 raft_consensus.cc:359] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
W20260709 13:06:04.405268 11865 leader_election.cc:343] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.405776 11865 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7; no voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:04.405649 12087 raft_consensus.cc:385] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.407464 12087 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
W20260709 13:06:04.407593 11925 tablet.cc:2406] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:04.408416 12087 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.407472 12099 raft_consensus.cc:2749] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.410189 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
W20260709 13:06:04.411748 11940 leader_election.cc:343] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.403627 12088 tablet_bootstrap.cc:654] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.413009 12087 ts_tablet_manager.cc:1434] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
W20260709 13:06:04.413388 11865 leader_election.cc:343] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.414752 11865 leader_election.cc:304] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7; no voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:04.415932 12099 raft_consensus.cc:2749] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.414078 12087 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.426894 12089 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.427520 12089 ts_tablet_manager.cc:1403] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.022s
I20260709 13:06:04.430567 12089 raft_consensus.cc:359] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.431283 12089 raft_consensus.cc:385] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.431602 12089 raft_consensus.cc:740] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.432399 12089 consensus_queue.cc:260] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.434571 12089 ts_tablet_manager.cc:1434] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:06:04.434324 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.435573 12089 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
W20260709 13:06:04.436401 11940 leader_election.cc:343] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.437100 11940 leader_election.cc:304] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01f3b6608004bd881304eecb689b308; no voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:04.438588 12093 raft_consensus.cc:2749] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.442153 12089 tablet_bootstrap.cc:654] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.444190 12087 tablet_bootstrap.cc:654] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.445252 12088 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.445822 12088 ts_tablet_manager.cc:1403] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.018s
I20260709 13:06:04.446926 12089 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.447407 12089 ts_tablet_manager.cc:1403] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:06:04.449065 12088 raft_consensus.cc:359] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.449797 12087 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.449856 12088 raft_consensus.cc:385] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.449733 12089 raft_consensus.cc:359] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.450453 12088 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.450541 12087 ts_tablet_manager.cc:1403] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.003s
I20260709 13:06:04.450690 12089 raft_consensus.cc:385] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.451156 12089 raft_consensus.cc:740] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.451324 12088 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.452214 12089 consensus_queue.cc:260] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.454512 12089 ts_tablet_manager.cc:1434] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:04.454653 12088 ts_tablet_manager.cc:1434] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:06:04.455447 12089 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.458331 12087 raft_consensus.cc:359] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.459265 12087 raft_consensus.cc:385] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.462361 12089 tablet_bootstrap.cc:654] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.459630 12087 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.470788 12088 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.471776 12087 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.479027 12088 tablet_bootstrap.cc:654] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:04.480394 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:04.481784 12089 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.482371 12089 ts_tablet_manager.cc:1403] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 13:06:04.485206 12089 raft_consensus.cc:359] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.485955 12089 raft_consensus.cc:385] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.486267 12089 raft_consensus.cc:740] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.487401 12089 consensus_queue.cc:260] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.493402 12089 ts_tablet_manager.cc:1434] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:06:04.494747 12089 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.496395 12087 ts_tablet_manager.cc:1434] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.002s
I20260709 13:06:04.497601 12087 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.503167 12088 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.503566 12089 tablet_bootstrap.cc:654] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.503903 12088 ts_tablet_manager.cc:1403] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.014s
I20260709 13:06:04.505406 12087 tablet_bootstrap.cc:654] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.547251 12088 raft_consensus.cc:359] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.550900 12088 raft_consensus.cc:385] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.551381 12088 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.553076 12088 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.555088 12089 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.555716 12087 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.556473 12089 ts_tablet_manager.cc:1403] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.002s
I20260709 13:06:04.557228 12087 ts_tablet_manager.cc:1403] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.010s
I20260709 13:06:04.562959 12089 raft_consensus.cc:359] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.564345 12088 ts_tablet_manager.cc:1434] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.011s
I20260709 13:06:04.564248 12089 raft_consensus.cc:385] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.563651 12087 raft_consensus.cc:359] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.565052 12089 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.565258 12087 raft_consensus.cc:385] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.565841 12087 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.566785 12088 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.566283 12089 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.567009 12087 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.573192 12089 ts_tablet_manager.cc:1434] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.002s
I20260709 13:06:04.573609 12087 ts_tablet_manager.cc:1434] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 13:06:04.575320 12089 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.566987 12099 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.579289 12099 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.584368 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.585384 11974 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 0.
I20260709 13:06:04.586992 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:04.587642 12088 tablet_bootstrap.cc:654] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.587797 12099 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.587642 12089 tablet_bootstrap.cc:654] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.588928 12097 raft_consensus.cc:493] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.590461 12097 raft_consensus.cc:515] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.592782 12087 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.600884 12097 leader_election.cc:290] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
W20260709 13:06:04.602845 11865 leader_election.cc:343] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.604269 11865 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:04.605183 12087 tablet_bootstrap.cc:654] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.613023 12093 raft_consensus.cc:493] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.613950 12093 raft_consensus.cc:515] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.616783 12093 leader_election.cc:290] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.619555 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.620343 11899 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 0.
I20260709 13:06:04.621832 12099 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.622328 12099 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.622685 12099 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.625543 12087 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.626271 12087 ts_tablet_manager.cc:1403] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.005s
I20260709 13:06:04.628327 11940 leader_election.cc:304] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:04.629401 12093 raft_consensus.cc:2804] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.629750 12093 raft_consensus.cc:493] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.630059 12093 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.630575 12087 raft_consensus.cc:359] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.632332 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:04.634884 12089 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.635471 12089 ts_tablet_manager.cc:1403] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.014s
W20260709 13:06:04.638028 11940 leader_election.cc:343] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.639190 12089 raft_consensus.cc:359] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.640499 12089 raft_consensus.cc:385] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.640928 12089 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.642024 12088 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.641894 12089 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.642627 12088 ts_tablet_manager.cc:1403] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.076s	user 0.028s	sys 0.028s
I20260709 13:06:04.645179 12089 ts_tablet_manager.cc:1434] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:04.646965 12089 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.631651 12087 raft_consensus.cc:385] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.654790 12087 raft_consensus.cc:740] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.663481 12087 consensus_queue.cc:260] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.665390 12089 tablet_bootstrap.cc:654] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.669538 12099 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.672240 12099 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 election: Requested vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.672540 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.673175 12087 ts_tablet_manager.cc:1434] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.001s
I20260709 13:06:04.673071 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:04.673796 11974 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:04.674325 12014 leader_election.cc:343] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.674794 12087 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.675653 12093 raft_consensus.cc:515] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.677558 12093 leader_election.cc:290] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:04.678853 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:04.679392 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
W20260709 13:06:04.680557 11940 leader_election.cc:343] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.681466 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:04.682091 11899 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.684208 12087 tablet_bootstrap.cc:654] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:04.685931 11865 leader_election.cc:343] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.669770 12088 raft_consensus.cc:359] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.687322 12088 raft_consensus.cc:385] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.687672 12088 raft_consensus.cc:740] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.686666 11974 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 1.
I20260709 13:06:04.693010 11899 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:04.694646 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.694904 11940 leader_election.cc:304] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:04.695916 12118 raft_consensus.cc:2804] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.696542 12118 raft_consensus.cc:697] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:04.697050 11865 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:04.697821 12118 consensus_queue.cc:237] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.698767 12099 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.699661 12099 raft_consensus.cc:697] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 LEADER]: Becoming Leader. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Running, Role: LEADER
I20260709 13:06:04.700700 12099 consensus_queue.cc:237] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.695364 11974 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:04.704101 12014 leader_election.cc:304] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:04.688617 12088 consensus_queue.cc:260] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.705413 12097 raft_consensus.cc:2804] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.705776 12097 raft_consensus.cc:493] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.706126 12097 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.707038 12088 ts_tablet_manager.cc:1434] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.003s
I20260709 13:06:04.709591 12088 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.716286 12097 raft_consensus.cc:515] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.719108 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
W20260709 13:06:04.720775 12014 leader_election.cc:343] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.726670 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:04.727298 11974 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.728783 11794 catalog_manager.cc:5697] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.730548 12097 leader_election.cc:290] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.729957 12088 tablet_bootstrap.cc:654] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.746655 12087 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.747568 12087 ts_tablet_manager.cc:1403] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.028s
I20260709 13:06:04.754536 11797 catalog_manager.cc:5697] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.751379 12087 raft_consensus.cc:359] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.759162 11974 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:04.759784 12087 raft_consensus.cc:385] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.760277 12087 raft_consensus.cc:740] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.754856 12097 raft_consensus.cc:493] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.760779 12014 leader_election.cc:304] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:04.761444 12097 raft_consensus.cc:515] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.761518 12087 consensus_queue.cc:260] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.764441 12089 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.765162 12089 ts_tablet_manager.cc:1403] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.035s
I20260709 13:06:04.766096 12124 raft_consensus.cc:2804] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.772910 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.766719 12097 leader_election.cc:290] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.774236 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
W20260709 13:06:04.774807 12014 leader_election.cc:343] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.775421 12124 raft_consensus.cc:697] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
W20260709 13:06:04.775534 12014 leader_election.cc:343] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.777138 12014 leader_election.cc:304] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7; no voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:04.778704 12097 raft_consensus.cc:2749] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.779299 12124 consensus_queue.cc:237] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.781946 12097 raft_consensus.cc:493] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.782943 12088 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.782783 12097 raft_consensus.cc:515] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.783512 12088 ts_tablet_manager.cc:1403] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.009s
I20260709 13:06:04.785565 12097 leader_election.cc:290] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167)
I20260709 13:06:04.788007 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.790274 11974 raft_consensus.cc:2468] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:04.791033 12089 raft_consensus.cc:359] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.788369 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.794610 12089 raft_consensus.cc:385] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.796367 12089 raft_consensus.cc:740] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.786937 12088 raft_consensus.cc:359] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.798331 12088 raft_consensus.cc:385] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.798661 12088 raft_consensus.cc:740] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.799456 12088 consensus_queue.cc:260] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.801878 12088 ts_tablet_manager.cc:1434] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:06:04.803170 12088 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.791685 12014 leader_election.cc:304] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
W20260709 13:06:04.804577 12014 leader_election.cc:343] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.788487 12087 ts_tablet_manager.cc:1434] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:06:04.805482 12097 raft_consensus.cc:2804] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.806066 12097 raft_consensus.cc:493] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.806375 12087 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.806581 12097 raft_consensus.cc:3060] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.814622 12124 raft_consensus.cc:493] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.814335 11794 catalog_manager.cc:5697] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.797524 12089 consensus_queue.cc:260] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.816066 12129 raft_consensus.cc:493] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.815241 12124 raft_consensus.cc:515] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.816591 12129 raft_consensus.cc:515] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.819630 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.820667 12129 leader_election.cc:290] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.821425 12129 raft_consensus.cc:493] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.822073 12014 leader_election.cc:343] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.821976 12129 raft_consensus.cc:515] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.823356 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.823989 11973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.824218 12129 leader_election.cc:290] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.825954 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.826561 11899 raft_consensus.cc:2468] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:04.828431 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:04.828702 11973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:04.829329 11899 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:04.830125 12014 leader_election.cc:304] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:04.830914 12129 raft_consensus.cc:2804] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:04.831063 12014 leader_election.cc:343] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.831359 12129 raft_consensus.cc:493] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.831557 12014 leader_election.cc:304] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7; no voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:04.831894 12129 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:04.832476 12014 leader_election.cc:343] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.833523 12014 leader_election.cc:304] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:04.829427 11973 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:04.814424 12087 tablet_bootstrap.cc:654] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.818872 12124 leader_election.cc:290] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.843005 12124 raft_consensus.cc:2749] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.843624 12124 raft_consensus.cc:2804] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.843971 12124 raft_consensus.cc:493] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.844265 12124 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.847292 12088 tablet_bootstrap.cc:654] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.847517 12129 raft_consensus.cc:515] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.849648 12129 leader_election.cc:290] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.856065 12097 raft_consensus.cc:515] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.857800 12124 raft_consensus.cc:515] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.858232 12097 leader_election.cc:290] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Requested vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167)
I20260709 13:06:04.858085 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:04.858865 11899 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.859926 12124 leader_election.cc:290] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:04.860363 12089 ts_tablet_manager.cc:1434] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.093s	user 0.011s	sys 0.001s
I20260709 13:06:04.861312 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:04.861871 12089 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.858381 11973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:04.864179 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:04.864781 11974 raft_consensus.cc:3060] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.869508 11898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
W20260709 13:06:04.871060 12014 leader_election.cc:343] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.871083 11897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:04.862074 11972 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:04.872835 12014 leader_election.cc:343] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.873075 11897 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.872821 12089 tablet_bootstrap.cc:654] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.881968 11972 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:04.883354 12014 leader_election.cc:304] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:04.884279 12124 raft_consensus.cc:2804] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.884187 11974 raft_consensus.cc:2468] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:04.885012 12124 raft_consensus.cc:697] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:04.885848 12014 leader_election.cc:304] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:04.886039 12124 consensus_queue.cc:237] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.886694 12097 raft_consensus.cc:2804] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.887183 12097 raft_consensus.cc:697] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:04.888178 12097 consensus_queue.cc:237] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.892431 12087 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.893033 12087 ts_tablet_manager.cc:1403] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.004s
I20260709 13:06:04.893411 11897 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:04.893148 11899 raft_consensus.cc:2468] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:04.895984 12014 leader_election.cc:304] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: e01f3b6608004bd881304eecb689b308
I20260709 13:06:04.896884 12129 raft_consensus.cc:2804] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.897208 12088 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.897409 12129 raft_consensus.cc:697] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:04.897738 12088 ts_tablet_manager.cc:1403] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.003s
I20260709 13:06:04.898260 12129 consensus_queue.cc:237] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.900982 12088 raft_consensus.cc:359] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.901786 12088 raft_consensus.cc:385] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.902100 12088 raft_consensus.cc:740] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.903049 12088 consensus_queue.cc:260] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:04.907261 12088 ts_tablet_manager.cc:1434] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:06:04.908536 12088 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.909137 12089 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.909783 12089 ts_tablet_manager.cc:1403] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.002s
I20260709 13:06:04.913514 12089 raft_consensus.cc:359] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.914425 12089 raft_consensus.cc:385] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.914736 12089 raft_consensus.cc:740] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.915122 12088 tablet_bootstrap.cc:654] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.916461 12087 raft_consensus.cc:359] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.917456 12087 raft_consensus.cc:385] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.917820 12087 raft_consensus.cc:740] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.921725 12087 consensus_queue.cc:260] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.924893 11797 catalog_manager.cc:5697] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.925800 12087 ts_tablet_manager.cc:1434] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:06:04.927834 12087 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.928587 12089 consensus_queue.cc:260] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.938059 12089 ts_tablet_manager.cc:1434] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.001s
I20260709 13:06:04.939654 12089 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.941373 12088 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.941413 12087 tablet_bootstrap.cc:654] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.942008 12088 ts_tablet_manager.cc:1403] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.017s
I20260709 13:06:04.945179 12088 raft_consensus.cc:359] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.945917 12088 raft_consensus.cc:385] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.946236 12088 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.946642 12089 tablet_bootstrap.cc:654] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.947137 12088 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.946911 11794 catalog_manager.cc:5697] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.949379 12088 ts_tablet_manager.cc:1434] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:04.949792 11794 catalog_manager.cc:5697] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.954649 12087 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.955174 12087 ts_tablet_manager.cc:1403] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.003s
I20260709 13:06:04.957993 12087 raft_consensus.cc:359] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.958662 12087 raft_consensus.cc:385] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.958918 12087 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.961103 12088 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.959596 12087 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.967932 12089 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:04.968502 12089 ts_tablet_manager.cc:1403] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.002s
I20260709 13:06:04.967993 12088 tablet_bootstrap.cc:654] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.969483 12087 ts_tablet_manager.cc:1434] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:04.971400 12089 raft_consensus.cc:359] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.972183 12089 raft_consensus.cc:385] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.972441 12089 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.973265 12089 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.975476 12089 ts_tablet_manager.cc:1434] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:04.976962 12087 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:04.980198 12089 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: Bootstrap starting.
I20260709 13:06:04.985275 12087 tablet_bootstrap.cc:654] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.985942 12088 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:04.986477 12088 ts_tablet_manager.cc:1403] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.011s
I20260709 13:06:04.987358 12089 tablet_bootstrap.cc:654] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.989579 12088 raft_consensus.cc:359] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.990288 12088 raft_consensus.cc:385] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.990536 12088 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.991297 12088 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:04.992937 12088 ts_tablet_manager.cc:1434] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:04.994102 12088 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:04.997016 12087 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:04.997673 12087 ts_tablet_manager.cc:1403] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:06:05.000229 12088 tablet_bootstrap.cc:654] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:05.000730 12087 raft_consensus.cc:359] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.001642 12087 raft_consensus.cc:385] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:05.001996 12087 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:05.002557 12089 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: No bootstrap required, opened a new log
I20260709 13:06:05.003301 12089 ts_tablet_manager.cc:1403] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:06:05.002835 12087 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.006327 12089 raft_consensus.cc:359] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.007001 12089 raft_consensus.cc:385] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:05.008327 12089 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Initialized, Role: FOLLOWER
I20260709 13:06:05.009410 12087 ts_tablet_manager.cc:1434] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:05.009205 12089 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.011075 12087 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: Bootstrap starting.
I20260709 13:06:05.014235 12089 ts_tablet_manager.cc:1434] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:06:05.015228 12118 raft_consensus.cc:493] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.015800 12118 raft_consensus.cc:515] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.018046 12118 leader_election.cc:290] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.019673 12088 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: No bootstrap required, opened a new log
I20260709 13:06:05.019972 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:05.020720 12048 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 0.
I20260709 13:06:05.020991 12087 tablet_bootstrap.cc:654] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:05.020227 12088 ts_tablet_manager.cc:1403] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:06:05.021659 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:05.021970 11940 leader_election.cc:304] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.022339 11899 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 0.
I20260709 13:06:05.022891 12118 raft_consensus.cc:2804] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:05.023278 12118 raft_consensus.cc:493] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.023617 12118 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.024591 12088 raft_consensus.cc:359] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.025386 12088 raft_consensus.cc:385] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:05.025704 12088 raft_consensus.cc:740] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:05.026561 12088 consensus_queue.cc:260] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.029486 12088 ts_tablet_manager.cc:1434] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:06:05.031695 12087 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: No bootstrap required, opened a new log
I20260709 13:06:05.032284 12087 ts_tablet_manager.cc:1403] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.007s
I20260709 13:06:05.034478 12118 raft_consensus.cc:515] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.035226 12087 raft_consensus.cc:359] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.035950 12087 raft_consensus.cc:385] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:05.036223 12087 raft_consensus.cc:740] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Initialized, Role: FOLLOWER
I20260709 13:06:05.036921 12087 consensus_queue.cc:260] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.037331 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:05.037925 11899 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.040431 12087 ts_tablet_manager.cc:1434] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:05.041803 12124 raft_consensus.cc:493] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.042333 12124 raft_consensus.cc:515] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.043718 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:05.044346 12048 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.045037 12124 leader_election.cc:290] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:05.046337 12118 leader_election.cc:290] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.046681 11897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:05.047209 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:05.047482 11897 raft_consensus.cc:2393] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3afb90c914b64383b7fa6d5ec19950a7 in current term 1: Already voted for candidate e01f3b6608004bd881304eecb689b308 in this term.
I20260709 13:06:05.046105 11899 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:05.049129 12014 leader_election.cc:304] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7; no voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:05.049585 11940 leader_election.cc:304] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.050287 12124 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.050477 12118 raft_consensus.cc:2804] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:05.051100 12118 raft_consensus.cc:697] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:05.052012 12118 consensus_queue.cc:237] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.052719 12048 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:05.057621 12124 raft_consensus.cc:2749] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:05.061282 11796 catalog_manager.cc:5697] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.108757 12099 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.118830 12048 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.128616 12099 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.141126 12099 raft_consensus.cc:493] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.141685 12099 raft_consensus.cc:515] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:05.143543 12099 leader_election.cc:290] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.144744 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:05.145174 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461f7e8beb6d4f3d9587184658c8e8fb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:05.145653 11974 raft_consensus.cc:2393] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3bce88262bd47f9b492bcb6126b9ba7 in current term 1: Already voted for candidate 3afb90c914b64383b7fa6d5ec19950a7 in this term.
I20260709 13:06:05.147025 11865 leader_election.cc:304] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7; no voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308
I20260709 13:06:05.147666 12099 raft_consensus.cc:3060] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.153530 12099 raft_consensus.cc:2749] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:05.155682 12124 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.156352 12099 raft_consensus.cc:493] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.156240 12124 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.156909 12099 raft_consensus.cc:515] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.157274 12118 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.158792 12099 leader_election.cc:290] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.159624 11899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:05.159853 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:05.160535 11899 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:05.160591 11972 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 0.
I20260709 13:06:05.159622 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:05.160647 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:05.161734 11974 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 0.
I20260709 13:06:05.162070 12048 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 0.
I20260709 13:06:05.158803 12124 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:05.162606 12014 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.164018 11865 leader_election.cc:304] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.164639 12124 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:05.165086 12124 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.165359 12124 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.167230 12099 raft_consensus.cc:2804] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:05.167714 12099 raft_consensus.cc:493] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.168120 12099 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.172381 12118 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.175058 12124 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.175858 12099 raft_consensus.cc:515] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.177489 12124 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:05.178259 11897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:05.178599 12099 leader_election.cc:290] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 election: Requested vote from peers e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.179144 11897 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.179366 11974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:05.179656 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:05.180279 11974 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.180280 12047 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.183948 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:05.184748 11972 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.188651 11974 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 1.
I20260709 13:06:05.190198 11865 leader_election.cc:304] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.191188 12099 raft_consensus.cc:2804] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:05.191727 12099 raft_consensus.cc:697] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 LEADER]: Becoming Leader. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Running, Role: LEADER
I20260709 13:06:05.192574 12047 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bce88262bd47f9b492bcb6126b9ba7 in term 1.
I20260709 13:06:05.192917 12099 consensus_queue.cc:237] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.198988 11972 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:05.199250 11897 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:05.200485 12014 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:05.201751 12124 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:05.202225 12124 raft_consensus.cc:697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:05.203040 12124 consensus_queue.cc:237] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.210950 11793 catalog_manager.cc:5697] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.214811 11897 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.220312 11796 catalog_manager.cc:5697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:05.222522 12124 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.234763 12097 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.275643 12153 raft_consensus.cc:493] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.276289 12153 raft_consensus.cc:515] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.278162 12153 leader_election.cc:290] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.279161 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:05.279166 11897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" is_pre_election: true
I20260709 13:06:05.280010 12047 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 0.
I20260709 13:06:05.280041 11897 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 0.
I20260709 13:06:05.281168 11940 leader_election.cc:304] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.281898 12153 raft_consensus.cc:2804] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:05.282260 12153 raft_consensus.cc:493] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.282672 12153 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.288619 12153 raft_consensus.cc:515] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.290215 12153 leader_election.cc:290] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:05.291150 11897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:05.291241 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:05.291874 11897 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.291924 12047 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.297279 12047 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:05.297381 11897 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:05.298399 11940 leader_election.cc:304] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:05.299103 12153 raft_consensus.cc:2804] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:05.299569 12153 raft_consensus.cc:697] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:05.300323 12153 consensus_queue.cc:237] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:05.307565 11796 catalog_manager.cc:5697] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.314291 12097 consensus_queue.cc:1048] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.323832 11972 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.332017 12124 consensus_queue.cc:1048] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.380229   431 ts_manager.cc:295] Set tserver state for d3bce88262bd47f9b492bcb6126b9ba7 to MAINTENANCE_MODE
I20260709 13:06:05.381171   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:05.405704   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:05.406656   431 tablet_replica.cc:333] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.407845   431 raft_consensus.cc:2243] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.408609   431 pending_rounds.cc:70] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Trying to abort 1 pending ops.
I20260709 13:06:05.408847   431 pending_rounds.cc:77] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635287839936512 op_type: NO_OP noop_request { }
I20260709 13:06:05.409163   431 raft_consensus.cc:2889] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:05.409446   431 raft_consensus.cc:2272] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.411916   431 tablet_replica.cc:333] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.412489   431 raft_consensus.cc:2243] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.412868   431 raft_consensus.cc:2272] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.414688   431 tablet_replica.cc:333] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.415244   431 raft_consensus.cc:2243] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.415656   431 raft_consensus.cc:2272] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.418130   431 tablet_replica.cc:333] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.418727   431 raft_consensus.cc:2243] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.419153   431 raft_consensus.cc:2272] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.421474   431 tablet_replica.cc:333] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.421541 12124 consensus_queue.cc:1048] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.422019   431 raft_consensus.cc:2243] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.422546   431 raft_consensus.cc:2272] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.425047   431 tablet_replica.cc:333] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.425660   431 raft_consensus.cc:2243] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.426065   431 raft_consensus.cc:2272] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.427893   431 tablet_replica.cc:333] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.428640   431 raft_consensus.cc:2243] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.429174   431 raft_consensus.cc:2272] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.432116   431 tablet_replica.cc:333] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.432775   431 raft_consensus.cc:2243] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.433266   431 raft_consensus.cc:2272] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.436839   431 tablet_replica.cc:333] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.437604   431 raft_consensus.cc:2243] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.438359   431 raft_consensus.cc:2272] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.441293 12014 consensus_peers.cc:597] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.441763   431 tablet_replica.cc:333] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: stopping tablet replica
I20260709 13:06:05.442539   431 raft_consensus.cc:2243] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.443115   431 raft_consensus.cc:2272] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.443971 12014 consensus_peers.cc:597] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.467451 12129 consensus_queue.cc:1048] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.475865   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:05.481858 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:05.484849 12153 consensus_queue.cc:1048] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:05.506027 11940 consensus_peers.cc:597] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.567363 11940 consensus_peers.cc:597] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.647876 12014 consensus_peers.cc:597] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.688023 12014 consensus_peers.cc:597] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.765434 12097 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:05.814000 11940 consensus_peers.cc:597] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.819212 12153 consensus_queue.cc:1048] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:05.902237 12014 consensus_peers.cc:597] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.482956 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:06.619897 12153 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3bce88262bd47f9b492bcb6126b9ba7)
I20260709 13:06:06.620491 12153 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.622346 12153 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:06.623996 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
W20260709 13:06:06.625959 11940 leader_election.cc:336] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111)
I20260709 13:06:06.626327 11940 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01f3b6608004bd881304eecb689b308; no voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:06.626896 12153 raft_consensus.cc:2749] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:06.631572 12097 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3bce88262bd47f9b492bcb6126b9ba7)
I20260709 13:06:06.632180 12097 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.633836 12097 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:06.635317 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308" is_pre_election: true
I20260709 13:06:06.635910 11972 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 1.
I20260709 13:06:06.637604 12014 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:06.638274 12097 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:06.638336 12014 leader_election.cc:336] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111)
I20260709 13:06:06.638669 12097 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3bce88262bd47f9b492bcb6126b9ba7)
I20260709 13:06:06.639068 12097 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.644277 12097 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.645984 12097 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:06.647544 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:06.648252 11972 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:06.649727 12014 leader_election.cc:336] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111)
I20260709 13:06:06.653942 11972 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:06.654897 12014 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:06.655515 12097 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.656157 12097 raft_consensus.cc:697] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:06.656911 12097 consensus_queue.cc:237] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.663111 11796 catalog_manager.cc:5697] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 1 to 2, leader changed from d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:06.883409 11940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111) [suppressed 127 similar messages]
I20260709 13:06:06.963435 12153 raft_consensus.cc:493] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.964054 12153 raft_consensus.cc:515] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.965772 12153 leader_election.cc:290] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:06.967559 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7" is_pre_election: true
I20260709 13:06:06.968232 12047 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 1.
I20260709 13:06:06.969180 11940 leader_election.cc:304] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:06.969815 12153 raft_consensus.cc:2804] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:06.970189 12153 raft_consensus.cc:493] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.970515 12153 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:06.971362 11940 leader_election.cc:336] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111)
I20260709 13:06:06.976074 12153 raft_consensus.cc:515] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.977751 12153 leader_election.cc:290] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:06.979218 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49301cd9cec046a69c616dc8792466bb" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:06.979879 12047 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:06.981343 11940 leader_election.cc:336] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111)
I20260709 13:06:06.984933 12047 raft_consensus.cc:2468] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:06.985985 11940 leader_election.cc:304] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: d3bce88262bd47f9b492bcb6126b9ba7
I20260709 13:06:06.986678 12153 raft_consensus.cc:2804] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.987170 12153 raft_consensus.cc:697] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:06.988018 12153 consensus_queue.cc:237] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:06.995424 11796 catalog_manager.cc:5697] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 1 to 2, leader changed from d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:07.044622 12014 consensus_peers.cc:597] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:07.136430 11974 raft_consensus.cc:1275] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.137707 12097 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:07.483898 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:07.487339 12048 raft_consensus.cc:1275] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:07.488902 12176 consensus_queue.cc:1048] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:07.502779 11940 consensus_peers.cc:597] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:07.898278 12014 consensus_peers.cc:597] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:07.913741 11940 consensus_peers.cc:597] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.056202 11940 consensus_peers.cc:597] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.073679 11940 consensus_peers.cc:597] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.083806 12014 consensus_peers.cc:597] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.097779 12014 consensus_peers.cc:597] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.245966 12014 consensus_peers.cc:597] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.425161 12014 consensus_peers.cc:597] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.484876 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:09.378875 12014 consensus_peers.cc:597] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.485879 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:10.486913 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:10.526986   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:10.535068 12193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:10.541122 12196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:10.542889 12194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:10.544832 11940 consensus_peers.cc:597] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:10.545418   431 server_base.cc:1061] running on GCE node
I20260709 13:06:10.547287   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:10.547497   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:10.547654   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602370547630 us; error 0 us; skew 500 ppm
I20260709 13:06:10.548207   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:10.551054   431 webserver.cc:533] Webserver started at http://127.0.107.193:33797/ using document root <none> and password file <none>
I20260709 13:06:10.551515   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:10.551705   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:06:10.555586   431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:10.559906 12201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:10.560808   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:10.561112   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-kgkf"
I20260709 13:06:10.561432   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:10.581660   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:10.582881   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:10.584300   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:10.588260 12208 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:06:10.593466 12014 consensus_peers.cc:597] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.639540 12014 consensus_peers.cc:597] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:10.644392   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:06:10.644946   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.003s	sys 0.000s
I20260709 13:06:10.645311   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:06:10.650607 12208 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.662362 12208 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.663270 12208 tablet_bootstrap.cc:492] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.664062 12208 ts_tablet_manager.cc:1403] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 13:06:10.666796 12208 raft_consensus.cc:359] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.667344 12208 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.668124 12208 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.670204 12208 ts_tablet_manager.cc:1434] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:06:10.670876 12208 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.681803 12208 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.682534 12208 tablet_bootstrap.cc:492] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.683223 12208 ts_tablet_manager.cc:1403] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 13:06:10.685371 12208 raft_consensus.cc:359] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:10.685984 12208 raft_consensus.cc:740] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.686771 12208 consensus_queue.cc:260] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } } peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } }
I20260709 13:06:10.687990   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:06:10.688360   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.043s	user 0.040s	sys 0.003s
I20260709 13:06:10.688495 12208 ts_tablet_manager.cc:1434] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:06:10.689337 12208 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.701725 12208 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.702564 12208 tablet_bootstrap.cc:492] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.703213 12208 ts_tablet_manager.cc:1403] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:06:10.705578 12208 raft_consensus.cc:359] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.706220 12208 raft_consensus.cc:740] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.707129 12208 consensus_queue.cc:260] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.709203 12208 ts_tablet_manager.cc:1434] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:06:10.710093 12208 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
W20260709 13:06:10.721244 11940 consensus_peers.cc:597] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.728374 11940 consensus_peers.cc:597] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:10.729647 12208 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.730473 12208 tablet_bootstrap.cc:492] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.731066 12208 ts_tablet_manager.cc:1403] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 13:06:10.733280 12208 raft_consensus.cc:359] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.733814 12208 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.734464 12208 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.736080 12208 ts_tablet_manager.cc:1434] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:06:10.736737 12208 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.748916 12208 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.749919 12208 tablet_bootstrap.cc:492] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.750605 12208 ts_tablet_manager.cc:1403] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 13:06:10.752859 12208 raft_consensus.cc:359] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.753415 12208 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.754137 12208 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.755612 12208 ts_tablet_manager.cc:1434] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:06:10.756326 12208 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.760094   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43167
I20260709 13:06:10.760188 12271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43167 every 8 connection(s)
I20260709 13:06:10.765040   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:10.779049 12208 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.780277 12208 tablet_bootstrap.cc:492] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.781239 12208 ts_tablet_manager.cc:1403] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.015s
I20260709 13:06:10.784695 12208 raft_consensus.cc:359] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.785683 12208 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.786893 12208 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.790251 12208 ts_tablet_manager.cc:1434] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:06:10.791222 12208 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.799432 12272 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40129
I20260709 13:06:10.800019 12272 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:10.800981 12272 heartbeater.cc:507] Master 127.0.107.254:40129 requested a full tablet report, sending...
I20260709 13:06:10.804052 12048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:10.804822 12048 raft_consensus.cc:606] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:10.805276 12208 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.806234 12208 tablet_bootstrap.cc:492] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.806867 12208 ts_tablet_manager.cc:1403] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:06:10.807720 12048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36215ec218514de9b8449e9461a16c49"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:10.807921 11796 ts_manager.cc:194] Re-registered known tserver with Master: d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167)
I20260709 13:06:10.808375 12048 raft_consensus.cc:606] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:10.809247 12208 raft_consensus.cc:359] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.809929 12208 raft_consensus.cc:740] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.810809 12208 consensus_queue.cc:260] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.812991 12208 ts_tablet_manager.cc:1434] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:06:10.813760 12208 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.822556 11972 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:10.823158 11972 raft_consensus.cc:606] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:10.824572   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:10.824796 11796 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36862
I20260709 13:06:10.825234   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:10.825920   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:10.826427 12208 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.827266 12208 tablet_bootstrap.cc:492] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.827988 12208 ts_tablet_manager.cc:1403] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.003s
I20260709 13:06:10.828912 12272 heartbeater.cc:499] Master 127.0.107.254:40129 was elected leader, sending a full tablet report...
I20260709 13:06:10.830448 12208 raft_consensus.cc:359] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.831151 12208 raft_consensus.cc:740] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.831849 12208 consensus_queue.cc:260] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.833701 12208 ts_tablet_manager.cc:1434] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:10.834353 12208 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.844705 12208 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.845430 12208 tablet_bootstrap.cc:492] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.845948 12208 ts_tablet_manager.cc:1403] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:10.848014 12208 raft_consensus.cc:359] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.848500 12208 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.849020 12208 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.850672 12208 ts_tablet_manager.cc:1434] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:10.851444 12208 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap starting.
I20260709 13:06:10.863121 12208 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:10.863982 12208 tablet_bootstrap.cc:492] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Bootstrap complete.
I20260709 13:06:10.864516 12208 ts_tablet_manager.cc:1403] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent bootstrapping tablet: real 0.013s	user 0.002s	sys 0.010s
I20260709 13:06:10.866268 12208 raft_consensus.cc:359] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.866882 12208 raft_consensus.cc:740] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bce88262bd47f9b492bcb6126b9ba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:10.867415 12208 consensus_queue.cc:260] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:10.869186 12208 ts_tablet_manager.cc:1434] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:06:10.997997 12186 raft_consensus.cc:993] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:10.998469 12231 raft_consensus.cc:1081] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:11.000018 12048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:11.000555 12048 raft_consensus.cc:493] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.000845 12048 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.005966 12048 raft_consensus.cc:515] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.007573 12048 leader_election.cc:290] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:11.008565 12246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:11.008955 11972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:11.009270 12246 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.009603 11972 raft_consensus.cc:3055] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:11.009972 11972 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:11.010785 11972 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.011853 11972 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.012913 12287 raft_consensus.cc:993] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.013496 12183 raft_consensus.cc:1081] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.015250 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:11.015857 11971 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.016225 11971 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.016894 12246 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:11.018208 12014 leader_election.cc:304] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:11.019055 11972 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:11.019166 12183 raft_consensus.cc:2804] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:11.020006 12183 raft_consensus.cc:697] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:11.020851 12183 consensus_queue.cc:237] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.023540 11971 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.024250 12246 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.025811 11971 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:11.031183 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:11.031957 12048 raft_consensus.cc:3055] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:11.032290 12048 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:11.033156 12048 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.034461 12048 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.034543 11796 catalog_manager.cc:5697] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 1 to 2, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.040395 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:11.041343 12245 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.043490 12048 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:11.045620 11940 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:11.046708 12231 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:11.047233 12231 raft_consensus.cc:697] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:11.048125 12231 consensus_queue.cc:237] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.057928 12245 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:11.068849 11793 catalog_manager.cc:5697] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 1 to 2, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:11.081302 12287 raft_consensus.cc:993] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.081794 12183 raft_consensus.cc:1081] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.083616 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36215ec218514de9b8449e9461a16c49"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:11.084295 11971 raft_consensus.cc:493] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.084656 11971 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.090565 11971 raft_consensus.cc:515] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.092443 11971 leader_election.cc:290] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:11.093384 12246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:11.093742 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36215ec218514de9b8449e9461a16c49" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:11.094123 12246 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.094380 12048 raft_consensus.cc:3055] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:11.094877 12048 raft_consensus.cc:740] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:11.095633 12048 consensus_queue.cc:260] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.096823 12048 raft_consensus.cc:3060] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.101850 12246 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:11.103379 11940 leader_election.cc:304] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:11.104338 12231 raft_consensus.cc:2804] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:11.104718 12048 raft_consensus.cc:2468] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:11.106041 12231 raft_consensus.cc:697] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:11.107671 12231 consensus_queue.cc:237] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:11.115541 11793 catalog_manager.cc:5697] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 1 to 2, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:11.131934 12069 debug-util.cc:398] Leaking SignalData structure 0x7b0800419d00 after lost signal to thread 434
I20260709 13:06:11.140882 12245 raft_consensus.cc:3060] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.147591 12245 pending_rounds.cc:85] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Aborting all ops after (but not including) 0
I20260709 13:06:11.147887 12245 pending_rounds.cc:107] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:11.148155 12245 raft_consensus.cc:2889] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:11.469455 11971 raft_consensus.cc:1275] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.471184 12287 consensus_queue.cc:1048] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:11.473765 12246 raft_consensus.cc:1275] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.475108 12287 consensus_queue.cc:1048] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:11.488687 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:11.566262 12245 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.567735 12231 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:11.576292 12048 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.579226 12186 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:11.653383 12048 raft_consensus.cc:1275] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.654933 12231 consensus_queue.cc:1048] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:11.666111 12245 raft_consensus.cc:1275] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:11.673307 12231 consensus_queue.cc:1048] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:11.833209   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:11.843932 11971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:11.844523 11971 raft_consensus.cc:606] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:11.847149 11971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:11.847683 11971 raft_consensus.cc:606] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:11.850165 12048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:11.850641 12048 raft_consensus.cc:606] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:11.851830   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:11.852480   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:11.853192   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:11.991146 12287 raft_consensus.cc:993] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.991709 12183 raft_consensus.cc:1081] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:11.993388 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:11.993892 11971 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.994196 11971 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:11.999460 11971 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.001260 11971 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:12.002092 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:12.002271 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:12.002843 12245 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:12.002947 12048 raft_consensus.cc:3055] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:12.003383 12048 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:12.004204 12048 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.005203 12048 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:12.010568 12048 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:12.011201 12245 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:12.011816 11940 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:12.012696 12231 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:12.013229 12231 raft_consensus.cc:697] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:12.014194 12231 consensus_queue.cc:237] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.021879 11796 catalog_manager.cc:5697] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 2 to 3, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.145186 12186 raft_consensus.cc:993] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:12.145761 12186 raft_consensus.cc:1081] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:12.148278 12048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:12.148953 12048 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:12.149360 12048 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:12.157884 12048 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.160181 12048 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 3 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:12.161015 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:12.161778 12245 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:12.161691 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:12.163821 11971 raft_consensus.cc:3055] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:12.164165 11971 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:12.164963 11971 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.166239 11971 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:12.170840 12245 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 3.
I20260709 13:06:12.172302 12014 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:12.173153 12183 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:12.173856 11971 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 3.
I20260709 13:06:12.173702 12183 raft_consensus.cc:697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:12.175105 12183 consensus_queue.cc:237] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.182387 11796 catalog_manager.cc:5697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 2 to 3, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:12.258846 12297 raft_consensus.cc:993] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:12.259446 12231 raft_consensus.cc:1081] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:12.261209 12048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:12.261860 12048 raft_consensus.cc:493] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:12.262261 12048 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.270254 12048 raft_consensus.cc:515] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.272506 12048 leader_election.cc:290] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:12.273186 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:12.273584 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cd2cdce4c040dab6503b6000324821" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:12.273900 12245 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.274309 11971 raft_consensus.cc:3055] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.274713 11971 raft_consensus.cc:740] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:12.275552 11971 consensus_queue.cc:260] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.276894 11971 raft_consensus.cc:3060] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.279464 12245 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:12.280707 12014 leader_election.cc:304] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:12.281440 12183 raft_consensus.cc:2804] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.281906 12183 raft_consensus.cc:697] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:12.282806 12183 consensus_queue.cc:237] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:12.285032 11971 raft_consensus.cc:2468] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:12.291033 11796 catalog_manager.cc:5697] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 1 to 2, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:12.413892 12245 raft_consensus.cc:1275] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:12.415560 12231 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:12.428272 12048 raft_consensus.cc:1275] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:12.430655 12186 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:12.489818 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:12.568367 12245 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:12.569852 12183 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:12.581370 11971 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:12.585860 12321 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:12.774559 11972 raft_consensus.cc:1275] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.775895 12321 consensus_queue.cc:1048] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.788341 12246 raft_consensus.cc:1275] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.790259 12320 consensus_queue.cc:1048] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:12.863456   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:12.875954 12048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:12.876626 12048 raft_consensus.cc:606] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:12.879478 12048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:12.879988 12048 raft_consensus.cc:606] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:12.883049 11972 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:12.883589 11972 raft_consensus.cc:606] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:12.884800   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:12.885294   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:12.885874   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:13.138504 12183 raft_consensus.cc:993] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:13.140336 12183 raft_consensus.cc:1081] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:13.142695 11972 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:13.143062 12327 raft_consensus.cc:993] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:13.143317 11972 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.143599 12327 raft_consensus.cc:1081] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:13.143795 11972 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:13.148870 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:13.149494 11971 raft_consensus.cc:493] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.149868 11971 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.151176 11972 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.153779 11972 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 4 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:13.155349 12048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:13.155623 12246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:13.156114 12048 raft_consensus.cc:3055] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:13.156280 12246 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:13.156524 12048 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:13.156876 11971 raft_consensus.cc:515] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.157454 12048 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.158911 12048 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:13.159071 11971 leader_election.cc:290] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:13.159806 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:13.160387 12245 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.163909 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:13.164616 12047 raft_consensus.cc:3055] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.164916 12047 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:13.165750 12047 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.167330 12047 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.167797 12246 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 4.
I20260709 13:06:13.169039 12048 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 4.
I20260709 13:06:13.169175 11940 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:13.170295 12326 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:13.170774 12326 raft_consensus.cc:697] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:13.171605 12326 consensus_queue.cc:237] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.176579 12245 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:13.178532 11940 leader_election.cc:304] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:13.179447 12047 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:13.179543 12326 raft_consensus.cc:2804] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.180442 12326 raft_consensus.cc:697] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:13.181255 12326 consensus_queue.cc:237] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.181432 11796 catalog_manager.cc:5697] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 3 to 4, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.200229 11796 catalog_manager.cc:5697] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 1 to 2, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.256026 12325 raft_consensus.cc:993] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:13.256580 12326 raft_consensus.cc:1081] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:13.258229 12047 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:13.258911 12047 raft_consensus.cc:493] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.259347 12047 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.268430 12047 raft_consensus.cc:515] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.271003 12047 leader_election.cc:290] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:13.271596 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:13.272073 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:13.272305 12245 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.272694 11971 raft_consensus.cc:3055] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.273015 11971 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:13.273813 11971 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.275058 11971 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.280751 12245 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:13.281903 12014 leader_election.cc:304] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:13.282311 11971 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 2.
I20260709 13:06:13.282747 12320 raft_consensus.cc:2804] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.283614 12320 raft_consensus.cc:697] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:13.284615 12320 consensus_queue.cc:237] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:13.292729 11796 catalog_manager.cc:5697] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 1 to 2, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:13.490748 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:13.572053 12047 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:13.573861 12326 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:13.584816 12245 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:13.586625 12326 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:13.669715 12245 raft_consensus.cc:1275] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.671433 12326 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.679126 12047 raft_consensus.cc:1275] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.685343 12325 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:13.703553 11971 raft_consensus.cc:1275] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.705560 12183 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.722054 12245 raft_consensus.cc:1275] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.725510 12320 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.892494   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:13.900185 11971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:13.900801 11971 raft_consensus.cc:606] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:13.903622 12047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:13.904181 12047 raft_consensus.cc:606] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:13.906688 11971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:13.907155 11971 raft_consensus.cc:606] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:13.908242   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:13.908723   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:13.909272   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:14.130343 12325 raft_consensus.cc:993] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:14.131021 12325 raft_consensus.cc:1081] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:14.133005 12048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:14.133631 12048 raft_consensus.cc:493] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.134011 12048 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.134104 12320 raft_consensus.cc:993] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:14.134845 12341 raft_consensus.cc:1081] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:14.136438 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:14.136991 11971 raft_consensus.cc:493] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.137339 11971 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.142195 12048 raft_consensus.cc:515] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.142922 11971 raft_consensus.cc:515] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.145109 11971 leader_election.cc:290] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:14.145927 12246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:14.147070 12246 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.147292 12047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c0dae7a44148d5abbfb963fa6eb387" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:14.145642 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:14.148214 12047 raft_consensus.cc:3055] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:14.148339 12245 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.148674 12047 raft_consensus.cc:740] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:14.149778 12047 consensus_queue.cc:260] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.151319 12047 raft_consensus.cc:3060] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.155901 12326 raft_consensus.cc:993] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:14.156589 12326 raft_consensus.cc:1081] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:14.158818 12046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:14.158939 12246 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:14.159515 12046 raft_consensus.cc:493] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.159984 12046 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:14.156570 12245 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 3.
I20260709 13:06:14.160574 11940 leader_election.cc:304] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:14.161549 12326 raft_consensus.cc:2804] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:14.161717 12048 leader_election.cc:290] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 3 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:14.162159 12326 raft_consensus.cc:697] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:14.164295 12014 leader_election.cc:304] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:14.165323 12341 raft_consensus.cc:2804] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:14.165807 12341 raft_consensus.cc:697] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:14.166074 12047 raft_consensus.cc:2468] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:14.166706 12341 consensus_queue.cc:237] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.168840 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290f48e0551d4a6fa4f08dede4d79ec7" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:14.169699 11971 raft_consensus.cc:3055] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:14.170078 11971 raft_consensus.cc:740] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:14.170939 11971 consensus_queue.cc:260] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.172729 11971 raft_consensus.cc:3060] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.163152 12326 consensus_queue.cc:237] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.177218 12046 raft_consensus.cc:515] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.178609 11796 catalog_manager.cc:5697] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 2 to 3, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.180498 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:14.188004 11971 raft_consensus.cc:2468] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 3.
I20260709 13:06:14.187848 11793 catalog_manager.cc:5697] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 2 to 3, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.190193 12046 leader_election.cc:290] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 4 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:14.181239 12245 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:14.193331 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e60f33c8f84f4db890e1d5c402de57f3" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:14.194075 11971 raft_consensus.cc:3055] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:14.194482 11971 raft_consensus.cc:740] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:14.195538 11971 consensus_queue.cc:260] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.197041 11971 raft_consensus.cc:3060] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:14.199074 12245 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 4.
I20260709 13:06:14.200625 12014 leader_election.cc:304] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:14.201624 12341 raft_consensus.cc:2804] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:14.202127 12341 raft_consensus.cc:697] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:14.203012 12341 consensus_queue.cc:237] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:14.212908 11971 raft_consensus.cc:2468] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 4.
I20260709 13:06:14.230201 11793 catalog_manager.cc:5697] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 3 to 4, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:14.491873 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:14.640213 12046 raft_consensus.cc:1275] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.641883 12325 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.652595 11971 raft_consensus.cc:1275] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.657310 12341 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.658185 12245 raft_consensus.cc:1275] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.661355 12344 consensus_queue.cc:1048] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.664604 12245 raft_consensus.cc:1275] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.672130 12353 consensus_queue.cc:1048] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.763588 12245 raft_consensus.cc:1275] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:14.764822 12353 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.774772 11971 raft_consensus.cc:1275] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:14.776418 12353 consensus_queue.cc:1048] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.915715   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:14.925143 11971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
mode: GRACEFUL
new_leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:45418
I20260709 13:06:14.925904 11971 raft_consensus.cc:606] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 LEADER]: Received request to transfer leadership to TS 3afb90c914b64383b7fa6d5ec19950a7
I20260709 13:06:14.928843 12046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:14.929344 12046 raft_consensus.cc:606] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:14.931842 12046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
mode: GRACEFUL
new_leader_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:34934
I20260709 13:06:14.932322 12046 raft_consensus.cc:606] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Received request to transfer leadership to TS e01f3b6608004bd881304eecb689b308
I20260709 13:06:14.933331   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:14.933832   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:14.934531   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:15.082070 12344 raft_consensus.cc:993] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: : Instructing follower 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:15.082729 12325 raft_consensus.cc:1081] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 LEADER]: Signalling peer 3afb90c914b64383b7fa6d5ec19950a7 to start an election
I20260709 13:06:15.084502 12046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8"
dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
 from {username='slave'} at 127.0.0.1:47698
I20260709 13:06:15.085191 12046 raft_consensus.cc:493] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.085527 12046 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:15.093859 12046 raft_consensus.cc:515] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.095930 12046 leader_election.cc:290] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 5 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569)
I20260709 13:06:15.096731 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:15.097571 12245 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:15.097769 11971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dad18b855bc43529c37d812ced3ffc8" candidate_uuid: "3afb90c914b64383b7fa6d5ec19950a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e01f3b6608004bd881304eecb689b308"
I20260709 13:06:15.098331 11971 raft_consensus.cc:3055] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:15.098567 11971 raft_consensus.cc:740] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:15.099426 11971 consensus_queue.cc:260] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.100539 11971 raft_consensus.cc:3060] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:15.103902 12245 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 5.
I20260709 13:06:15.105345 12014 leader_election.cc:304] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afb90c914b64383b7fa6d5ec19950a7, d3bce88262bd47f9b492bcb6126b9ba7; no voters: 
I20260709 13:06:15.106194 12353 raft_consensus.cc:2804] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:15.106604 12353 raft_consensus.cc:697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 5 LEADER]: Becoming Leader. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:15.107468 11971 raft_consensus.cc:2468] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb90c914b64383b7fa6d5ec19950a7 in term 5.
I20260709 13:06:15.107651 12353 consensus_queue.cc:237] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.116859 11793 catalog_manager.cc:5697] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 reported cstate change: term changed from 4 to 5, leader changed from e01f3b6608004bd881304eecb689b308 (127.0.107.194) to 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "3afb90c914b64383b7fa6d5ec19950a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:15.184304 12341 raft_consensus.cc:993] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:15.184790 12353 raft_consensus.cc:1081] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:15.186323 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:15.186787 11971 raft_consensus.cc:493] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.187073 11971 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.193181 11971 raft_consensus.cc:515] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.195189 11971 leader_election.cc:290] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:15.195906 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:15.196322 12046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c417277bdea4b6ba9c42a3f1fcada34" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:15.196655 12245 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.197016 12046 raft_consensus.cc:3055] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:15.197348 12046 raft_consensus.cc:740] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:15.198331 12046 consensus_queue.cc:260] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.199779 12046 raft_consensus.cc:3060] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.204036 12245 raft_consensus.cc:2468] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:15.205412 11940 leader_election.cc:304] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:15.206204 12325 raft_consensus.cc:2804] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.206656 12325 raft_consensus.cc:697] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:15.207639 12325 consensus_queue.cc:237] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.208052 12046 raft_consensus.cc:2468] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 2.
I20260709 13:06:15.214648 11793 catalog_manager.cc:5697] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 1 to 2, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.328121 12365 raft_consensus.cc:993] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: : Instructing follower e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:15.328692 12354 raft_consensus.cc:1081] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Signalling peer e01f3b6608004bd881304eecb689b308 to start an election
I20260709 13:06:15.330067 11971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e"
dest_uuid: "e01f3b6608004bd881304eecb689b308"
 from {username='slave'} at 127.0.0.1:60068
I20260709 13:06:15.330547 11971 raft_consensus.cc:493] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.330858 11971 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:15.336309 11971 raft_consensus.cc:515] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.337709 11971 leader_election.cc:290] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Requested vote from peers d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167), 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195:42237)
I20260709 13:06:15.338536 12245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3bce88262bd47f9b492bcb6126b9ba7"
I20260709 13:06:15.338836 12046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fcc90cbf664798be4314a8076dbf1e" candidate_uuid: "e01f3b6608004bd881304eecb689b308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3afb90c914b64383b7fa6d5ec19950a7"
I20260709 13:06:15.339183 12245 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:15.339433 12046 raft_consensus.cc:3055] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:15.339816 12046 raft_consensus.cc:740] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3afb90c914b64383b7fa6d5ec19950a7, State: Running, Role: LEADER
I20260709 13:06:15.340634 12046 consensus_queue.cc:260] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.341682 12046 raft_consensus.cc:3060] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:15.344380 12245 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:15.345403 11940 leader_election.cc:304] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bce88262bd47f9b492bcb6126b9ba7, e01f3b6608004bd881304eecb689b308; no voters: 
I20260709 13:06:15.346107 12325 raft_consensus.cc:2804] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:15.346640 12325 raft_consensus.cc:697] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Becoming Leader. State: Replica: e01f3b6608004bd881304eecb689b308, State: Running, Role: LEADER
I20260709 13:06:15.347960 12046 raft_consensus.cc:2468] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01f3b6608004bd881304eecb689b308 in term 3.
I20260709 13:06:15.347852 12325 consensus_queue.cc:237] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } }
I20260709 13:06:15.355893 11796 catalog_manager.cc:5697] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 reported cstate change: term changed from 2 to 3, leader changed from 3afb90c914b64383b7fa6d5ec19950a7 (127.0.107.195) to e01f3b6608004bd881304eecb689b308 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "e01f3b6608004bd881304eecb689b308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:15.493183 11845 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:15.566735 12245 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 5 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:06:15.568444 12354 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:15.580760 11971 raft_consensus.cc:1275] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 5 FOLLOWER]: Refusing update from remote peer 3afb90c914b64383b7fa6d5ec19950a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:06:15.583459 12365 consensus_queue.cc:1048] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01f3b6608004bd881304eecb689b308" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:15.630695 12245 raft_consensus.cc:1275] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:15.632113 12325 consensus_queue.cc:1048] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:15.643563 12046 raft_consensus.cc:1275] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:15.645963 12325 consensus_queue.cc:1048] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:15.825852 12245 raft_consensus.cc:1275] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:15.827211 12325 consensus_queue.cc:1048] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bce88262bd47f9b492bcb6126b9ba7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:15.836751 12046 raft_consensus.cc:1275] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Refusing update from remote peer e01f3b6608004bd881304eecb689b308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:15.838730 12326 consensus_queue.cc:1048] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb90c914b64383b7fa6d5ec19950a7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:15.940606   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:15.972090   431 tablet_server.cc:179] TabletServer@127.0.107.193:43167 shutting down...
I20260709 13:06:16.002542   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:16.003346   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.004223   431 raft_consensus.cc:2243] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.004802   431 raft_consensus.cc:2272] T e3cd2cdce4c040dab6503b6000324821 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.007225   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.007939   431 raft_consensus.cc:2243] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.008361   431 raft_consensus.cc:2272] T 6c417277bdea4b6ba9c42a3f1fcada34 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.011068   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.011734   431 raft_consensus.cc:2243] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.012252   431 raft_consensus.cc:2272] T 0dad18b855bc43529c37d812ced3ffc8 P d3bce88262bd47f9b492bcb6126b9ba7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.014921   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.015537   431 raft_consensus.cc:2243] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.016054   431 raft_consensus.cc:2272] T e60f33c8f84f4db890e1d5c402de57f3 P d3bce88262bd47f9b492bcb6126b9ba7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.018254   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.018847   431 raft_consensus.cc:2243] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.019302   431 raft_consensus.cc:2272] T 290f48e0551d4a6fa4f08dede4d79ec7 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.021696   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.022269   431 raft_consensus.cc:2243] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.022714   431 raft_consensus.cc:2272] T b5c0dae7a44148d5abbfb963fa6eb387 P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.025285   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:16.025879 12014 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 75 similar messages]
I20260709 13:06:16.026006   431 raft_consensus.cc:2243] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.026885   431 raft_consensus.cc:2272] T 36215ec218514de9b8449e9461a16c49 P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:16.029042 12014 consensus_peers.cc:597] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.030068   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.030616   431 raft_consensus.cc:2243] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.031045   431 raft_consensus.cc:2272] T 461f7e8beb6d4f3d9587184658c8e8fb P d3bce88262bd47f9b492bcb6126b9ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.033650   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:16.034255   431 raft_consensus.cc:2243] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.034870   431 raft_consensus.cc:2272] T 49301cd9cec046a69c616dc8792466bb P d3bce88262bd47f9b492bcb6126b9ba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.037690   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:16.037886 11940 consensus_peers.cc:597] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.038303   431 raft_consensus.cc:2243] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.038870   431 raft_consensus.cc:2272] T 77fcc90cbf664798be4314a8076dbf1e P d3bce88262bd47f9b492bcb6126b9ba7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.062211   431 tablet_server.cc:196] TabletServer@127.0.107.193:43167 shutdown complete.
I20260709 13:06:16.094892   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:16.098330 11940 consensus_peers.cc:597] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.125552   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:06:16.127612 12014 consensus_peers.cc:597] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.128345   431 tablet_replica.cc:333] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.129444   431 raft_consensus.cc:2243] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.130375   431 raft_consensus.cc:2272] T 49301cd9cec046a69c616dc8792466bb P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.133215   431 tablet_replica.cc:333] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.133931   431 raft_consensus.cc:2243] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.134711   431 raft_consensus.cc:2272] T 77fcc90cbf664798be4314a8076dbf1e P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.137921   431 tablet_replica.cc:333] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.138546   431 raft_consensus.cc:2243] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.139369   431 raft_consensus.cc:2272] T 6c417277bdea4b6ba9c42a3f1fcada34 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.142014   431 tablet_replica.cc:333] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.142642   431 raft_consensus.cc:2243] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.143139   431 raft_consensus.cc:2272] T e3cd2cdce4c040dab6503b6000324821 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.145484   431 tablet_replica.cc:333] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.146036   431 raft_consensus.cc:2243] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.146929   431 raft_consensus.cc:2272] T b5c0dae7a44148d5abbfb963fa6eb387 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.149209   431 tablet_replica.cc:333] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.149837   431 raft_consensus.cc:2243] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.150717   431 raft_consensus.cc:2272] T 36215ec218514de9b8449e9461a16c49 P e01f3b6608004bd881304eecb689b308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.153030   431 tablet_replica.cc:333] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.153602   431 raft_consensus.cc:2243] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.154050   431 raft_consensus.cc:2272] T 290f48e0551d4a6fa4f08dede4d79ec7 P e01f3b6608004bd881304eecb689b308 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.156302   431 tablet_replica.cc:333] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.156858   431 raft_consensus.cc:2243] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.157274   431 raft_consensus.cc:2272] T 0dad18b855bc43529c37d812ced3ffc8 P e01f3b6608004bd881304eecb689b308 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.159845   431 tablet_replica.cc:333] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.160413   431 raft_consensus.cc:2243] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.160899   431 raft_consensus.cc:2272] T e60f33c8f84f4db890e1d5c402de57f3 P e01f3b6608004bd881304eecb689b308 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.163276   431 tablet_replica.cc:333] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308: stopping tablet replica
I20260709 13:06:16.163743   431 raft_consensus.cc:2243] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.164337   431 raft_consensus.cc:2272] T 461f7e8beb6d4f3d9587184658c8e8fb P e01f3b6608004bd881304eecb689b308 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:16.170239 12014 consensus_peers.cc:597] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Couldn't send request to peer e01f3b6608004bd881304eecb689b308. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:16.173736 12014 consensus_peers.cc:597] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:16.191280 12014 consensus_peers.cc:597] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.192773   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:16.218251 12014 consensus_peers.cc:597] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer d3bce88262bd47f9b492bcb6126b9ba7 (127.0.107.193:43167): Couldn't send request to peer d3bce88262bd47f9b492bcb6126b9ba7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.221135   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:16.232650 12014 consensus_peers.cc:597] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 -> Peer e01f3b6608004bd881304eecb689b308 (127.0.107.194:35569): Couldn't send request to peer e01f3b6608004bd881304eecb689b308. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35569: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:16.254122   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:16.254952   431 tablet_replica.cc:333] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.255936   431 raft_consensus.cc:2243] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.257019   431 raft_consensus.cc:2272] T e3cd2cdce4c040dab6503b6000324821 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.259300   431 tablet_replica.cc:333] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.259996   431 raft_consensus.cc:2243] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.260437   431 raft_consensus.cc:2272] T 6c417277bdea4b6ba9c42a3f1fcada34 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.263089   431 tablet_replica.cc:333] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.263703   431 raft_consensus.cc:2243] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.264572   431 raft_consensus.cc:2272] T e60f33c8f84f4db890e1d5c402de57f3 P 3afb90c914b64383b7fa6d5ec19950a7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.266860   431 tablet_replica.cc:333] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.267457   431 raft_consensus.cc:2243] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.268349   431 raft_consensus.cc:2272] T 0dad18b855bc43529c37d812ced3ffc8 P 3afb90c914b64383b7fa6d5ec19950a7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.270983   431 tablet_replica.cc:333] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.271564   431 raft_consensus.cc:2243] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.272047   431 raft_consensus.cc:2272] T 77fcc90cbf664798be4314a8076dbf1e P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.274534   431 tablet_replica.cc:333] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.275149   431 raft_consensus.cc:2243] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.275588   431 raft_consensus.cc:2272] T 49301cd9cec046a69c616dc8792466bb P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.277653   431 tablet_replica.cc:333] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.278168   431 raft_consensus.cc:2243] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.278604   431 raft_consensus.cc:2272] T b5c0dae7a44148d5abbfb963fa6eb387 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.280893   431 tablet_replica.cc:333] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.281426   431 raft_consensus.cc:2243] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.281921   431 raft_consensus.cc:2272] T 36215ec218514de9b8449e9461a16c49 P 3afb90c914b64383b7fa6d5ec19950a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.284091   431 tablet_replica.cc:333] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.284578   431 raft_consensus.cc:2243] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.285403   431 raft_consensus.cc:2272] T 461f7e8beb6d4f3d9587184658c8e8fb P 3afb90c914b64383b7fa6d5ec19950a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.287477   431 tablet_replica.cc:333] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7: stopping tablet replica
I20260709 13:06:16.288039   431 raft_consensus.cc:2243] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.288765   431 raft_consensus.cc:2272] T 290f48e0551d4a6fa4f08dede4d79ec7 P 3afb90c914b64383b7fa6d5ec19950a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.340669   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:16.366955   431 master.cc:562] Master@127.0.107.254:40129 shutting down...
I20260709 13:06:16.396395   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.397011   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.397358   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P b55d41aff28c4895a9f2412bd11d573c: stopping tablet replica
I20260709 13:06:16.417871   431 master.cc:584] Master@127.0.107.254:40129 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13124 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:06:16.465577   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:45273
I20260709 13:06:16.466789   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.472880 12385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.472916 12386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.473875 12388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.474604   431 server_base.cc:1061] running on GCE node
I20260709 13:06:16.475615   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.475859   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.475987   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376475975 us; error 0 us; skew 500 ppm
I20260709 13:06:16.476572   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.479012   431 webserver.cc:533] Webserver started at http://127.0.107.254:42417/ using document root <none> and password file <none>
I20260709 13:06:16.479528   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.479717   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.480059   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.481230   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "f25741850fde4b9692abaab6e7150f4f"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.485854   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:16.489519 12393 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.490322   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260709 13:06:16.490615   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "f25741850fde4b9692abaab6e7150f4f"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.490890   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.506886   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.508270   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.547461   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:45273
I20260709 13:06:16.547583 12444 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:45273 every 8 connection(s)
I20260709 13:06:16.551534 12445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.565058 12445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f: Bootstrap starting.
I20260709 13:06:16.570734 12445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:16.576108 12445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f: No bootstrap required, opened a new log
I20260709 13:06:16.578616 12445 raft_consensus.cc:359] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER }
I20260709 13:06:16.579084 12445 raft_consensus.cc:385] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:16.579394 12445 raft_consensus.cc:740] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25741850fde4b9692abaab6e7150f4f, State: Initialized, Role: FOLLOWER
I20260709 13:06:16.580076 12445 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER }
I20260709 13:06:16.580619 12445 raft_consensus.cc:399] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:16.580859 12445 raft_consensus.cc:493] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:16.581190 12445 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:16.586851 12445 raft_consensus.cc:515] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER }
I20260709 13:06:16.587494 12445 leader_election.cc:304] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f25741850fde4b9692abaab6e7150f4f; no voters: 
I20260709 13:06:16.588833 12445 leader_election.cc:290] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:16.589203 12448 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:16.590628 12448 raft_consensus.cc:697] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 1 LEADER]: Becoming Leader. State: Replica: f25741850fde4b9692abaab6e7150f4f, State: Running, Role: LEADER
I20260709 13:06:16.591336 12448 consensus_queue.cc:237] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER }
I20260709 13:06:16.592268 12445 sys_catalog.cc:565] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:16.596309 12450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f25741850fde4b9692abaab6e7150f4f. Latest consensus state: current_term: 1 leader_uuid: "f25741850fde4b9692abaab6e7150f4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER } }
I20260709 13:06:16.596757 12449 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f25741850fde4b9692abaab6e7150f4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25741850fde4b9692abaab6e7150f4f" member_type: VOTER } }
I20260709 13:06:16.597307 12449 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:16.597360 12450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:16.602837 12456 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:16.607640 12456 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:16.610090   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:16.616559 12456 catalog_manager.cc:1383] Generated new cluster ID: a7ed35b01e4e43248716c6eb5d12362b
I20260709 13:06:16.616837 12456 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:16.632504 12456 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:16.633865 12456 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:16.648571 12456 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f: Generated new TSK 0
I20260709 13:06:16.649243 12456 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:16.676847   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.683265 12466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.684217 12467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.686064   431 server_base.cc:1061] running on GCE node
W20260709 13:06:16.685922 12469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.687508   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.687708   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.687937   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376687920 us; error 0 us; skew 500 ppm
I20260709 13:06:16.688429   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.690864   431 webserver.cc:533] Webserver started at http://127.0.107.193:42563/ using document root <none> and password file <none>
I20260709 13:06:16.691322   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.691478   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.691708   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.693193   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "12c6255369984d74afa72e35e0f6129d"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.698107   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:16.701617 12474 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.702452   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:06:16.702724   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "12c6255369984d74afa72e35e0f6129d"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.702967   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.717229   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.718415   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.719884   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.722232   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.722421   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.722733   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.722915   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.764097   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:40273
I20260709 13:06:16.764221 12536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:40273 every 8 connection(s)
I20260709 13:06:16.769218   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.776912 12541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.780332 12542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.782095 12544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.783017   431 server_base.cc:1061] running on GCE node
I20260709 13:06:16.783876 12537 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45273
I20260709 13:06:16.784313 12537 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:16.784508   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.784803   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.784976   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376784945 us; error 0 us; skew 500 ppm
I20260709 13:06:16.785171 12537 heartbeater.cc:507] Master 127.0.107.254:45273 requested a full tablet report, sending...
I20260709 13:06:16.785642   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.787541 12410 ts_manager.cc:194] Registered new tserver with Master: 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273)
I20260709 13:06:16.788997   431 webserver.cc:533] Webserver started at http://127.0.107.194:41651/ using document root <none> and password file <none>
I20260709 13:06:16.789743   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.789980   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.790278   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.790407 12410 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47744
I20260709 13:06:16.791944   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.796766   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:16.800540 12549 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.801270   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:06:16.801571   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.801877   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.833161   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.834401   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.835945   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.838172   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.838415   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.838660   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.838860   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.879062   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40827
I20260709 13:06:16.879164 12611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40827 every 8 connection(s)
I20260709 13:06:16.884219   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.890928 12615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.892037 12616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.894980   431 server_base.cc:1061] running on GCE node
W20260709 13:06:16.896096 12618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.896632 12612 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45273
I20260709 13:06:16.897078 12612 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:16.897161   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.897486   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.897671   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376897652 us; error 0 us; skew 500 ppm
I20260709 13:06:16.897938 12612 heartbeater.cc:507] Master 127.0.107.254:45273 requested a full tablet report, sending...
I20260709 13:06:16.898399   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.900377 12410 ts_manager.cc:194] Registered new tserver with Master: 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:16.901418   431 webserver.cc:533] Webserver started at http://127.0.107.195:40039/ using document root <none> and password file <none>
I20260709 13:06:16.902124   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.902396   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.902369 12410 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47750
I20260709 13:06:16.903040   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.904675   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "6ff390ca35814b52bdee91878c62fe2d"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.909170   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:16.912515 12623 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.913354   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:16.913658   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "6ff390ca35814b52bdee91878c62fe2d"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-kgkf"
I20260709 13:06:16.913923   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.937783   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.938961   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.940470   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.942768   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.943008   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.943260   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.943436   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.983645   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:39145
I20260709 13:06:16.983843 12685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:39145 every 8 connection(s)
I20260709 13:06:16.999208 12686 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45273
I20260709 13:06:16.999809 12686 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:17.000670 12686 heartbeater.cc:507] Master 127.0.107.254:45273 requested a full tablet report, sending...
I20260709 13:06:17.003145 12410 ts_manager.cc:194] Registered new tserver with Master: 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.003289   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016497939s
I20260709 13:06:17.005463 12410 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47766
I20260709 13:06:17.032583 12410 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47772:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:17.034894 12410 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:17.085963 12502 tablet_service.cc:1511] Processing CreateTablet for tablet 59867d71f9644b6999c665086e02e2c4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.086555 12501 tablet_service.cc:1511] Processing CreateTablet for tablet 6468455d947e4baa8fe1291c19904ae3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.087556 12502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59867d71f9644b6999c665086e02e2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.087596 12651 tablet_service.cc:1511] Processing CreateTablet for tablet 59867d71f9644b6999c665086e02e2c4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.088362 12501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6468455d947e4baa8fe1291c19904ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.089048 12651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59867d71f9644b6999c665086e02e2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.091928 12577 tablet_service.cc:1511] Processing CreateTablet for tablet 59867d71f9644b6999c665086e02e2c4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.092371 12650 tablet_service.cc:1511] Processing CreateTablet for tablet 6468455d947e4baa8fe1291c19904ae3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.093348 12577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59867d71f9644b6999c665086e02e2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.093811 12650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6468455d947e4baa8fe1291c19904ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.097824 12576 tablet_service.cc:1511] Processing CreateTablet for tablet 6468455d947e4baa8fe1291c19904ae3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=37c2f1b98f0a4bafbc3356b79c96d3a5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.099356 12576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6468455d947e4baa8fe1291c19904ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.118434 12701 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: Bootstrap starting.
I20260709 13:06:17.126696 12701 tablet_bootstrap.cc:654] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.129946 12702 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: Bootstrap starting.
I20260709 13:06:17.135213 12703 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: Bootstrap starting.
I20260709 13:06:17.136036 12701 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: No bootstrap required, opened a new log
I20260709 13:06:17.136631 12701 ts_tablet_manager.cc:1403] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.009s
I20260709 13:06:17.138971 12702 tablet_bootstrap.cc:654] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.140000 12701 raft_consensus.cc:359] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.140918 12701 raft_consensus.cc:385] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.141256 12701 raft_consensus.cc:740] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.142067 12701 consensus_queue.cc:260] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.143424 12703 tablet_bootstrap.cc:654] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.147423 12537 heartbeater.cc:499] Master 127.0.107.254:45273 was elected leader, sending a full tablet report...
I20260709 13:06:17.148303 12701 ts_tablet_manager.cc:1434] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 13:06:17.148947 12702 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: No bootstrap required, opened a new log
I20260709 13:06:17.149617 12702 ts_tablet_manager.cc:1403] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.010s
I20260709 13:06:17.150527 12701 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: Bootstrap starting.
I20260709 13:06:17.152868 12702 raft_consensus.cc:359] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.153746 12702 raft_consensus.cc:385] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.154093 12702 raft_consensus.cc:740] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff390ca35814b52bdee91878c62fe2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.155097 12702 consensus_queue.cc:260] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.156571 12703 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: No bootstrap required, opened a new log
I20260709 13:06:17.157176 12703 ts_tablet_manager.cc:1403] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
I20260709 13:06:17.159703 12701 tablet_bootstrap.cc:654] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.160626 12703 raft_consensus.cc:359] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.161496 12703 raft_consensus.cc:385] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.161824 12703 raft_consensus.cc:740] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.162812 12703 consensus_queue.cc:260] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.166445 12686 heartbeater.cc:499] Master 127.0.107.254:45273 was elected leader, sending a full tablet report...
I20260709 13:06:17.166749 12702 ts_tablet_manager.cc:1434] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.006s
I20260709 13:06:17.168224 12702 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: Bootstrap starting.
I20260709 13:06:17.170011 12703 ts_tablet_manager.cc:1434] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260709 13:06:17.170842 12612 heartbeater.cc:499] Master 127.0.107.254:45273 was elected leader, sending a full tablet report...
I20260709 13:06:17.177707 12702 tablet_bootstrap.cc:654] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.182061 12703 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: Bootstrap starting.
I20260709 13:06:17.187510 12702 tablet_bootstrap.cc:492] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: No bootstrap required, opened a new log
I20260709 13:06:17.187824 12701 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: No bootstrap required, opened a new log
I20260709 13:06:17.188246 12702 ts_tablet_manager.cc:1403] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.008s
I20260709 13:06:17.188563 12701 ts_tablet_manager.cc:1403] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.019s
I20260709 13:06:17.189946 12703 tablet_bootstrap.cc:654] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.191932 12701 raft_consensus.cc:359] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.191876 12702 raft_consensus.cc:359] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.192848 12701 raft_consensus.cc:385] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.192979 12702 raft_consensus.cc:385] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.193228 12701 raft_consensus.cc:740] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.193482 12702 raft_consensus.cc:740] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff390ca35814b52bdee91878c62fe2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.194170 12701 consensus_queue.cc:260] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.194429 12702 consensus_queue.cc:260] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.196442 12701 ts_tablet_manager.cc:1434] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:17.196841 12702 ts_tablet_manager.cc:1434] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:06:17.197981 12703 tablet_bootstrap.cc:492] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: No bootstrap required, opened a new log
I20260709 13:06:17.198572 12703 ts_tablet_manager.cc:1403] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:06:17.201812 12703 raft_consensus.cc:359] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.202467 12703 raft_consensus.cc:385] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.202768 12703 raft_consensus.cc:740] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.203444 12703 consensus_queue.cc:260] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.205662 12703 ts_tablet_manager.cc:1434] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:17.369333 12708 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.369926 12708 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.371480 12710 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.372162 12710 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.372639 12708 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:17.378659 12710 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.379949 12706 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.380614 12706 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.390312 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" is_pre_election: true
I20260709 13:06:17.391021 12661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d" is_pre_election: true
I20260709 13:06:17.391896 12661 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 0.
I20260709 13:06:17.393198 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d" is_pre_election: true
I20260709 13:06:17.393435 12553 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff390ca35814b52bdee91878c62fe2d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.394001 12512 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 0.
I20260709 13:06:17.391229 12587 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 0.
I20260709 13:06:17.395430 12710 raft_consensus.cc:2804] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.396896 12706 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:17.398494 12710 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.399085 12710 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.395741 12625 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 6ff390ca35814b52bdee91878c62fe2d; no voters: 
I20260709 13:06:17.404428 12708 raft_consensus.cc:2804] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.404882 12708 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.404793 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d" is_pre_election: true
I20260709 13:06:17.405584 12708 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.405895 12512 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 0.
I20260709 13:06:17.408710 12710 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.411580 12710 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 election: Requested vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.412839 12661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:17.412837 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d"
I20260709 13:06:17.414327 12512 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.417327 12708 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.420444 12661 raft_consensus.cc:2393] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c9a6f3d6f254996bafa1d7af50135fe in current term 1: Already voted for candidate 6ff390ca35814b52bdee91878c62fe2d in this term.
I20260709 13:06:17.421168 12708 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 election: Requested vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:17.421953 12708 raft_consensus.cc:493] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.422511 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
I20260709 13:06:17.423511 12587 raft_consensus.cc:2393] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ff390ca35814b52bdee91878c62fe2d in current term 1: Already voted for candidate 7c9a6f3d6f254996bafa1d7af50135fe in this term.
I20260709 13:06:17.424633 12511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d"
I20260709 13:06:17.426344 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d" is_pre_election: true
I20260709 13:06:17.426867 12625 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ff390ca35814b52bdee91878c62fe2d; no voters: 12c6255369984d74afa72e35e0f6129d, 7c9a6f3d6f254996bafa1d7af50135fe
I20260709 13:06:17.427338 12660 raft_consensus.cc:2393] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12c6255369984d74afa72e35e0f6129d in current term 1: Already voted for candidate 6ff390ca35814b52bdee91878c62fe2d in this term.
I20260709 13:06:17.428095 12512 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 1.
I20260709 13:06:17.428493 12719 raft_consensus.cc:2749] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:17.422577 12708 raft_consensus.cc:515] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.429656 12551 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 6ff390ca35814b52bdee91878c62fe2d
I20260709 13:06:17.430500 12710 raft_consensus.cc:2804] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.432861 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6468455d947e4baa8fe1291c19904ae3" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d" is_pre_election: true
I20260709 13:06:17.432942 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6468455d947e4baa8fe1291c19904ae3" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" is_pre_election: true
I20260709 13:06:17.433986 12587 raft_consensus.cc:2468] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 0.
I20260709 13:06:17.430781 12708 leader_election.cc:290] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:17.435595 12625 leader_election.cc:304] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff390ca35814b52bdee91878c62fe2d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.435611 12710 raft_consensus.cc:697] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Becoming Leader. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Running, Role: LEADER
I20260709 13:06:17.436671 12708 raft_consensus.cc:2804] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.434013 12512 raft_consensus.cc:2468] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 0.
I20260709 13:06:17.437279 12708 raft_consensus.cc:493] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.437357 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" is_pre_election: true
I20260709 13:06:17.437801 12708 raft_consensus.cc:3060] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.437593 12710 consensus_queue.cc:237] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:17.438949 12476 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d; no voters: 6ff390ca35814b52bdee91878c62fe2d, 7c9a6f3d6f254996bafa1d7af50135fe
I20260709 13:06:17.439735 12706 raft_consensus.cc:2749] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:17.445732 12708 raft_consensus.cc:515] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.448354 12708 leader_election.cc:290] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 election: Requested vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:17.447810 12410 catalog_manager.cc:5697] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.449054 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6468455d947e4baa8fe1291c19904ae3" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d"
I20260709 13:06:17.449731 12512 raft_consensus.cc:3060] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.449481 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6468455d947e4baa8fe1291c19904ae3" candidate_uuid: "6ff390ca35814b52bdee91878c62fe2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
I20260709 13:06:17.450271 12587 raft_consensus.cc:3060] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.457700 12512 raft_consensus.cc:2468] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 1.
I20260709 13:06:17.457875 12587 raft_consensus.cc:2468] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff390ca35814b52bdee91878c62fe2d in term 1.
I20260709 13:06:17.459107 12625 leader_election.cc:304] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 6ff390ca35814b52bdee91878c62fe2d; no voters: 
I20260709 13:06:17.459895 12708 raft_consensus.cc:2804] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.460346 12708 raft_consensus.cc:697] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 1 LEADER]: Becoming Leader. State: Replica: 6ff390ca35814b52bdee91878c62fe2d, State: Running, Role: LEADER
I20260709 13:06:17.460999 12708 consensus_queue.cc:237] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.468365 12410 catalog_manager.cc:5697] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6ff390ca35814b52bdee91878c62fe2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.589854 12410 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47784:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:17.592087 12410 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:06:17.603101 12458 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:17.616510 12501 tablet_service.cc:1511] Processing CreateTablet for tablet b4685dae7a3e4a39a9b782d1e988aca9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.617980 12501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4685dae7a3e4a39a9b782d1e988aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.618012 12577 tablet_service.cc:1511] Processing CreateTablet for tablet b4685dae7a3e4a39a9b782d1e988aca9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.619643 12577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4685dae7a3e4a39a9b782d1e988aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.619410 12651 tablet_service.cc:1511] Processing CreateTablet for tablet b4685dae7a3e4a39a9b782d1e988aca9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:17.620889 12651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4685dae7a3e4a39a9b782d1e988aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.623847 12650 tablet_service.cc:1511] Processing CreateTablet for tablet d9e71650f1554c55932fe0a6b1c27679 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.625306 12650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e71650f1554c55932fe0a6b1c27679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.625836 12576 tablet_service.cc:1511] Processing CreateTablet for tablet d9e71650f1554c55932fe0a6b1c27679 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.627079 12576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e71650f1554c55932fe0a6b1c27679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.632849 12502 tablet_service.cc:1511] Processing CreateTablet for tablet d9e71650f1554c55932fe0a6b1c27679 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b5493e08fdc8457cba2132f30b7b6a56]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:17.634308 12502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e71650f1554c55932fe0a6b1c27679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.637023 12701 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: Bootstrap starting.
I20260709 13:06:17.643368 12701 tablet_bootstrap.cc:654] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.644263 12703 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: Bootstrap starting.
I20260709 13:06:17.652885 12703 tablet_bootstrap.cc:654] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.668182 12701 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: No bootstrap required, opened a new log
I20260709 13:06:17.668382 12702 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: Bootstrap starting.
I20260709 13:06:17.668865 12701 ts_tablet_manager.cc:1403] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.023s
I20260709 13:06:17.672082 12701 raft_consensus.cc:359] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.672986 12701 raft_consensus.cc:385] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.673377 12701 raft_consensus.cc:740] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.674288 12701 consensus_queue.cc:260] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.675854 12702 tablet_bootstrap.cc:654] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.676754 12701 ts_tablet_manager.cc:1434] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:06:17.684482 12703 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: No bootstrap required, opened a new log
I20260709 13:06:17.685380 12703 ts_tablet_manager.cc:1403] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.019s
I20260709 13:06:17.686372 12701 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: Bootstrap starting.
I20260709 13:06:17.689662 12703 raft_consensus.cc:359] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.690302 12702 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: No bootstrap required, opened a new log
I20260709 13:06:17.690526 12703 raft_consensus.cc:385] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.690910 12702 ts_tablet_manager.cc:1403] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.010s
I20260709 13:06:17.690964 12703 raft_consensus.cc:740] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.691975 12703 consensus_queue.cc:260] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.694149 12702 raft_consensus.cc:359] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.694991 12702 raft_consensus.cc:385] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.695516 12702 raft_consensus.cc:740] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff390ca35814b52bdee91878c62fe2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.696333 12703 ts_tablet_manager.cc:1434] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:17.696455 12702 consensus_queue.cc:260] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.697374 12701 tablet_bootstrap.cc:654] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.697446 12703 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Bootstrap starting.
I20260709 13:06:17.701746 12702 ts_tablet_manager.cc:1434] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:17.702984 12702 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: Bootstrap starting.
I20260709 13:06:17.705235 12703 tablet_bootstrap.cc:654] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.712734 12701 tablet_bootstrap.cc:492] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: No bootstrap required, opened a new log
I20260709 13:06:17.713212 12701 ts_tablet_manager.cc:1403] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.001s
I20260709 13:06:17.714584 12703 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: No bootstrap required, opened a new log
I20260709 13:06:17.715116 12703 ts_tablet_manager.cc:1403] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:06:17.716195 12701 raft_consensus.cc:359] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.716982 12701 raft_consensus.cc:385] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.717283 12701 raft_consensus.cc:740] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.718006 12703 raft_consensus.cc:359] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.718010 12701 consensus_queue.cc:260] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.718819 12702 tablet_bootstrap.cc:654] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.718932 12703 raft_consensus.cc:385] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.719481 12703 raft_consensus.cc:740] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.720587 12701 ts_tablet_manager.cc:1434] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:06:17.720245 12703 consensus_queue.cc:260] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.722815 12703 ts_tablet_manager.cc:1434] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:17.729727 12702 tablet_bootstrap.cc:492] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: No bootstrap required, opened a new log
I20260709 13:06:17.730356 12702 ts_tablet_manager.cc:1403] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.001s
I20260709 13:06:17.733738 12702 raft_consensus.cc:359] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.734540 12702 raft_consensus.cc:385] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.734884 12702 raft_consensus.cc:740] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff390ca35814b52bdee91878c62fe2d, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.735554 12702 consensus_queue.cc:260] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.737859 12702 ts_tablet_manager.cc:1434] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:17.809810 12706 raft_consensus.cc:493] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.810369 12706 raft_consensus.cc:515] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.812036 12706 leader_election.cc:290] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.812826 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" is_pre_election: true
I20260709 13:06:17.813027 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d" is_pre_election: true
I20260709 13:06:17.813504 12587 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 0.
I20260709 13:06:17.813661 12660 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 0.
I20260709 13:06:17.814704 12476 leader_election.cc:304] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.815366 12706 raft_consensus.cc:2804] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.815798 12706 raft_consensus.cc:493] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.816207 12706 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.822045 12706 raft_consensus.cc:515] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.823837 12706 leader_election.cc:290] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 election: Requested vote from peers 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.824793 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
I20260709 13:06:17.824985 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:17.825536 12587 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.825567 12660 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.832369 12710 consensus_queue.cc:1048] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:17.833335 12587 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 1.
I20260709 13:06:17.833389 12660 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 1.
I20260709 13:06:17.834903 12476 leader_election.cc:304] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.835742 12706 raft_consensus.cc:2804] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.839833 12706 raft_consensus.cc:697] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 LEADER]: Becoming Leader. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Running, Role: LEADER
I20260709 13:06:17.841284 12706 consensus_queue.cc:237] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.847177 12710 consensus_queue.cc:1048] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:17.851063 12720 raft_consensus.cc:493] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.851644 12720 raft_consensus.cc:515] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.853619 12410 catalog_manager.cc:5697] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d reported cstate change: term changed from 0 to 1, leader changed from <none> to 12c6255369984d74afa72e35e0f6129d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "12c6255369984d74afa72e35e0f6129d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.855069 12511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d" is_pre_election: true
I20260709 13:06:17.855345 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d" is_pre_election: true
I20260709 13:06:17.856096 12660 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 0.
I20260709 13:06:17.856010 12511 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 0.
I20260709 13:06:17.857597 12553 leader_election.cc:304] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff390ca35814b52bdee91878c62fe2d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.858397 12720 leader_election.cc:290] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.858487 12710 raft_consensus.cc:2804] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.859084 12710 raft_consensus.cc:493] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.859524 12710 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.868526 12710 raft_consensus.cc:515] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.871132 12710 leader_election.cc:290] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 election: Requested vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:17.871655 12511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c6255369984d74afa72e35e0f6129d"
I20260709 13:06:17.872465 12511 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.871655 12661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:17.876417 12661 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.880931 12511 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 1.
I20260709 13:06:17.882230 12551 leader_election.cc:304] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:17.883069 12710 raft_consensus.cc:2804] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.883579 12710 raft_consensus.cc:697] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Becoming Leader. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Running, Role: LEADER
I20260709 13:06:17.884292 12661 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 1.
I20260709 13:06:17.884490 12710 consensus_queue.cc:237] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:17.894976 12410 catalog_manager.cc:5697] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.927621 12708 consensus_queue.cc:1048] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:17.939996 12744 consensus_queue.cc:1048] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:17.968145 12587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4"
dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
mode: GRACEFUL
new_leader_uuid: "12c6255369984d74afa72e35e0f6129d"
 from {username='slave'} at 127.0.0.1:47514
I20260709 13:06:17.968694 12587 raft_consensus.cc:606] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Received request to transfer leadership to TS 12c6255369984d74afa72e35e0f6129d
I20260709 13:06:17.980191 12587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679"
dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
mode: GRACEFUL
new_leader_uuid: "12c6255369984d74afa72e35e0f6129d"
 from {username='slave'} at 127.0.0.1:47514
I20260709 13:06:17.980700 12587 raft_consensus.cc:606] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Received request to transfer leadership to TS 12c6255369984d74afa72e35e0f6129d
I20260709 13:06:18.239704 12706 consensus_queue.cc:1048] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.253746 12736 consensus_queue.cc:1048] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.333102 12710 consensus_queue.cc:1048] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.339797 12720 raft_consensus.cc:993] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: : Instructing follower 12c6255369984d74afa72e35e0f6129d to start an election
I20260709 13:06:18.340739 12710 raft_consensus.cc:1081] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Signalling peer 12c6255369984d74afa72e35e0f6129d to start an election
I20260709 13:06:18.343801 12512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679"
dest_uuid: "12c6255369984d74afa72e35e0f6129d"
 from {username='slave'} at 127.0.0.1:58198
I20260709 13:06:18.344393 12512 raft_consensus.cc:493] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.344781 12512 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.348232 12752 consensus_queue.cc:1048] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:18.352365 12512 raft_consensus.cc:515] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.354790 12512 leader_election.cc:290] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 2 election: Requested vote from peers 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:18.356307 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:18.357024 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e71650f1554c55932fe0a6b1c27679" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
I20260709 13:06:18.357053 12660 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.357719 12587 raft_consensus.cc:3055] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.358031 12587 raft_consensus.cc:740] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Running, Role: LEADER
I20260709 13:06:18.358760 12587 consensus_queue.cc:260] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.359745 12587 raft_consensus.cc:3060] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.364104 12660 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 2.
I20260709 13:06:18.365460 12478 leader_election.cc:304] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 6ff390ca35814b52bdee91878c62fe2d; no voters: 
I20260709 13:06:18.365550 12587 raft_consensus.cc:2468] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 2.
I20260709 13:06:18.366429 12736 raft_consensus.cc:2804] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.367023 12736 raft_consensus.cc:697] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 2 LEADER]: Becoming Leader. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Running, Role: LEADER
I20260709 13:06:18.368045 12736 consensus_queue.cc:237] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.375128 12410 catalog_manager.cc:5697] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d reported cstate change: term changed from 1 to 2, leader changed from 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194) to 12c6255369984d74afa72e35e0f6129d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "12c6255369984d74afa72e35e0f6129d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.377246 12710 raft_consensus.cc:993] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: : Instructing follower 12c6255369984d74afa72e35e0f6129d to start an election
I20260709 13:06:18.377770 12752 raft_consensus.cc:1081] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Signalling peer 12c6255369984d74afa72e35e0f6129d to start an election
I20260709 13:06:18.379534 12512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4"
dest_uuid: "12c6255369984d74afa72e35e0f6129d"
 from {username='slave'} at 127.0.0.1:58198
I20260709 13:06:18.380155 12512 raft_consensus.cc:493] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.380527 12512 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.386312 12512 raft_consensus.cc:515] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:18.387995 12512 leader_election.cc:290] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 2 election: Requested vote from peers 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145), 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827)
I20260709 13:06:18.389178 12587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
I20260709 13:06:18.389155 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59867d71f9644b6999c665086e02e2c4" candidate_uuid: "12c6255369984d74afa72e35e0f6129d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:18.390017 12587 raft_consensus.cc:3055] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.390072 12660 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.390441 12587 raft_consensus.cc:740] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Running, Role: LEADER
I20260709 13:06:18.391337 12587 consensus_queue.cc:260] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:18.392680 12587 raft_consensus.cc:3060] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.397595 12660 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 2.
I20260709 13:06:18.399050 12478 leader_election.cc:304] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c6255369984d74afa72e35e0f6129d, 6ff390ca35814b52bdee91878c62fe2d; no voters: 
I20260709 13:06:18.399909 12736 raft_consensus.cc:2804] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.399930 12587 raft_consensus.cc:2468] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c6255369984d74afa72e35e0f6129d in term 2.
I20260709 13:06:18.400442 12736 raft_consensus.cc:697] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 2 LEADER]: Becoming Leader. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Running, Role: LEADER
I20260709 13:06:18.401226 12736 consensus_queue.cc:237] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } }
I20260709 13:06:18.408089 12410 catalog_manager.cc:5697] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d reported cstate change: term changed from 1 to 2, leader changed from 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194) to 12c6255369984d74afa72e35e0f6129d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "12c6255369984d74afa72e35e0f6129d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.702191   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:06:18.704201   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:06:18.704691   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:06:18.715917 12512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9"
dest_uuid: "12c6255369984d74afa72e35e0f6129d"
mode: GRACEFUL
new_leader_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
 from {username='slave'} at 127.0.0.1:52216
I20260709 13:06:18.716495 12512 raft_consensus.cc:606] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 LEADER]: Received request to transfer leadership to TS 7c9a6f3d6f254996bafa1d7af50135fe
I20260709 13:06:18.717605   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:06:18.783221 12660 raft_consensus.cc:1275] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Refusing update from remote peer 12c6255369984d74afa72e35e0f6129d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.784448 12736 consensus_queue.cc:1048] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.794211 12587 raft_consensus.cc:1275] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Refusing update from remote peer 12c6255369984d74afa72e35e0f6129d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.796605 12749 consensus_queue.cc:1048] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.804224 12736 raft_consensus.cc:993] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: : Instructing follower 7c9a6f3d6f254996bafa1d7af50135fe to start an election
I20260709 13:06:18.804816 12749 raft_consensus.cc:1081] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 LEADER]: Signalling peer 7c9a6f3d6f254996bafa1d7af50135fe to start an election
I20260709 13:06:18.806576 12586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9"
dest_uuid: "7c9a6f3d6f254996bafa1d7af50135fe"
 from {username='slave'} at 127.0.0.1:47500
I20260709 13:06:18.807116 12586 raft_consensus.cc:493] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.807494 12586 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.814150 12586 raft_consensus.cc:515] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.816646 12586 leader_election.cc:290] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 2 election: Requested vote from peers 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273), 6ff390ca35814b52bdee91878c62fe2d (127.0.107.195:39145)
I20260709 13:06:18.816895 12512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12c6255369984d74afa72e35e0f6129d"
I20260709 13:06:18.817286 12660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4685dae7a3e4a39a9b782d1e988aca9" candidate_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ff390ca35814b52bdee91878c62fe2d"
I20260709 13:06:18.817704 12512 raft_consensus.cc:3055] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.817958 12660 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.818142 12512 raft_consensus.cc:740] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12c6255369984d74afa72e35e0f6129d, State: Running, Role: LEADER
I20260709 13:06:18.819098 12512 consensus_queue.cc:260] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.820555 12512 raft_consensus.cc:3060] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.826210 12660 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 2.
I20260709 13:06:18.827836 12553 leader_election.cc:304] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff390ca35814b52bdee91878c62fe2d, 7c9a6f3d6f254996bafa1d7af50135fe; no voters: 
I20260709 13:06:18.828613 12512 raft_consensus.cc:2468] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9a6f3d6f254996bafa1d7af50135fe in term 2.
I20260709 13:06:18.828779 12752 raft_consensus.cc:2804] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.829385 12752 raft_consensus.cc:697] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 LEADER]: Becoming Leader. State: Replica: 7c9a6f3d6f254996bafa1d7af50135fe, State: Running, Role: LEADER
I20260709 13:06:18.830246 12752 consensus_queue.cc:237] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } }
I20260709 13:06:18.836031 12410 catalog_manager.cc:5697] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe reported cstate change: term changed from 1 to 2, leader changed from 12c6255369984d74afa72e35e0f6129d (127.0.107.193) to 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c6255369984d74afa72e35e0f6129d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.855470   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:18.859984 12586 raft_consensus.cc:1275] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Refusing update from remote peer 12c6255369984d74afa72e35e0f6129d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.861107 12660 raft_consensus.cc:1275] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Refusing update from remote peer 12c6255369984d74afa72e35e0f6129d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.861464 12749 consensus_queue.cc:1048] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9a6f3d6f254996bafa1d7af50135fe" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.862651 12736 consensus_queue.cc:1048] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff390ca35814b52bdee91878c62fe2d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:18.901954   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:18.902583   431 tablet_replica.cc:333] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d: stopping tablet replica
I20260709 13:06:18.903363   431 raft_consensus.cc:2243] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.904270   431 raft_consensus.cc:2272] T d9e71650f1554c55932fe0a6b1c27679 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.906692   431 tablet_replica.cc:333] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d: stopping tablet replica
I20260709 13:06:18.907224   431 raft_consensus.cc:2243] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.907601   431 raft_consensus.cc:2272] T b4685dae7a3e4a39a9b782d1e988aca9 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.909564   431 tablet_replica.cc:333] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d: stopping tablet replica
I20260709 13:06:18.910058   431 raft_consensus.cc:2243] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.910452   431 raft_consensus.cc:2272] T 6468455d947e4baa8fe1291c19904ae3 P 12c6255369984d74afa72e35e0f6129d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.912479   431 tablet_replica.cc:333] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d: stopping tablet replica
I20260709 13:06:18.912947   431 raft_consensus.cc:2243] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.913652   431 raft_consensus.cc:2272] T 59867d71f9644b6999c665086e02e2c4 P 12c6255369984d74afa72e35e0f6129d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.927608   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:06:18.939215   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:06:18.959290   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:18.960088   431 tablet_replica.cc:333] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: stopping tablet replica
I20260709 13:06:18.960750   431 raft_consensus.cc:2243] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.961404   431 pending_rounds.cc:70] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Trying to abort 1 pending ops.
I20260709 13:06:18.961621   431 pending_rounds.cc:77] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635343696171008 op_type: NO_OP noop_request { }
I20260709 13:06:18.961952   431 raft_consensus.cc:2889] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:18.962245   431 raft_consensus.cc:2272] T b4685dae7a3e4a39a9b782d1e988aca9 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:18.963347 12625 consensus_peers.cc:597] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d -> Peer 12c6255369984d74afa72e35e0f6129d (127.0.107.193:40273): Couldn't send request to peer 12c6255369984d74afa72e35e0f6129d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:18.964792   431 tablet_replica.cc:333] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe: stopping tablet replica
I20260709 13:06:18.965370   431 raft_consensus.cc:2243] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.965750   431 raft_consensus.cc:2272] T d9e71650f1554c55932fe0a6b1c27679 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.968178   431 tablet_replica.cc:333] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe: stopping tablet replica
I20260709 13:06:18.968634   431 raft_consensus.cc:2243] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.968986   431 raft_consensus.cc:2272] T 6468455d947e4baa8fe1291c19904ae3 P 7c9a6f3d6f254996bafa1d7af50135fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.970686   431 tablet_replica.cc:333] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe: stopping tablet replica
I20260709 13:06:18.971108   431 raft_consensus.cc:2243] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.971477   431 raft_consensus.cc:2272] T 59867d71f9644b6999c665086e02e2c4 P 7c9a6f3d6f254996bafa1d7af50135fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.994874   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:06:19.007022   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:19.026904 12625 consensus_peers.cc:597] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d -> Peer 7c9a6f3d6f254996bafa1d7af50135fe (127.0.107.194:40827): Couldn't send request to peer 7c9a6f3d6f254996bafa1d7af50135fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40827: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:19.028383   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:19.029126   431 tablet_replica.cc:333] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d: stopping tablet replica
I20260709 13:06:19.029742   431 raft_consensus.cc:2243] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.030205   431 raft_consensus.cc:2272] T b4685dae7a3e4a39a9b782d1e988aca9 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.032531   431 tablet_replica.cc:333] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d: stopping tablet replica
I20260709 13:06:19.033078   431 raft_consensus.cc:2243] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.033511   431 raft_consensus.cc:2272] T d9e71650f1554c55932fe0a6b1c27679 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.035725   431 tablet_replica.cc:333] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d: stopping tablet replica
I20260709 13:06:19.036381   431 raft_consensus.cc:2243] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.036823   431 raft_consensus.cc:2272] T 59867d71f9644b6999c665086e02e2c4 P 6ff390ca35814b52bdee91878c62fe2d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.039378   431 tablet_replica.cc:333] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d: stopping tablet replica
I20260709 13:06:19.039937   431 raft_consensus.cc:2243] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.040724   431 raft_consensus.cc:2272] T 6468455d947e4baa8fe1291c19904ae3 P 6ff390ca35814b52bdee91878c62fe2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.063323   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:19.075467   431 master.cc:562] Master@127.0.107.254:45273 shutting down...
I20260709 13:06:19.091292   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.091866   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.092164   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P f25741850fde4b9692abaab6e7150f4f: stopping tablet replica
I20260709 13:06:19.103909   431 master.cc:584] Master@127.0.107.254:45273 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2667 ms)
[----------] 6 tests from LeaderRebalancerTest (386893 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:06:19.132967   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39485
I20260709 13:06:19.134135   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:19.139297 12764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.139362 12765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.141153 12767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.141937   431 server_base.cc:1061] running on GCE node
I20260709 13:06:19.142864   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:19.143043   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:19.143301   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602379143279 us; error 0 us; skew 500 ppm
I20260709 13:06:19.144039   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:19.146407   431 webserver.cc:533] Webserver started at http://127.0.107.254:38197/ using document root <none> and password file <none>
I20260709 13:06:19.146854   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:19.147009   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:19.147238   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:19.148406   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "48b59da8fab84ad4a5e3da5aaa583a17"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.153153   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:19.156553 12772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.157388   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:19.157704   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "48b59da8fab84ad4a5e3da5aaa583a17"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.158006   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:19.174697   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:19.175834   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:19.211069   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39485
I20260709 13:06:19.211174 12823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39485 every 8 connection(s)
I20260709 13:06:19.214795 12824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.225798 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: Bootstrap starting.
I20260709 13:06:19.230477 12824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.234506 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: No bootstrap required, opened a new log
I20260709 13:06:19.236644 12824 raft_consensus.cc:359] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER }
I20260709 13:06:19.237018 12824 raft_consensus.cc:385] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:19.237290 12824 raft_consensus.cc:740] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b59da8fab84ad4a5e3da5aaa583a17, State: Initialized, Role: FOLLOWER
I20260709 13:06:19.237946 12824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER }
I20260709 13:06:19.238430 12824 raft_consensus.cc:399] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:19.238672 12824 raft_consensus.cc:493] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:19.238998 12824 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:19.246995 12824 raft_consensus.cc:515] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER }
I20260709 13:06:19.247759 12824 leader_election.cc:304] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48b59da8fab84ad4a5e3da5aaa583a17; no voters: 
I20260709 13:06:19.249224 12824 leader_election.cc:290] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:19.249562 12827 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:19.254168 12824 sys_catalog.cc:565] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:19.254575 12827 raft_consensus.cc:697] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 1 LEADER]: Becoming Leader. State: Replica: 48b59da8fab84ad4a5e3da5aaa583a17, State: Running, Role: LEADER
I20260709 13:06:19.255494 12827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER }
I20260709 13:06:19.259390 12829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER } }
I20260709 13:06:19.259387 12828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48b59da8fab84ad4a5e3da5aaa583a17. Latest consensus state: current_term: 1 leader_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b59da8fab84ad4a5e3da5aaa583a17" member_type: VOTER } }
I20260709 13:06:19.260046 12829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:19.260113 12828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:19.265594 12833 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:19.270912 12833 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:19.273346   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:19.279971 12833 catalog_manager.cc:1383] Generated new cluster ID: dc25974d5a734855a219e6eebf6c6dea
I20260709 13:06:19.280288 12833 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:19.309590 12833 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:19.310947 12833 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:19.329607 12833 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: Generated new TSK 0
I20260709 13:06:19.330273 12833 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:19.340327   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:19.346102 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.347245 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.349956 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.350067   431 server_base.cc:1061] running on GCE node
I20260709 13:06:19.351054   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:19.351240   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:19.351459   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602379351423 us; error 0 us; skew 500 ppm
I20260709 13:06:19.352072   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:19.354408   431 webserver.cc:533] Webserver started at http://127.0.107.193:36175/ using document root <none> and password file <none>
I20260709 13:06:19.354910   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:19.355108   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:19.355422   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:19.356621   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "a8ce4463a21a47afa02aa37a1d09441a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.361156   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:06:19.364596 12853 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.365352   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:19.365670   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "a8ce4463a21a47afa02aa37a1d09441a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.365942   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:19.387012   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:19.388339   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:19.389947   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:19.392251   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:19.392481   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.392743   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:19.392911   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.432045   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38443
I20260709 13:06:19.432157 12915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38443 every 8 connection(s)
I20260709 13:06:19.437122   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:19.444542 12921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.445765 12920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.448143   431 server_base.cc:1061] running on GCE node
W20260709 13:06:19.448904 12923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.449640 12916 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39485
I20260709 13:06:19.450100 12916 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:19.450196   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:19.450506   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:19.450714   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602379450695 us; error 0 us; skew 500 ppm
I20260709 13:06:19.450978 12916 heartbeater.cc:507] Master 127.0.107.254:39485 requested a full tablet report, sending...
I20260709 13:06:19.451511   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:19.453543 12789 ts_manager.cc:194] Registered new tserver with Master: a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443)
I20260709 13:06:19.454278   431 webserver.cc:533] Webserver started at http://127.0.107.194:42079/ using document root <none> and password file <none>
I20260709 13:06:19.454813   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:19.455003   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:19.455291   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:19.455644 12789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58598
I20260709 13:06:19.456568   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "579916bf94474548a6de9b8ea1cffb4a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.461617   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:06:19.464749 12928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.465557   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:06:19.465867   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "579916bf94474548a6de9b8ea1cffb4a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.466156   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:19.479643   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:19.480851   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:19.482424   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:19.484709   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:19.484943   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.485188   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:19.485360   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.523262   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43447
I20260709 13:06:19.523352 12990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43447 every 8 connection(s)
I20260709 13:06:19.528144   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:19.534399 12994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:19.535029 12995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.537676   431 server_base.cc:1061] running on GCE node
W20260709 13:06:19.538543 12998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:19.539541   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:19.539839   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:19.539997 12991 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39485
I20260709 13:06:19.540073   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602379540044 us; error 0 us; skew 500 ppm
I20260709 13:06:19.540539 12991 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:19.540937   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:19.541332 12991 heartbeater.cc:507] Master 127.0.107.254:39485 requested a full tablet report, sending...
I20260709 13:06:19.543629 12789 ts_manager.cc:194] Registered new tserver with Master: 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:19.543687   431 webserver.cc:533] Webserver started at http://127.0.107.195:38757/ using document root <none> and password file <none>
I20260709 13:06:19.544467   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:19.544736   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:19.545056   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:19.545580 12789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58606
I20260709 13:06:19.546536   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.551477   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:06:19.554770 13002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.555583   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:19.555935   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
format_stamp: "Formatted at 2026-07-09 13:06:19 on dist-test-slave-kgkf"
I20260709 13:06:19.556207   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:19.571130   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:19.572329   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:19.573735   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:19.576189   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:19.576385   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.576697   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:19.576862   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:19.617273   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:38223
I20260709 13:06:19.617391 13064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:38223 every 8 connection(s)
I20260709 13:06:19.630877 13065 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39485
I20260709 13:06:19.631331 13065 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:19.632032 13065 heartbeater.cc:507] Master 127.0.107.254:39485 requested a full tablet report, sending...
I20260709 13:06:19.633975 12789 ts_manager.cc:194] Registered new tserver with Master: 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:19.634114   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013649383s
I20260709 13:06:19.635474 12789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58608
I20260709 13:06:19.659409 12789 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58614:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:19.662161 12789 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:19.744827 12956 tablet_service.cc:1511] Processing CreateTablet for tablet 63548b1cfe3b41aea9086ab46cf5df6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:19.746388 12956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63548b1cfe3b41aea9086ab46cf5df6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.746497 12955 tablet_service.cc:1511] Processing CreateTablet for tablet 785b40ccad454ddaadb93567cb933a3f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:19.748028 12955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 785b40ccad454ddaadb93567cb933a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.752912 12953 tablet_service.cc:1511] Processing CreateTablet for tablet 83f3533d969441f28f11504f41dd42cb (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:19.754490 12953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f3533d969441f28f11504f41dd42cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.756986 12952 tablet_service.cc:1511] Processing CreateTablet for tablet 52c0f24d250a4bcbae0d8e08f2bd813e (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:19.757181 12951 tablet_service.cc:1511] Processing CreateTablet for tablet 1bbc272093324888a170e4c87badda6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:19.751631 12954 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:19.758601 12952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c0f24d250a4bcbae0d8e08f2bd813e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.760871 12949 tablet_service.cc:1511] Processing CreateTablet for tablet 29882fd52664468c848fd7e5c82b7907 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:19.762042 12951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bbc272093324888a170e4c87badda6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.763435 12948 tablet_service.cc:1511] Processing CreateTablet for tablet c7eea2e74a5e4380b6fef269d5030e4f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:19.764185 12954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.765054 12948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7eea2e74a5e4380b6fef269d5030e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.758802 12950 tablet_service.cc:1511] Processing CreateTablet for tablet fb56c49b42064eea93557f6c053db8b9 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:19.767882 12950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb56c49b42064eea93557f6c053db8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.775897 12949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29882fd52664468c848fd7e5c82b7907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.778266 12879 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:19.779971 12881 tablet_service.cc:1511] Processing CreateTablet for tablet 63548b1cfe3b41aea9086ab46cf5df6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:19.780963 12878 tablet_service.cc:1511] Processing CreateTablet for tablet 83f3533d969441f28f11504f41dd42cb (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:19.783341 12877 tablet_service.cc:1511] Processing CreateTablet for tablet 52c0f24d250a4bcbae0d8e08f2bd813e (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:19.784785 12874 tablet_service.cc:1511] Processing CreateTablet for tablet 29882fd52664468c848fd7e5c82b7907 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:19.785773 12876 tablet_service.cc:1511] Processing CreateTablet for tablet 1bbc272093324888a170e4c87badda6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:19.787015 13030 tablet_service.cc:1511] Processing CreateTablet for tablet 63548b1cfe3b41aea9086ab46cf5df6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:19.788400 13030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63548b1cfe3b41aea9086ab46cf5df6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.792889 13029 tablet_service.cc:1511] Processing CreateTablet for tablet 785b40ccad454ddaadb93567cb933a3f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:19.794328 13029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 785b40ccad454ddaadb93567cb933a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.796888 13080 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:19.797971 13027 tablet_service.cc:1511] Processing CreateTablet for tablet 83f3533d969441f28f11504f41dd42cb (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:19.799435 13027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f3533d969441f28f11504f41dd42cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.800895 13026 tablet_service.cc:1511] Processing CreateTablet for tablet 52c0f24d250a4bcbae0d8e08f2bd813e (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:19.803185 13025 tablet_service.cc:1511] Processing CreateTablet for tablet 1bbc272093324888a170e4c87badda6c (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:19.806384 12875 tablet_service.cc:1511] Processing CreateTablet for tablet fb56c49b42064eea93557f6c053db8b9 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:19.796891 13028 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:19.807858 12880 tablet_service.cc:1511] Processing CreateTablet for tablet 785b40ccad454ddaadb93567cb933a3f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:19.810426 13025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bbc272093324888a170e4c87badda6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.811158 13026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c0f24d250a4bcbae0d8e08f2bd813e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.810863 13023 tablet_service.cc:1511] Processing CreateTablet for tablet 29882fd52664468c848fd7e5c82b7907 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:19.787345 12879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.810463 13024 tablet_service.cc:1511] Processing CreateTablet for tablet fb56c49b42064eea93557f6c053db8b9 (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:19.812387 13023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29882fd52664468c848fd7e5c82b7907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.813148 12881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63548b1cfe3b41aea9086ab46cf5df6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.816082 12874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29882fd52664468c848fd7e5c82b7907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.816665 13022 tablet_service.cc:1511] Processing CreateTablet for tablet c7eea2e74a5e4380b6fef269d5030e4f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:19.817015 12873 tablet_service.cc:1511] Processing CreateTablet for tablet c7eea2e74a5e4380b6fef269d5030e4f (DEFAULT_TABLE table=test-workload [id=8e301a7a506f4181bc9e90c328fdebbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:19.818171 13022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7eea2e74a5e4380b6fef269d5030e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.818367 12873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7eea2e74a5e4380b6fef269d5030e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.820214 12880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 785b40ccad454ddaadb93567cb933a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.824255 12875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb56c49b42064eea93557f6c053db8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.828210 13028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1fdaa5a06b42abaf04c2e79bbf7d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.831751 13024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb56c49b42064eea93557f6c053db8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.834661 12877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c0f24d250a4bcbae0d8e08f2bd813e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.844192 12878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f3533d969441f28f11504f41dd42cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.849951 12876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bbc272093324888a170e4c87badda6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:19.803241 13080 tablet_bootstrap.cc:654] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.939312 13081 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:19.949687 13081 tablet_bootstrap.cc:654] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.950318 13082 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:19.955605 13081 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:19.956090 13080 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:19.956225 13081 ts_tablet_manager.cc:1403] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:06:19.956763 13080 ts_tablet_manager.cc:1403] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.160s	user 0.015s	sys 0.039s
I20260709 13:06:19.957860 13082 tablet_bootstrap.cc:654] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.959425 13081 raft_consensus.cc:359] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:19.959713 13080 raft_consensus.cc:359] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:19.960407 13081 raft_consensus.cc:385] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:19.960642 13080 raft_consensus.cc:385] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:19.960870 13081 raft_consensus.cc:740] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:19.961127 13080 raft_consensus.cc:740] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:19.961612 13081 consensus_queue.cc:260] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:19.962056 13080 consensus_queue.cc:260] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:19.963420 13082 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:19.964131 13065 heartbeater.cc:499] Master 127.0.107.254:39485 was elected leader, sending a full tablet report...
I20260709 13:06:19.964519 13081 ts_tablet_manager.cc:1434] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:06:19.965592 13081 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:19.968307 13082 ts_tablet_manager.cc:1403] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 13:06:19.971562 13081 tablet_bootstrap.cc:654] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.971318 13082 raft_consensus.cc:359] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:19.972095 13082 raft_consensus.cc:385] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:19.972402 13082 raft_consensus.cc:740] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:19.973281 13082 consensus_queue.cc:260] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:19.975180 12991 heartbeater.cc:499] Master 127.0.107.254:39485 was elected leader, sending a full tablet report...
I20260709 13:06:19.975731 13080 ts_tablet_manager.cc:1434] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.016s
I20260709 13:06:19.976926 13080 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:19.981805 12916 heartbeater.cc:499] Master 127.0.107.254:39485 was elected leader, sending a full tablet report...
I20260709 13:06:19.982863 13082 ts_tablet_manager.cc:1434] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:06:19.984131 13080 tablet_bootstrap.cc:654] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.987088 13082 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:19.988775 13081 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:19.989348 13081 ts_tablet_manager.cc:1403] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.004s
I20260709 13:06:19.992174 13081 raft_consensus.cc:359] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:19.992900 13081 raft_consensus.cc:385] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:19.993068 13082 tablet_bootstrap.cc:654] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:19.993232 13081 raft_consensus.cc:740] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:19.994311 13081 consensus_queue.cc:260] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:19.997258 13081 ts_tablet_manager.cc:1434] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:19.998607 13081 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.007668 13081 tablet_bootstrap.cc:654] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.011646 13080 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.011801 13082 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.012305 13080 ts_tablet_manager.cc:1403] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.036s	user 0.029s	sys 0.006s
I20260709 13:06:20.012622 13082 ts_tablet_manager.cc:1403] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.012s
I20260709 13:06:20.015869 13080 raft_consensus.cc:359] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.016816 13080 raft_consensus.cc:385] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.016543 13082 raft_consensus.cc:359] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.017247 13080 raft_consensus.cc:740] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.017733 13082 raft_consensus.cc:385] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.018152 13082 raft_consensus.cc:740] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.018313 13080 consensus_queue.cc:260] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.019309 13082 consensus_queue.cc:260] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.021106 13080 ts_tablet_manager.cc:1434] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:20.022320 13080 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.022498 13082 ts_tablet_manager.cc:1434] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:20.023875 13082 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.030511 13080 tablet_bootstrap.cc:654] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.030839 13082 tablet_bootstrap.cc:654] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.039613 13087 raft_consensus.cc:493] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.040434 13087 raft_consensus.cc:515] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.042858 13080 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.043514 13080 ts_tablet_manager.cc:1403] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.006s
I20260709 13:06:20.046960 13080 raft_consensus.cc:359] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.047816 13080 raft_consensus.cc:385] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.048135 13080 raft_consensus.cc:740] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.048563 13087 leader_election.cc:290] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.049252 13080 consensus_queue.cc:260] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.051911 13080 ts_tablet_manager.cc:1434] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:06:20.053370 13080 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.060225 13080 tablet_bootstrap.cc:654] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.065577 13080 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.066164 13080 ts_tablet_manager.cc:1403] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:06:20.070649 13081 tablet_bootstrap.cc:492] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.070423 13080 raft_consensus.cc:359] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.071548 13080 raft_consensus.cc:385] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.071719 13081 ts_tablet_manager.cc:1403] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.021s
I20260709 13:06:20.071954 13080 raft_consensus.cc:740] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.073108 13080 consensus_queue.cc:260] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.075646 13080 ts_tablet_manager.cc:1434] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:20.075531 13081 raft_consensus.cc:359] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.076726 13081 raft_consensus.cc:385] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.077126 13081 raft_consensus.cc:740] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.077217 13080 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.078259 13081 consensus_queue.cc:260] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.080751 13081 ts_tablet_manager.cc:1434] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:20.084798 13081 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.084798 13080 tablet_bootstrap.cc:654] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.094084 13081 tablet_bootstrap.cc:654] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.097669 13080 tablet_bootstrap.cc:492] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.098481 13080 ts_tablet_manager.cc:1403] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.009s
I20260709 13:06:20.102916 13080 raft_consensus.cc:359] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:20.109522 13080 raft_consensus.cc:385] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.110000 13080 raft_consensus.cc:740] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.111335 13080 consensus_queue.cc:260] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:20.114981 13080 ts_tablet_manager.cc:1434] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:06:20.116627 13080 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.126821 13080 tablet_bootstrap.cc:654] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.126821 13081 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.127732 13081 ts_tablet_manager.cc:1403] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.020s
I20260709 13:06:20.136583 13081 raft_consensus.cc:359] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.137842 13081 raft_consensus.cc:385] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.138489 13081 raft_consensus.cc:740] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.139935 13081 consensus_queue.cc:260] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.141446 13082 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.142045 13082 ts_tablet_manager.cc:1403] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.118s	user 0.055s	sys 0.059s
I20260709 13:06:20.143792 13081 ts_tablet_manager.cc:1434] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:06:20.145071 13082 raft_consensus.cc:359] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.145922 13082 raft_consensus.cc:385] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.146270 13082 raft_consensus.cc:740] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.145637 13081 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.147094 13082 consensus_queue.cc:260] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.149121 13080 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.149739 13080 ts_tablet_manager.cc:1403] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.017s
I20260709 13:06:20.152109 13082 ts_tablet_manager.cc:1434] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:06:20.152225 13087 raft_consensus.cc:493] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.153294 13082 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.152878 13087 raft_consensus.cc:515] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.155403 13087 leader_election.cc:290] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.157482 13080 raft_consensus.cc:359] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.158324 13080 raft_consensus.cc:385] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.158674 13080 raft_consensus.cc:740] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.159631 13080 consensus_queue.cc:260] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.160190 13081 tablet_bootstrap.cc:654] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.163888 12891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c0f24d250a4bcbae0d8e08f2bd813e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.164613 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.165856 13082 tablet_bootstrap.cc:654] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:20.166615 12929 leader_election.cc:343] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:20.167423 12929 leader_election.cc:343] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.167795 13080 ts_tablet_manager.cc:1434] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.004s
I20260709 13:06:20.169554 13080 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.175218 13081 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.176030 13081 ts_tablet_manager.cc:1403] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.005s
I20260709 13:06:20.177616 13080 tablet_bootstrap.cc:654] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.179957 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c0f24d250a4bcbae0d8e08f2bd813e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.180817 13040 raft_consensus.cc:2468] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.182634 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.183364 13082 tablet_bootstrap.cc:492] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.183849 13082 ts_tablet_manager.cc:1403] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.004s
I20260709 13:06:20.184691 12929 leader_election.cc:304] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.186244 13087 raft_consensus.cc:2804] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.186745 13087 raft_consensus.cc:493] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.186013 13081 raft_consensus.cc:359] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.187083 13087 raft_consensus.cc:3060] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.187410 13081 raft_consensus.cc:385] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.187103 13082 raft_consensus.cc:359] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.188198 13082 raft_consensus.cc:385] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.188097 13081 raft_consensus.cc:740] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.188699 13082 raft_consensus.cc:740] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.189646 13082 consensus_queue.cc:260] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.189694 13081 consensus_queue.cc:260] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
W20260709 13:06:20.186067 12929 leader_election.cc:343] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.193956 12929 leader_election.cc:304] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.197952 13081 ts_tablet_manager.cc:1434] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:06:20.199271 13082 ts_tablet_manager.cc:1434] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:20.201099 13081 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.202683 13080 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.203276 13080 ts_tablet_manager.cc:1403] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.019s
I20260709 13:06:20.207351 13080 raft_consensus.cc:359] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.208205 13080 raft_consensus.cc:385] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.208529 13080 raft_consensus.cc:740] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.209052 13081 tablet_bootstrap.cc:654] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.208832 13087 raft_consensus.cc:515] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.209554 13080 consensus_queue.cc:260] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.212594 13082 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.212961 13087 leader_election.cc:290] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.213567 13087 raft_consensus.cc:2749] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:20.220328 13080 ts_tablet_manager.cc:1434] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.009s
I20260709 13:06:20.220587 13082 tablet_bootstrap.cc:654] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.221891 13080 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.231370 13080 tablet_bootstrap.cc:654] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:20.231501 12929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:38223, user_credentials={real_user=slave}} blocked reactor thread for 45455us
I20260709 13:06:20.233918 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c0f24d250a4bcbae0d8e08f2bd813e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.234692 12890 raft_consensus.cc:3060] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.235076 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c0f24d250a4bcbae0d8e08f2bd813e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.235690 13039 raft_consensus.cc:3060] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.237694 13082 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.237861 13081 tablet_bootstrap.cc:492] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.240221 13081 ts_tablet_manager.cc:1403] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.041s	user 0.004s	sys 0.021s
I20260709 13:06:20.238338 13082 ts_tablet_manager.cc:1403] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.005s
I20260709 13:06:20.243901 13089 raft_consensus.cc:493] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.244514 13089 raft_consensus.cc:515] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:20.247515 13082 raft_consensus.cc:359] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.248420 13082 raft_consensus.cc:385] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.248741 13082 raft_consensus.cc:740] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.249509 13082 consensus_queue.cc:260] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.250785 13089 leader_election.cc:290] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.252004 13081 raft_consensus.cc:359] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.252789 13081 raft_consensus.cc:385] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.253098 13081 raft_consensus.cc:740] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.253929 13081 consensus_queue.cc:260] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.256399 13081 ts_tablet_manager.cc:1434] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 13:06:20.257905 13081 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.253132 12890 raft_consensus.cc:2468] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.264889 12929 leader_election.cc:304] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.266194 13105 raft_consensus.cc:493] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.266801 13105 raft_consensus.cc:515] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.267350 13089 raft_consensus.cc:493] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.270845 13081 tablet_bootstrap.cc:654] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.270691 13089 raft_consensus.cc:515] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
W20260709 13:06:20.271134 12837 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:20.269240 13105 leader_election.cc:290] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.273066 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.275308 13104 raft_consensus.cc:2804] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.275468 13040 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.268489 13087 raft_consensus.cc:493] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.276871 13087 raft_consensus.cc:515] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.277220 12929 leader_election.cc:304] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:20.275509 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.278987 12890 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.279377 13087 leader_election.cc:290] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.273334 13089 leader_election.cc:290] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.269743 13039 raft_consensus.cc:2468] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.281105 13081 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.281635 13081 ts_tablet_manager.cc:1403] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:06:20.285231 13104 raft_consensus.cc:697] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.286307 13104 consensus_queue.cc:237] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.286865 13082 ts_tablet_manager.cc:1434] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.009s
I20260709 13:06:20.286981 13105 raft_consensus.cc:2804] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.287523 13105 raft_consensus.cc:493] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.287945 13105 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.289934 13082 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.292440 13080 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.293094 13080 ts_tablet_manager.cc:1403] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.072s	user 0.027s	sys 0.012s
I20260709 13:06:20.293084 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
W20260709 13:06:20.294930 12929 leader_election.cc:343] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.296748 13080 raft_consensus.cc:359] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.297637 13080 raft_consensus.cc:385] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.298003 13080 raft_consensus.cc:740] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.298255 13038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f3533d969441f28f11504f41dd42cb" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.298985 13080 consensus_queue.cc:260] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.299077 13038 raft_consensus.cc:2468] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:20.299167 13081 raft_consensus.cc:359] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.299707 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63548b1cfe3b41aea9086ab46cf5df6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.301925 13081 raft_consensus.cc:385] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.302273 13081 raft_consensus.cc:740] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.302388 13040 raft_consensus.cc:2468] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:20.301353 12854 leader_election.cc:304] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.303184 13081 consensus_queue.cc:260] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.304102 12854 leader_election.cc:304] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.304388 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63548b1cfe3b41aea9086ab46cf5df6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.305181 13110 raft_consensus.cc:2804] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.305644 13110 raft_consensus.cc:493] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.305505 12966 raft_consensus.cc:2468] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:20.306272 13110 raft_consensus.cc:3060] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.303555 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.309784 13081 ts_tablet_manager.cc:1434] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:06:20.311184 13081 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.311961 13089 raft_consensus.cc:2804] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.312458 13089 raft_consensus.cc:493] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.312851 13089 raft_consensus.cc:3060] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.315603 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f3533d969441f28f11504f41dd42cb" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.316695 12966 raft_consensus.cc:2468] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:20.320075 13039 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.321765 12929 leader_election.cc:304] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.305636 13082 tablet_bootstrap.cc:654] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.324033 13087 raft_consensus.cc:2804] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.324671 13087 raft_consensus.cc:493] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.325187 13087 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.327095 13081 tablet_bootstrap.cc:654] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.300741 13105 raft_consensus.cc:515] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.330035 13105 leader_election.cc:290] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.333917 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.334677 12890 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.335758 13089 raft_consensus.cc:515] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:20.323581 13110 raft_consensus.cc:515] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.340250 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63548b1cfe3b41aea9086ab46cf5df6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.341147 13039 raft_consensus.cc:3060] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.344363 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f3533d969441f28f11504f41dd42cb" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.345276 13038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.346268 13038 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.349830 13110 leader_election.cc:290] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.345474 13040 raft_consensus.cc:3060] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.361426 13089 leader_election.cc:290] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Requested vote from peers 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.365092 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f3533d969441f28f11504f41dd42cb" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:20.366074 12966 raft_consensus.cc:3060] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.377235 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63548b1cfe3b41aea9086ab46cf5df6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:20.378302 12965 raft_consensus.cc:3060] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.401710 13038 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.403949 12929 leader_election.cc:304] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:20.405493 13115 raft_consensus.cc:2804] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.406189 13115 raft_consensus.cc:697] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.407433 13115 consensus_queue.cc:237] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.426584 12784 catalog_manager.cc:5697] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.436697 12965 raft_consensus.cc:2468] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:20.436967 12966 raft_consensus.cc:2468] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:20.440727 12890 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.443375 13039 raft_consensus.cc:2468] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:20.445502 12854 leader_election.cc:304] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.446846 13089 raft_consensus.cc:2804] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.448559 13089 raft_consensus.cc:697] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:20.449537 13089 consensus_queue.cc:237] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } }
I20260709 13:06:20.452423 13082 tablet_bootstrap.cc:492] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.452486 12857 leader_election.cc:304] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.431408 13040 raft_consensus.cc:2468] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:20.453245 13082 ts_tablet_manager.cc:1403] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.164s	user 0.020s	sys 0.030s
I20260709 13:06:20.455708 13080 ts_tablet_manager.cc:1434] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.162s	user 0.050s	sys 0.094s
I20260709 13:06:20.456807 13080 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:20.456485 13110 raft_consensus.cc:2804] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.459618 13110 raft_consensus.cc:697] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:20.466820 13081 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.468310 13110 consensus_queue.cc:237] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.467653 13081 ts_tablet_manager.cc:1403] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.157s	user 0.051s	sys 0.086s
I20260709 13:06:20.473284 13080 tablet_bootstrap.cc:654] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.478451 13081 raft_consensus.cc:359] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.479616 13081 raft_consensus.cc:385] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.481086 13081 raft_consensus.cc:740] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.488402 13081 consensus_queue.cc:260] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.492385 13081 ts_tablet_manager.cc:1434] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:06:20.492177 13087 raft_consensus.cc:515] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.493937 13081 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:20.487083 13114 raft_consensus.cc:493] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.494655 13114 raft_consensus.cc:515] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.497047 13114 leader_election.cc:290] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.499256 13087 leader_election.cc:290] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.504343 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.504609 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.505218 13039 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.506515 12788 catalog_manager.cc:5697] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.507884 13081 tablet_bootstrap.cc:654] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:20.512727 12929 leader_election.cc:343] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.514918 13039 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.517009 12929 leader_election.cc:304] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:20.517766 13115 raft_consensus.cc:493] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.518370 13115 raft_consensus.cc:515] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.519924 13080 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:20.520491 13080 ts_tablet_manager.cc:1403] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.006s
W20260709 13:06:20.502611 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43447, user_credentials={real_user=slave}} blocked reactor thread for 50422.3us
I20260709 13:06:20.520884 13087 raft_consensus.cc:2804] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.501977 13082 raft_consensus.cc:359] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.521498 13087 raft_consensus.cc:697] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.521572 13082 raft_consensus.cc:385] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.522048 13082 raft_consensus.cc:740] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.522228 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.522444 13087 consensus_queue.cc:237] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.522960 12890 raft_consensus.cc:2468] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.523043 13082 consensus_queue.cc:260] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.524286 13080 raft_consensus.cc:359] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.525034 13080 raft_consensus.cc:385] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.524854 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.525427 13080 raft_consensus.cc:740] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.525712 13082 ts_tablet_manager.cc:1434] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 13:06:20.526440 13080 consensus_queue.cc:260] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.526893 13082 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.527329 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.527499 12929 leader_election.cc:304] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:20.528087 12890 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.528401 13114 raft_consensus.cc:2804] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:20.528821 12929 leader_election.cc:343] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.528939 13114 raft_consensus.cc:493] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.529206 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.529444 13114 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.520668 13115 leader_election.cc:290] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.529982 13039 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:20.531713 12929 leader_election.cc:304] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.536104 13086 raft_consensus.cc:493] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.536464 13115 raft_consensus.cc:2804] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.536924 13115 raft_consensus.cc:493] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.536669 13086 raft_consensus.cc:515] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.537222 13080 ts_tablet_manager.cc:1434] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:06:20.537338 13115 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.541782 13114 raft_consensus.cc:515] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.544153 13114 leader_election.cc:290] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.545722 13115 raft_consensus.cc:515] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.546150 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.546993 12890 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.547840 13086 leader_election.cc:290] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.556319 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.557900 12891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.558224 13115 leader_election.cc:290] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.558745 12891 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.564221 12890 raft_consensus.cc:2468] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.573014 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.535405 13082 tablet_bootstrap.cc:654] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.575815 13039 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.573450 12929 leader_election.cc:304] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.580760 13115 raft_consensus.cc:2804] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.581537 13115 raft_consensus.cc:697] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.583204 13086 raft_consensus.cc:493] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.582893 13115 consensus_queue.cc:237] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.584558 13081 tablet_bootstrap.cc:492] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:20.584008 13086 raft_consensus.cc:515] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.585471 13081 ts_tablet_manager.cc:1403] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.000s
I20260709 13:06:20.586068 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
W20260709 13:06:20.584654 12929 leader_election.cc:343] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.590507 12891 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:20.588003 13086 leader_election.cc:290] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.589197 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.590407 13039 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
W20260709 13:06:20.593358 13003 leader_election.cc:343] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.594130 13081 raft_consensus.cc:359] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.594892 13081 raft_consensus.cc:385] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.595153 13081 raft_consensus.cc:740] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.595911 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.595878 13081 consensus_queue.cc:260] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.596273 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.596679 12966 raft_consensus.cc:2468] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:20.596966 12965 raft_consensus.cc:2468] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:20.598090 13081 ts_tablet_manager.cc:1434] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:20.598160 12929 leader_election.cc:304] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.598526 12784 catalog_manager.cc:5697] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.599517 13087 raft_consensus.cc:2804] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.600317 13087 raft_consensus.cc:697] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.600461 12784 catalog_manager.cc:5697] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.601687 13087 consensus_queue.cc:237] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.608274 13006 leader_election.cc:304] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
W20260709 13:06:20.608835 13003 leader_election.cc:343] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.609577 13006 leader_election.cc:304] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.609797 13082 tablet_bootstrap.cc:492] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.610457 13082 ts_tablet_manager.cc:1403] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.009s
I20260709 13:06:20.612740 13086 raft_consensus.cc:2804] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.613138 13086 raft_consensus.cc:493] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.612532 12786 catalog_manager.cc:5697] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.613667 13082 raft_consensus.cc:359] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.614382 13082 raft_consensus.cc:385] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.614650 13082 raft_consensus.cc:740] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.615360 13082 consensus_queue.cc:260] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.613468 13086 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.617411 13082 ts_tablet_manager.cc:1434] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:20.618340 13082 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.620790 13135 raft_consensus.cc:2804] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:20.621407 13135 raft_consensus.cc:493] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.621910 13135 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.625730 13082 tablet_bootstrap.cc:654] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.632934 13086 raft_consensus.cc:515] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.636140 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.636730 13135 raft_consensus.cc:515] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
W20260709 13:06:20.637696 13003 leader_election.cc:343] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:20.639377 13135 leader_election.cc:290] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.640187 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:20.640951 12965 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.641601 13086 leader_election.cc:290] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.643556 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:20.644256 12889 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.648665 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:20.649479 12966 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.650398 12786 catalog_manager.cc:5697] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.652330 12786 catalog_manager.cc:5697] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.653496 12889 raft_consensus.cc:2468] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:20.654241 12965 raft_consensus.cc:2468] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:20.654702 13003 leader_election.cc:304] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.655725 13086 raft_consensus.cc:2804] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.656237 13086 raft_consensus.cc:697] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:20.656643 13082 tablet_bootstrap.cc:492] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.657142 13082 ts_tablet_manager.cc:1403] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.002s
I20260709 13:06:20.657044 13086 consensus_queue.cc:237] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.657938 12966 raft_consensus.cc:2468] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:20.659204 13006 leader_election.cc:304] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.660107 13135 raft_consensus.cc:2804] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:20.660183 13082 raft_consensus.cc:359] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.661218 13082 raft_consensus.cc:385] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.661506 13082 raft_consensus.cc:740] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.662299 13082 consensus_queue.cc:260] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.669866 13135 raft_consensus.cc:697] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:20.671340 13135 consensus_queue.cc:237] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.672044 13082 ts_tablet_manager.cc:1434] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:06:20.674273 13082 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:20.677301 12784 catalog_manager.cc:5697] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:20.682068 13082 tablet_bootstrap.cc:654] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:20.686578 13082 tablet_bootstrap.cc:492] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:20.687039 13082 ts_tablet_manager.cc:1403] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:06:20.689153 13082 raft_consensus.cc:359] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.689690 13082 raft_consensus.cc:385] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:20.689921 13082 raft_consensus.cc:740] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:20.690467 13082 consensus_queue.cc:260] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.692322 13082 ts_tablet_manager.cc:1434] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:20.693146 12784 catalog_manager.cc:5697] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:20.738821 13087 consensus_queue.cc:1048] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.750416 13087 consensus_queue.cc:1048] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.882072 12966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:57688
I20260709 13:06:20.882640 12966 raft_consensus.cc:606] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.888267 12966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:57688
I20260709 13:06:20.888815 12966 raft_consensus.cc:606] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.907346 13087 consensus_queue.cc:1048] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.908098 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bbc272093324888a170e4c87badda6c"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:20.908689 13039 raft_consensus.cc:606] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:20.914616 13104 raft_consensus.cc:993] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:20.915045 13104 raft_consensus.cc:1081] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Signalling peer a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:20.921202 12891 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:06:20.921836 12891 raft_consensus.cc:493] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:20.922236 12891 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.925645 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:20.925962 13087 consensus_queue.cc:1048] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.926365 13039 raft_consensus.cc:606] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:20.929064 12891 raft_consensus.cc:515] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.930846 12889 raft_consensus.cc:1240] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Rejecting Update request from peer 579916bf94474548a6de9b8ea1cffb4a for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.932936 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:20.933728 12966 raft_consensus.cc:3055] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.933710 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1fdaa5a06b42abaf04c2e79bbf7d1d" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:20.933080 12891 leader_election.cc:290] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.934422 13040 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.936685 12966 raft_consensus.cc:740] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:20.937970 12965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:57688
I20260709 13:06:20.938597 12965 raft_consensus.cc:606] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Received request to transfer leadership to TS 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:20.938774 12966 consensus_queue.cc:260] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.939746 13104 consensus_queue.cc:1059] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.940922 13120 consensus_queue.cc:1048] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:20.942268 13040 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:20.943264 12966 raft_consensus.cc:3060] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.943562 12854 leader_election.cc:304] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:20.945130 13110 raft_consensus.cc:2804] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.945607 13110 raft_consensus.cc:697] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:20.946422 13110 consensus_queue.cc:237] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.951444 13120 consensus_queue.cc:1048] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:20.955004 12966 raft_consensus.cc:2468] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:20.962581 12786 catalog_manager.cc:5697] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 1 to 2, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.972000 13152 consensus_queue.cc:1048] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:20.976131 13135 raft_consensus.cc:493] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.976697 13135 raft_consensus.cc:515] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.978794 13135 leader_election.cc:290] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:20.980125 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.980995 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:20.981864 12889 raft_consensus.cc:2393] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in current term 1: Already voted for candidate 579916bf94474548a6de9b8ea1cffb4a in this term.
I20260709 13:06:20.983175 13003 leader_election.cc:304] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a; no voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:20.983903 13135 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.985977 13152 raft_consensus.cc:493] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.986564 13152 raft_consensus.cc:515] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.989607 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:20.989920 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:20.990866 13040 raft_consensus.cc:2393] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8ce4463a21a47afa02aa37a1d09441a in current term 1: Already voted for candidate 579916bf94474548a6de9b8ea1cffb4a in this term.
I20260709 13:06:20.991611 13152 leader_election.cc:290] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.991593 13135 raft_consensus.cc:2749] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:20.992178 12854 leader_election.cc:304] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:20.993139 13152 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:20.994436 13120 raft_consensus.cc:493] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.994977 13120 raft_consensus.cc:515] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:20.997053 13120 leader_election.cc:290] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:20.998796 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:21.001444 13089 consensus_queue.cc:1048] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.002619 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:21.003599 12966 raft_consensus.cc:2393] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8ce4463a21a47afa02aa37a1d09441a in current term 1: Already voted for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in this term.
I20260709 13:06:21.004774 12857 leader_election.cc:304] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:21.006093 13089 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.011915 13152 raft_consensus.cc:2749] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:21.015901 13089 raft_consensus.cc:2749] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:21.030179 13115 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.041409 13114 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.049504 13114 raft_consensus.cc:993] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:21.050009 13087 raft_consensus.cc:1081] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Signalling peer a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:21.051635 12889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:06:21.052224 12889 raft_consensus.cc:493] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.052603 12889 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.057483 13087 consensus_queue.cc:1048] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.059113 12889 raft_consensus.cc:515] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.061321 12889 leader_election.cc:290] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:21.062387 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:21.063009 13039 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.063035 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:21.063704 12966 raft_consensus.cc:3055] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.064059 12966 raft_consensus.cc:740] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:21.065274 12966 consensus_queue.cc:260] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.067049 12966 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.074679 12966 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:21.075897 12857 leader_election.cc:304] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:21.076759 13089 raft_consensus.cc:2804] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.076848 13039 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:21.077289 13089 raft_consensus.cc:697] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:21.078217 13089 consensus_queue.cc:237] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.083956 13114 consensus_queue.cc:1048] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:21.087771 12786 catalog_manager.cc:5697] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 1 to 2, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.088907 13115 consensus_queue.cc:1048] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:21.090317 13114 consensus_queue.cc:1048] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:21.111610 13115 raft_consensus.cc:993] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:21.112160 13104 raft_consensus.cc:1081] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Signalling peer 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:21.116254 13039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:39156
I20260709 13:06:21.116892 13039 raft_consensus.cc:493] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.117270 13039 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.126979 13039 raft_consensus.cc:515] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.130223 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:21.130393 12891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7eea2e74a5e4380b6fef269d5030e4f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:21.130980 12966 raft_consensus.cc:3055] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.130980 12891 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.131450 12966 raft_consensus.cc:740] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:21.132313 12966 consensus_queue.cc:260] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.133445 12966 raft_consensus.cc:3060] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.134111 13086 consensus_queue.cc:1048] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.137164 13135 consensus_queue.cc:1048] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.144099 12966 raft_consensus.cc:2468] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 2.
I20260709 13:06:21.129668 13039 leader_election.cc:290] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:21.146517 13006 leader_election.cc:304] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:21.147459 13135 raft_consensus.cc:2804] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.148157 13135 raft_consensus.cc:697] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:21.149338 13135 consensus_queue.cc:237] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.152087 12891 raft_consensus.cc:2468] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 2.
I20260709 13:06:21.161160 13137 raft_consensus.cc:993] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:21.161626 13137 raft_consensus.cc:1081] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:21.164451 12965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1bbc272093324888a170e4c87badda6c"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57674
I20260709 13:06:21.164294 12786 catalog_manager.cc:5697] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 1 to 2, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.166079 12965 raft_consensus.cc:493] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.166513 12965 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.175036 12965 raft_consensus.cc:515] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.181016 13137 consensus_queue.cc:1048] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:21.182884 13137 consensus_queue.cc:1048] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:21.190972 12965 leader_election.cc:290] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:21.192684 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:21.192684 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bbc272093324888a170e4c87badda6c" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:21.196436 13039 raft_consensus.cc:3055] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.196818 13039 raft_consensus.cc:740] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:21.197587 13039 consensus_queue.cc:260] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.198307 13039 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.205459 13039 raft_consensus.cc:2468] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:21.208302 12929 leader_election.cc:304] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:21.209124 13104 raft_consensus.cc:2804] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.209623 13104 raft_consensus.cc:697] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:21.210342 13104 consensus_queue.cc:237] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.216802 13165 raft_consensus.cc:993] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:21.217315 13137 raft_consensus.cc:1081] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:21.218991 12965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57674
I20260709 13:06:21.219599 12965 raft_consensus.cc:493] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.219987 12965 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.225400 12965 raft_consensus.cc:515] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.227622 12965 leader_election.cc:290] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:21.231073 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:21.231408 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29882fd52664468c848fd7e5c82b7907" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:21.231745 13039 raft_consensus.cc:3055] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.232144 13039 raft_consensus.cc:740] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:21.232183 12890 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.232479 12786 catalog_manager.cc:5697] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.232957 13039 consensus_queue.cc:260] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.234053 13039 raft_consensus.cc:3060] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.239751 12890 raft_consensus.cc:2468] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:21.241866 12929 leader_election.cc:304] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:21.241870 13039 raft_consensus.cc:2468] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:21.242758 13104 raft_consensus.cc:2804] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.243261 13104 raft_consensus.cc:697] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:21.244141 13104 consensus_queue.cc:237] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:21.253079 12786 catalog_manager.cc:5697] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.378157 12965 raft_consensus.cc:1275] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.379348 13154 consensus_queue.cc:1048] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.389326 13039 raft_consensus.cc:1275] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.391129 13154 consensus_queue.cc:1048] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.607497 12965 raft_consensus.cc:1275] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.607889 12966 raft_consensus.cc:1275] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.608995 13165 consensus_queue.cc:1048] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.609455 13154 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.620441 12890 raft_consensus.cc:1275] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.621424 13039 raft_consensus.cc:1275] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.622154 13165 consensus_queue.cc:1048] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.624958 13152 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.662233 12890 raft_consensus.cc:1275] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.663604 13114 consensus_queue.cc:1048] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.673229 13039 raft_consensus.cc:1275] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.675504 13115 consensus_queue.cc:1048] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.706589 12890 raft_consensus.cc:3060] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.712356 12890 raft_consensus.cc:1275] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.713796 13115 consensus_queue.cc:1048] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.722878 13039 raft_consensus.cc:1275] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.724640 13114 consensus_queue.cc:1048] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.970192 12788 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58626:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:23.973040 12788 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:24.023036 12880 tablet_service.cc:1511] Processing CreateTablet for tablet 9325d6826fd14512839a4b2c93d2298e (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:24.024097 12949 tablet_service.cc:1511] Processing CreateTablet for tablet 9325d6826fd14512839a4b2c93d2298e (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:24.024595 12880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9325d6826fd14512839a4b2c93d2298e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.025554 12949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9325d6826fd14512839a4b2c93d2298e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.025717 13024 tablet_service.cc:1511] Processing CreateTablet for tablet 9325d6826fd14512839a4b2c93d2298e (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:24.026954 13024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9325d6826fd14512839a4b2c93d2298e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.034463 12874 tablet_service.cc:1511] Processing CreateTablet for tablet 596bd6b656f245809e62b2e35f3be4c5 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:24.036032 12874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596bd6b656f245809e62b2e35f3be4c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.039852 13022 tablet_service.cc:1511] Processing CreateTablet for tablet 596bd6b656f245809e62b2e35f3be4c5 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:24.040602 12876 tablet_service.cc:1511] Processing CreateTablet for tablet f80854bf157f4a54b3c61baaad17db99 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:24.041386 13022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596bd6b656f245809e62b2e35f3be4c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.041133 13023 tablet_service.cc:1511] Processing CreateTablet for tablet f80854bf157f4a54b3c61baaad17db99 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:24.042017 12876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80854bf157f4a54b3c61baaad17db99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.042428 13023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80854bf157f4a54b3c61baaad17db99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.043856 13025 tablet_service.cc:1511] Processing CreateTablet for tablet 8cadafed43a34bfcacbab799f708ce5a (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:24.044672 12875 tablet_service.cc:1511] Processing CreateTablet for tablet 6b83673be2ef4d758c3fcae0cede98db (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:24.045254 13025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cadafed43a34bfcacbab799f708ce5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.046028 12875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b83673be2ef4d758c3fcae0cede98db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.045894 12873 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3b2593d934f60a42794b62a86f3ba (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:24.047122 12873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3b2593d934f60a42794b62a86f3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.048187 13028 tablet_service.cc:1511] Processing CreateTablet for tablet 6b83673be2ef4d758c3fcae0cede98db (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:24.048847 12881 tablet_service.cc:1511] Processing CreateTablet for tablet 8cadafed43a34bfcacbab799f708ce5a (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:24.049630 13028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b83673be2ef4d758c3fcae0cede98db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.050473 12881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cadafed43a34bfcacbab799f708ce5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.052158 12950 tablet_service.cc:1511] Processing CreateTablet for tablet 8cadafed43a34bfcacbab799f708ce5a (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:24.052693 12948 tablet_service.cc:1511] Processing CreateTablet for tablet f80854bf157f4a54b3c61baaad17db99 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:24.053604 12950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cadafed43a34bfcacbab799f708ce5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.053663 12953 tablet_service.cc:1511] Processing CreateTablet for tablet 596bd6b656f245809e62b2e35f3be4c5 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:24.055383 12953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596bd6b656f245809e62b2e35f3be4c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.054975 12955 tablet_service.cc:1511] Processing CreateTablet for tablet 6b83673be2ef4d758c3fcae0cede98db (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:24.057051 13029 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3b2593d934f60a42794b62a86f3ba (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:24.058549 13029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3b2593d934f60a42794b62a86f3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.060148 12951 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3b2593d934f60a42794b62a86f3ba (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:24.062270 12948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80854bf157f4a54b3c61baaad17db99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.064122 12951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3b2593d934f60a42794b62a86f3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.064880 12955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b83673be2ef4d758c3fcae0cede98db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.099582 12877 tablet_service.cc:1511] Processing CreateTablet for tablet 58d3d6fc2eb741909d8b21801de96033 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:24.101183 12952 tablet_service.cc:1511] Processing CreateTablet for tablet 58d3d6fc2eb741909d8b21801de96033 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:24.102633 12952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d3d6fc2eb741909d8b21801de96033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.101143 12877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d3d6fc2eb741909d8b21801de96033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.103814 13026 tablet_service.cc:1511] Processing CreateTablet for tablet 58d3d6fc2eb741909d8b21801de96033 (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:24.106278 13026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d3d6fc2eb741909d8b21801de96033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.130343 12880 tablet_service.cc:1511] Processing CreateTablet for tablet c8d4d0352f524065a1ce767e0081e2fc (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:24.131855 12880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d4d0352f524065a1ce767e0081e2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.135711 12956 tablet_service.cc:1511] Processing CreateTablet for tablet c8d4d0352f524065a1ce767e0081e2fc (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:24.137133 12956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d4d0352f524065a1ce767e0081e2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.161583 12875 tablet_service.cc:1511] Processing CreateTablet for tablet c52e2f491bf14241bcbfcfb6314b88ae (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:24.157738 13186 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.163537 12875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52e2f491bf14241bcbfcfb6314b88ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.170047 13028 tablet_service.cc:1511] Processing CreateTablet for tablet c8d4d0352f524065a1ce767e0081e2fc (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:24.184796 12949 tablet_service.cc:1511] Processing CreateTablet for tablet c52e2f491bf14241bcbfcfb6314b88ae (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:24.185254 13024 tablet_service.cc:1511] Processing CreateTablet for tablet c52e2f491bf14241bcbfcfb6314b88ae (DEFAULT_TABLE table=soft_deleted_table [id=9f60da3f54b04646a9446176049573be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:24.186187 12949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52e2f491bf14241bcbfcfb6314b88ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.190047 13187 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.195979 13187 tablet_bootstrap.cc:654] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.197943 13186 tablet_bootstrap.cc:654] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.203549 13028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d4d0352f524065a1ce767e0081e2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.208091 13024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52e2f491bf14241bcbfcfb6314b88ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.210246 13188 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.220556 13188 tablet_bootstrap.cc:654] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.228458 13186 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.228902 13186 ts_tablet_manager.cc:1403] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.000s
I20260709 13:06:24.229298 13188 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.229795 13188 ts_tablet_manager.cc:1403] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.003s
I20260709 13:06:24.231132 13186 raft_consensus.cc:359] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.231726 13186 raft_consensus.cc:385] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.232065 13186 raft_consensus.cc:740] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.232029 13188 raft_consensus.cc:359] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.232892 13188 raft_consensus.cc:385] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.233227 13188 raft_consensus.cc:740] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.233016 13186 consensus_queue.cc:260] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.233834 13187 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.234361 13187 ts_tablet_manager.cc:1403] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.045s	user 0.000s	sys 0.012s
I20260709 13:06:24.234007 13188 consensus_queue.cc:260] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.235719 13186 ts_tablet_manager.cc:1434] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:24.236423 13188 ts_tablet_manager.cc:1434] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.007s
I20260709 13:06:24.236732 13186 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.237468 13188 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.245225 13186 tablet_bootstrap.cc:654] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.250854 13188 tablet_bootstrap.cc:654] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.260561 13186 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.261114 13186 ts_tablet_manager.cc:1403] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 13:06:24.261973 13187 raft_consensus.cc:359] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.264590 13187 raft_consensus.cc:385] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.264943 13187 raft_consensus.cc:740] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.264645 13186 raft_consensus.cc:359] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.265516 13186 raft_consensus.cc:385] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.265837 13186 raft_consensus.cc:740] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.266607 13186 consensus_queue.cc:260] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.268154 13188 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.269577 13186 ts_tablet_manager.cc:1434] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:06:24.270612 13186 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.272457 13188 ts_tablet_manager.cc:1403] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
I20260709 13:06:24.275751 13188 raft_consensus.cc:359] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.265878 13187 consensus_queue.cc:260] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.276538 13188 raft_consensus.cc:385] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.276906 13188 raft_consensus.cc:740] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.277870 13186 tablet_bootstrap.cc:654] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.277647 13188 consensus_queue.cc:260] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.280122 13188 ts_tablet_manager.cc:1434] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:24.281191 13188 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.284039 13186 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.284698 13186 ts_tablet_manager.cc:1403] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:06:24.288026 13188 tablet_bootstrap.cc:654] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.288049 13186 raft_consensus.cc:359] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.288841 13186 raft_consensus.cc:385] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.289034 13186 raft_consensus.cc:740] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.289628 13186 consensus_queue.cc:260] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.291357 13186 ts_tablet_manager.cc:1434] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:24.292543 13186 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.295284 13187 ts_tablet_manager.cc:1434] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.003s
I20260709 13:06:24.297333 13188 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.299563 13186 tablet_bootstrap.cc:654] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.299934 13188 ts_tablet_manager.cc:1403] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.000s
I20260709 13:06:24.303133 13188 raft_consensus.cc:359] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.303329 13187 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.307497 13188 raft_consensus.cc:385] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.307960 13188 raft_consensus.cc:740] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
W20260709 13:06:24.308684 12837 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:24.308794 13188 consensus_queue.cc:260] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.311299 13188 ts_tablet_manager.cc:1434] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:06:24.312392 13188 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.316907 13187 tablet_bootstrap.cc:654] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.319265 13188 tablet_bootstrap.cc:654] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.322063 13186 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.322820 13186 ts_tablet_manager.cc:1403] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.030s	user 0.026s	sys 0.003s
I20260709 13:06:24.327029 13186 raft_consensus.cc:359] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.327860 13186 raft_consensus.cc:385] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.328182 13186 raft_consensus.cc:740] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.328948 13186 consensus_queue.cc:260] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.331327 13186 ts_tablet_manager.cc:1434] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 13:06:24.332388 13186 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.332466 13187 tablet_bootstrap.cc:492] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.333616 13187 ts_tablet_manager.cc:1403] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.004s
I20260709 13:06:24.335893 13188 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.336504 13188 ts_tablet_manager.cc:1403] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:06:24.336941 13187 raft_consensus.cc:359] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.337756 13187 raft_consensus.cc:385] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.338045 13187 raft_consensus.cc:740] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.338792 13187 consensus_queue.cc:260] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.339699 13186 tablet_bootstrap.cc:654] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.339850 13188 raft_consensus.cc:359] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.340745 13188 raft_consensus.cc:385] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.341109 13188 raft_consensus.cc:740] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.341313 13187 ts_tablet_manager.cc:1434] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:24.342362 13187 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.341987 13188 consensus_queue.cc:260] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.352069 13188 ts_tablet_manager.cc:1434] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:06:24.353163 13188 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.358807 13187 tablet_bootstrap.cc:654] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.346882 13172 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.360055 13188 tablet_bootstrap.cc:654] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.359985 13172 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.362309 13186 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.362802 13186 ts_tablet_manager.cc:1403] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.018s
I20260709 13:06:24.365913 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:24.365952 13186 raft_consensus.cc:359] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.366901 13186 raft_consensus.cc:385] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.367269 13186 raft_consensus.cc:740] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
W20260709 13:06:24.367921 13003 leader_election.cc:343] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.368072 13186 consensus_queue.cc:260] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.370417 13186 ts_tablet_manager.cc:1434] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:24.371409 13186 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.372339 13188 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.372865 13188 ts_tablet_manager.cc:1403] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.006s
I20260709 13:06:24.375147 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.375856 12965 raft_consensus.cc:2468] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:24.375731 13188 raft_consensus.cc:359] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.376708 13188 raft_consensus.cc:385] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.377017 13188 raft_consensus.cc:740] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.377522 13006 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:24.377992 13188 consensus_queue.cc:260] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.379199 13186 tablet_bootstrap.cc:654] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.379112 13200 raft_consensus.cc:2804] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.379606 13200 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.379951 13200 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.382215 13172 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.389652 13200 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.395872 13200 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.396718 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
W20260709 13:06:24.398650 13003 leader_election.cc:343] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.401285 13188 ts_tablet_manager.cc:1434] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.015s
I20260709 13:06:24.402436 13188 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.402874 13187 tablet_bootstrap.cc:492] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.403669 13187 ts_tablet_manager.cc:1403] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.015s
I20260709 13:06:24.403717 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.404531 12966 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.408216 13187 raft_consensus.cc:359] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.409353 13187 raft_consensus.cc:385] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.409799 13188 tablet_bootstrap.cc:654] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.409797 13187 raft_consensus.cc:740] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.411012 13187 consensus_queue.cc:260] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.414270 13187 ts_tablet_manager.cc:1434] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:06:24.425376 13187 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.443964 13187 tablet_bootstrap.cc:654] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.455142 12966 raft_consensus.cc:2468] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:24.457074 13006 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:24.457964 13203 raft_consensus.cc:2804] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.458470 13203 raft_consensus.cc:697] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:24.459311 13203 consensus_queue.cc:237] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.459937 13187 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.460515 13187 ts_tablet_manager.cc:1403] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.015s
I20260709 13:06:24.460870 13188 tablet_bootstrap.cc:492] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.461572 13188 ts_tablet_manager.cc:1403] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.029s
I20260709 13:06:24.461709 13186 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.464977 13188 raft_consensus.cc:359] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.465804 13186 ts_tablet_manager.cc:1403] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.095s	user 0.055s	sys 0.014s
I20260709 13:06:24.465947 13188 raft_consensus.cc:385] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.466425 13188 raft_consensus.cc:740] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.467243 13188 consensus_queue.cc:260] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.464833 13187 raft_consensus.cc:359] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.469456 13187 raft_consensus.cc:385] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.469868 13187 raft_consensus.cc:740] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.470700 13188 ts_tablet_manager.cc:1434] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:24.471982 13188 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.472796 13186 raft_consensus.cc:359] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.470922 13187 consensus_queue.cc:260] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.476894 13169 raft_consensus.cc:493] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.477461 13169 raft_consensus.cc:515] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.473999 13186 raft_consensus.cc:385] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.478487 13186 raft_consensus.cc:740] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.479658 13188 tablet_bootstrap.cc:654] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.479558 13169 leader_election.cc:290] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.479347 13186 consensus_queue.cc:260] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.482066 13186 ts_tablet_manager.cc:1434] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.004s
I20260709 13:06:24.484233 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.484601 13187 ts_tablet_manager.cc:1434] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.004s
I20260709 13:06:24.484923 12966 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:24.487977 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.488891 13039 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:24.490212 13186 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.497861 13187 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.497478 12857 leader_election.cc:304] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.502220 13175 raft_consensus.cc:2804] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.502761 13175 raft_consensus.cc:493] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.503165 13175 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.506088 13186 tablet_bootstrap.cc:654] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.506088 13187 tablet_bootstrap.cc:654] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.513110 12788 catalog_manager.cc:5697] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.522766 13187 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.524860 13187 ts_tablet_manager.cc:1403] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.006s
I20260709 13:06:24.526136 13186 tablet_bootstrap.cc:492] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.526347 13169 raft_consensus.cc:493] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.526741 13186 ts_tablet_manager.cc:1403] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.018s
I20260709 13:06:24.527029 13169 raft_consensus.cc:515] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.529237 13174 raft_consensus.cc:493] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.529774 13174 raft_consensus.cc:515] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.530253 13186 raft_consensus.cc:359] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.530572 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.531080 13186 raft_consensus.cc:385] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.531455 13186 raft_consensus.cc:740] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.531492 12966 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:24.532604 13174 leader_election.cc:290] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.532388 13186 consensus_queue.cc:260] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.533488 12857 leader_election.cc:304] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.534526 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.535404 13040 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:24.528574 13187 raft_consensus.cc:359] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.537119 12929 leader_election.cc:304] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:24.538434 13174 raft_consensus.cc:2804] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.537410 13187 raft_consensus.cc:385] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.538971 13174 raft_consensus.cc:493] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.539126 13187 raft_consensus.cc:740] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.540776 13169 leader_election.cc:290] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.542917 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.548390 13040 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:24.549173 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:24.539758 13174 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.544504 13187 consensus_queue.cc:260] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.550205 12889 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 0.
I20260709 13:06:24.547384 13213 raft_consensus.cc:2804] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.554479 13213 raft_consensus.cc:493] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.555044 13213 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.559939 13203 raft_consensus.cc:493] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.570696 13203 raft_consensus.cc:515] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.571246 13186 ts_tablet_manager.cc:1434] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.028s
I20260709 13:06:24.572950 13186 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.574257 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:24.575268 12889 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:24.579270 13003 leader_election.cc:304] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.580371 13200 raft_consensus.cc:2804] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.580758 13200 raft_consensus.cc:493] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.581166 13200 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.587013 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.588027 12966 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:24.590039 13200 raft_consensus.cc:515] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.591452 13175 raft_consensus.cc:515] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.592507 13203 leader_election.cc:290] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.581161 13186 tablet_bootstrap.cc:654] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.604543 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.605156 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:24.605472 12965 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.605907 13187 ts_tablet_manager.cc:1434] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.013s
I20260709 13:06:24.605978 13040 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.607226 13187 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.599480 13213 raft_consensus.cc:515] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.612974 12964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.613670 12964 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.614406 13040 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.597810 13200 leader_election.cc:290] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.599181 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:24.615295 12889 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.599982 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.616266 12965 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.616667 12966 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.618108 12857 leader_election.cc:304] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.624053 13212 raft_consensus.cc:2804] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.624841 13212 raft_consensus.cc:697] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:24.626116 13212 consensus_queue.cc:237] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.630856 12964 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.632917 12857 leader_election.cc:304] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.634275 13169 raft_consensus.cc:2804] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.637449 12889 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:24.638792 13003 leader_election.cc:304] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.639596 13200 raft_consensus.cc:2804] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.639978 13169 raft_consensus.cc:697] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:24.640159 13200 raft_consensus.cc:697] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:24.640743 13175 leader_election.cc:290] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.641032 13169 consensus_queue.cc:237] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.641048 13200 consensus_queue.cc:237] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.642094 13174 raft_consensus.cc:515] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.651912 13203 raft_consensus.cc:493] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.652697 13203 raft_consensus.cc:515] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.658202 13203 leader_election.cc:290] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.658349 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
W20260709 13:06:24.659932 13006 leader_election.cc:343] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.661407 13213 leader_election.cc:290] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.662292 13213 raft_consensus.cc:493] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.668126 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:24.668988 13039 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.667668 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:24.673032 12889 raft_consensus.cc:2468] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:24.673782 13174 leader_election.cc:290] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.635574 12966 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:24.662942 13213 raft_consensus.cc:515] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.677345 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:24.678104 12890 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.679886 13003 leader_election.cc:304] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.680886 13200 raft_consensus.cc:2804] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.681298 13200 raft_consensus.cc:493] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.681643 13200 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.681807 12964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.682828 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:24.683442 13040 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.684465 13186 tablet_bootstrap.cc:492] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.685083 13186 ts_tablet_manager.cc:1403] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.112s	user 0.022s	sys 0.023s
W20260709 13:06:24.684895 12857 leader_election.cc:343] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.688122 13186 raft_consensus.cc:359] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.688752 13186 raft_consensus.cc:385] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.689002 13186 raft_consensus.cc:740] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.689638 13186 consensus_queue.cc:260] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.691354 13187 tablet_bootstrap.cc:654] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.692562 12890 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:24.695104 13188 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.695665 13188 ts_tablet_manager.cc:1403] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.224s	user 0.060s	sys 0.086s
I20260709 13:06:24.696029 13186 ts_tablet_manager.cc:1434] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:24.695943 12788 catalog_manager.cc:5697] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.696520 12929 leader_election.cc:304] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.697235 13186 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: Bootstrap starting.
I20260709 13:06:24.697669 13174 raft_consensus.cc:2804] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.701496 13174 raft_consensus.cc:697] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:24.699162 13188 raft_consensus.cc:359] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.703099 13188 raft_consensus.cc:385] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.703450 13188 raft_consensus.cc:740] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.704419 13039 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.709091 13038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.708917 13213 leader_election.cc:290] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.703681 13186 tablet_bootstrap.cc:654] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.706132 13040 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 1.
I20260709 13:06:24.706089 13174 consensus_queue.cc:237] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.712348 13200 raft_consensus.cc:515] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.716952 13200 leader_election.cc:290] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.713454 13188 consensus_queue.cc:260] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.717800 13038 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 0.
I20260709 13:06:24.718139 13187 tablet_bootstrap.cc:492] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.718302 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.719012 13187 ts_tablet_manager.cc:1403] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.002s
W20260709 13:06:24.720232 13006 leader_election.cc:343] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.722962 13188 ts_tablet_manager.cc:1434] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:06:24.723558 13187 raft_consensus.cc:359] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.724318 13188 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.724653 13187 raft_consensus.cc:385] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.725116 13187 raft_consensus.cc:740] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.726061 13187 consensus_queue.cc:260] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.729677 13187 ts_tablet_manager.cc:1434] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:06:24.730938 13187 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.733486 13186 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: No bootstrap required, opened a new log
I20260709 13:06:24.734076 13186 ts_tablet_manager.cc:1403] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.007s
I20260709 13:06:24.734483 13188 tablet_bootstrap.cc:654] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.799300 13213 raft_consensus.cc:493] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.805344 13213 raft_consensus.cc:515] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.808869 12854 leader_election.cc:304] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.809875 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:24.810786 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.811473 12890 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.814862 13212 raft_consensus.cc:2804] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.818914 13212 raft_consensus.cc:493] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:24.820495 12857 leader_election.cc:343] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.825265 13038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.821195 13212 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:24.827759 12854 leader_election.cc:343] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.828685 12854 leader_election.cc:304] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.826390 12784 catalog_manager.cc:5697] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.830111 13169 raft_consensus.cc:2749] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:24.820580 13213 leader_election.cc:290] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.837559 13187 tablet_bootstrap.cc:654] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.852710 13186 raft_consensus.cc:359] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.853914 13186 raft_consensus.cc:385] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.854269 13186 raft_consensus.cc:740] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.855468 13186 consensus_queue.cc:260] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.855953 13212 raft_consensus.cc:515] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.858359 13186 ts_tablet_manager.cc:1434] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: Time spent starting tablet: real 0.124s	user 0.011s	sys 0.000s
I20260709 13:06:24.860419 13213 raft_consensus.cc:493] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.861037 13213 raft_consensus.cc:515] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.861234 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:24.863023 12890 raft_consensus.cc:2468] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:24.864122 13212 leader_election.cc:290] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.864972 13038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.866201 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:24.866492 13003 leader_election.cc:304] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.867164 13039 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.868126 13200 raft_consensus.cc:2804] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.869118 13200 raft_consensus.cc:697] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:24.870720 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.871167 13200 consensus_queue.cc:237] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.870719 12788 catalog_manager.cc:5697] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.865304 13213 leader_election.cc:290] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
W20260709 13:06:24.875392 12854 leader_election.cc:343] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:24.879082 12857 leader_election.cc:343] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:24.880033 12857 leader_election.cc:343] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.880541 12857 leader_election.cc:304] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.881659 13213 raft_consensus.cc:2749] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:24.885128 13039 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.886713 13187 tablet_bootstrap.cc:492] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:24.887449 13187 ts_tablet_manager.cc:1403] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.157s	user 0.024s	sys 0.004s
I20260709 13:06:24.888747 12854 leader_election.cc:304] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.892930 13212 raft_consensus.cc:2804] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.892016 13187 raft_consensus.cc:359] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.894826 13187 raft_consensus.cc:385] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.895260 13187 raft_consensus.cc:740] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.895234 13212 raft_consensus.cc:697] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:24.897954 13188 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.898574 13187 consensus_queue.cc:260] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.898808 13188 ts_tablet_manager.cc:1403] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.007s
I20260709 13:06:24.899937 13212 consensus_queue.cc:237] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.919202 13187 ts_tablet_manager.cc:1434] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.016s
I20260709 13:06:24.921248 13187 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:24.923625 13188 raft_consensus.cc:359] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.924324 12889 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.925107 13188 raft_consensus.cc:385] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.925657 13188 raft_consensus.cc:740] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.926769 13188 consensus_queue.cc:260] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.931103 13188 ts_tablet_manager.cc:1434] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:24.931125 13187 tablet_bootstrap.cc:654] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.932889 13188 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: Bootstrap starting.
I20260709 13:06:24.934046 12785 catalog_manager.cc:5697] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.940078 13188 tablet_bootstrap.cc:654] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.970687 13200 raft_consensus.cc:493] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.971385 13200 raft_consensus.cc:515] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.971004 12786 catalog_manager.cc:5697] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.974848 12891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:24.975273 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.975610 12891 raft_consensus.cc:2468] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
W20260709 13:06:24.977186 13006 leader_election.cc:343] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.977876 13200 leader_election.cc:290] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:24.977828 13003 leader_election.cc:304] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:24.979171 13200 raft_consensus.cc:2804] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.979674 13200 raft_consensus.cc:493] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.980216 13200 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.988746 13213 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.989462 13213 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:24.994190 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:24.994503 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:24.993356 13213 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:24.995040 12966 raft_consensus.cc:2468] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 1.
I20260709 13:06:24.997084 12857 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:24.998288 13213 raft_consensus.cc:2804] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:24.998020 13203 consensus_queue.cc:1048] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:24.998739 13213 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.999142 13213 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:24.999362 13188 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: No bootstrap required, opened a new log
I20260709 13:06:24.999871 13188 ts_tablet_manager.cc:1403] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.014s
I20260709 13:06:25.003753 13188 raft_consensus.cc:359] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.004213 13187 tablet_bootstrap.cc:492] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:25.004487 13188 raft_consensus.cc:385] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.004858 13188 raft_consensus.cc:740] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.004860 13187 ts_tablet_manager.cc:1403] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.084s	user 0.033s	sys 0.037s
I20260709 13:06:25.005605 13188 consensus_queue.cc:260] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.006152 13203 consensus_queue.cc:1048] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.008894 13200 raft_consensus.cc:515] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.010246 13187 raft_consensus.cc:359] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.011089 13187 raft_consensus.cc:385] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.011296 13200 leader_election.cc:290] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:25.011390 13187 raft_consensus.cc:740] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.012360 13187 consensus_queue.cc:260] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.012950 12964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.013840 12964 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.015007 13187 ts_tablet_manager.cc:1434] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:25.016389 13187 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: Bootstrap starting.
I20260709 13:06:25.017201 12890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.017907 12890 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.025031 13187 tablet_bootstrap.cc:654] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.029102 12890 raft_consensus.cc:2468] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:25.032449 12785 catalog_manager.cc:5697] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.039217 13003 leader_election.cc:304] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:25.040417 13200 raft_consensus.cc:2804] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.040901 13200 raft_consensus.cc:697] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.040923 13213 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.041771 13200 consensus_queue.cc:237] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.043125 12889 raft_consensus.cc:1240] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Rejecting Update request from peer 0a2cdf08ef784d4593c11c42bf6c5d2a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:06:25.049151 12964 raft_consensus.cc:2468] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:25.050381 13187 tablet_bootstrap.cc:492] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: No bootstrap required, opened a new log
I20260709 13:06:25.052070 13187 ts_tablet_manager.cc:1403] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.002s
I20260709 13:06:25.056046 13187 raft_consensus.cc:359] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.057067 13187 raft_consensus.cc:385] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.057436 13187 raft_consensus.cc:740] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.058293 13187 consensus_queue.cc:260] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.051184 13188 ts_tablet_manager.cc:1434] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: Time spent starting tablet: real 0.051s	user 0.022s	sys 0.011s
I20260709 13:06:25.064987 13200 consensus_queue.cc:1059] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.067216 13187 ts_tablet_manager.cc:1434] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:06:25.071518 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.072161 12964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.073804 12857 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:25.074052 13213 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.073800 13203 consensus_queue.cc:1048] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.074853 13213 raft_consensus.cc:2749] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:25.078148 12788 catalog_manager.cc:5697] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:25.082868 13233 raft_consensus.cc:3055] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.084048 12890 raft_consensus.cc:1240] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Rejecting Update request from peer 0a2cdf08ef784d4593c11c42bf6c5d2a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:06:25.086024 13233 raft_consensus.cc:740] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.086241 13203 consensus_queue.cc:1059] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.091037 13233 consensus_queue.cc:260] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.092912 13233 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.102475 13238 consensus_queue.cc:1048] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.112000 13212 consensus_queue.cc:1048] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.113597 13212 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.143607 13220 consensus_queue.cc:1048] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.146823 13212 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.165376 13174 raft_consensus.cc:493] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.165930 13174 raft_consensus.cc:515] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.169044 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:25.169425 13174 leader_election.cc:290] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.169893 12889 raft_consensus.cc:2393] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 579916bf94474548a6de9b8ea1cffb4a in current term 1: Already voted for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in this term.
I20260709 13:06:25.171053 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:25.172518 12929 leader_election.cc:304] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.173658 13174 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.180197 13174 raft_consensus.cc:2749] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.262692 13200 consensus_queue.cc:1048] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.266547 13174 raft_consensus.cc:493] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.267053 13174 raft_consensus.cc:515] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.269340 13174 leader_election.cc:290] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.270201 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:25.270234 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:25.271057 13039 raft_consensus.cc:2393] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 579916bf94474548a6de9b8ea1cffb4a in current term 1: Already voted for candidate a8ce4463a21a47afa02aa37a1d09441a in this term.
I20260709 13:06:25.272310 12929 leader_election.cc:304] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.274613 13203 consensus_queue.cc:1048] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.275640 13217 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.284986 13217 raft_consensus.cc:2749] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.291967 13174 consensus_queue.cc:1048] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.305240 13174 consensus_queue.cc:1048] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:25.318050 12837 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:25.394480 13212 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.406276 13175 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.408116 13238 raft_consensus.cc:493] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.408739 13238 raft_consensus.cc:515] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.411109 13238 leader_election.cc:290] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:25.412848 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:25.413352 12965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:25.413626 12889 raft_consensus.cc:2468] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:25.414105 12965 raft_consensus.cc:2468] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 0.
I20260709 13:06:25.415163 13003 leader_election.cc:304] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:25.417088 13238 raft_consensus.cc:2804] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.417593 13238 raft_consensus.cc:493] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.418016 13238 raft_consensus.cc:3060] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.427951 13238 raft_consensus.cc:515] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.430472 13238 leader_election.cc:290] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:25.430862 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.431260 12963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52e2f491bf14241bcbfcfb6314b88ae" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.431571 12889 raft_consensus.cc:3060] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.431981 12963 raft_consensus.cc:3060] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.437615 12963 raft_consensus.cc:2468] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:25.437642 12889 raft_consensus.cc:2468] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 1.
I20260709 13:06:25.438781 13006 leader_election.cc:304] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:25.439507 13238 raft_consensus.cc:2804] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.440026 13238 raft_consensus.cc:697] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.440764 13238 consensus_queue.cc:237] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.448324 13238 consensus_queue.cc:1048] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.449281 12788 catalog_manager.cc:5697] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:25.461472 13238 consensus_queue.cc:1048] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.480930 12963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:57688
I20260709 13:06:25.481742 12963 raft_consensus.cc:606] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.487248 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:25.487663 13039 raft_consensus.cc:606] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.492276 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:25.492710 13039 raft_consensus.cc:606] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.497465 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:25.497954 13039 raft_consensus.cc:606] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Received request to transfer leadership to TS a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:25.498265 13039 raft_consensus.cc:614] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:25.513435 12889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:59302
I20260709 13:06:25.513993 12889 raft_consensus.cc:606] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:25.519515 12889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:59302
I20260709 13:06:25.520087 12889 raft_consensus.cc:606] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:25.524679 12889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:59302
I20260709 13:06:25.525194 12889 raft_consensus.cc:606] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:25.530093 13039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
mode: GRACEFUL
new_leader_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:39174
I20260709 13:06:25.530656 13039 raft_consensus.cc:606] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Received request to transfer leadership to TS 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:25.533264 13220 raft_consensus.cc:993] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.533785 13220 raft_consensus.cc:1081] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.535511 12963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57668
I20260709 13:06:25.536116 12963 raft_consensus.cc:493] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.536495 12963 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.542174 12963 raft_consensus.cc:515] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.544076 12963 leader_election.cc:290] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.544863 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.545133 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3b2593d934f60a42794b62a86f3ba" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.545576 12889 raft_consensus.cc:3055] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.545661 13039 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.545964 12889 raft_consensus.cc:740] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.546865 12889 consensus_queue.cc:260] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.548259 12889 raft_consensus.cc:3060] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.553004 13039 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.554623 12929 leader_election.cc:304] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:25.555459 13174 raft_consensus.cc:2804] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.555948 13174 raft_consensus.cc:697] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:25.556036 12889 raft_consensus.cc:2468] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.556901 13174 consensus_queue.cc:237] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.563828 12788 catalog_manager.cc:5697] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.630293 13238 raft_consensus.cc:993] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: : Instructing follower a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.630746 13233 raft_consensus.cc:1081] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Signalling peer a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.632129 12889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:59292
I20260709 13:06:25.632615 12889 raft_consensus.cc:493] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.632908 12889 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.638309 12889 raft_consensus.cc:515] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.640329 12889 leader_election.cc:290] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.641074 12963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.641449 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596bd6b656f245809e62b2e35f3be4c5" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.641845 12963 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.642061 13039 raft_consensus.cc:3055] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.642462 13039 raft_consensus.cc:740] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.643392 13039 consensus_queue.cc:260] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.644742 13039 raft_consensus.cc:3060] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.649813 12963 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.651342 12857 leader_election.cc:304] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:25.651779 13039 raft_consensus.cc:2468] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.652130 13175 raft_consensus.cc:2804] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.652753 13175 raft_consensus.cc:697] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.653756 13175 consensus_queue.cc:237] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.655622 13169 raft_consensus.cc:993] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.656194 13220 raft_consensus.cc:1081] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.657859 12963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57668
I20260709 13:06:25.658404 12963 raft_consensus.cc:493] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.658717 12963 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.661262 12788 catalog_manager.cc:5697] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.664443 12963 raft_consensus.cc:515] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.666692 12963 leader_election.cc:290] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.667721 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.667829 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.668520 13039 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.668629 12889 raft_consensus.cc:3055] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.669087 12889 raft_consensus.cc:740] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.670027 12889 consensus_queue.cc:260] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.671551 12889 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.673794 13039 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.674872 12929 leader_election.cc:304] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:25.675633 13174 raft_consensus.cc:2804] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.676136 13174 raft_consensus.cc:697] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:25.677292 12889 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.677052 13174 consensus_queue.cc:237] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.680830 13203 raft_consensus.cc:993] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: : Instructing follower a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.681255 13203 raft_consensus.cc:1081] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Signalling peer a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.686758 12786 catalog_manager.cc:5697] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.689113 12889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:59292
I20260709 13:06:25.689728 12889 raft_consensus.cc:493] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.690089 12889 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.695832 12889 raft_consensus.cc:515] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.697288 12889 leader_election.cc:290] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.698083 12963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.698357 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80854bf157f4a54b3c61baaad17db99" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.698768 12963 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.698879 13039 raft_consensus.cc:3055] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.699319 13039 raft_consensus.cc:740] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.700109 13039 consensus_queue.cc:260] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.701253 13039 raft_consensus.cc:3060] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.705904 12963 raft_consensus.cc:2468] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.707134 12857 leader_election.cc:304] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:25.707708 13039 raft_consensus.cc:2468] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.707885 13220 raft_consensus.cc:2804] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.708662 13220 raft_consensus.cc:697] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.709594 13220 consensus_queue.cc:237] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.716351 12786 catalog_manager.cc:5697] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.840445 13169 raft_consensus.cc:993] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.840984 13220 raft_consensus.cc:1081] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.842389 12963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57668
I20260709 13:06:25.842954 12963 raft_consensus.cc:493] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.843335 12963 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.848588 12963 raft_consensus.cc:515] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.850111 12963 leader_election.cc:290] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.850870 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.851135 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.851477 12889 raft_consensus.cc:3055] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.851752 13039 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.851899 12889 raft_consensus.cc:740] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.852653 12889 consensus_queue.cc:260] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.853524 12889 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.856398 13039 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.857349 12929 leader_election.cc:304] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:25.858018 13174 raft_consensus.cc:2804] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.858530 13174 raft_consensus.cc:697] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:25.858762 12889 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.859421 13174 consensus_queue.cc:237] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.865124 13238 raft_consensus.cc:993] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: : Instructing follower 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.865628 13203 raft_consensus.cc:1081] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Signalling peer 579916bf94474548a6de9b8ea1cffb4a to start an election
I20260709 13:06:25.867141 12963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
 from {username='slave'} at 127.0.0.1:57674
I20260709 13:06:25.866926 12788 catalog_manager.cc:5697] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.867837 12963 raft_consensus.cc:493] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.868312 12963 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.874055 12963 raft_consensus.cc:515] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.876083 12963 leader_election.cc:290] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.876734 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:25.876917 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d4d0352f524065a1ce767e0081e2fc" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.877430 12889 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.877547 13039 raft_consensus.cc:3055] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.878036 13039 raft_consensus.cc:740] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:25.878705 13039 consensus_queue.cc:260] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.879676 13039 raft_consensus.cc:3060] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.886376 13039 raft_consensus.cc:2468] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.887467 13217 raft_consensus.cc:993] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.888089 13260 raft_consensus.cc:1081] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Signalling peer a8ce4463a21a47afa02aa37a1d09441a to start an election
I20260709 13:06:25.888183 12929 leader_election.cc:304] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:25.889016 13217 raft_consensus.cc:2804] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.889554 13217 raft_consensus.cc:697] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:25.889789 12890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:06:25.890296 12890 raft_consensus.cc:493] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:25.890532 12890 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.890618 12889 raft_consensus.cc:2468] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:25.890595 13217 consensus_queue.cc:237] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.896633 13203 consensus_queue.cc:1048] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.896790 12890 raft_consensus.cc:515] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.899448 12963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:25.900166 12963 raft_consensus.cc:3055] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:25.900601 12963 raft_consensus.cc:740] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:25.901326 12963 consensus_queue.cc:260] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.902074 12788 catalog_manager.cc:5697] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.902199 12963 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.905411 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9325d6826fd14512839a4b2c93d2298e" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:25.905989 13039 raft_consensus.cc:3060] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.910287 13233 consensus_queue.cc:1048] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.912658 12890 leader_election.cc:290] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Requested vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:25.915635 12963 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.917203 12857 leader_election.cc:304] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579916bf94474548a6de9b8ea1cffb4a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:25.918004 13220 raft_consensus.cc:2804] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:25.918476 13220 raft_consensus.cc:697] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Leader. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:25.919502 13220 consensus_queue.cc:237] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:25.922950 13039 raft_consensus.cc:2468] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ce4463a21a47afa02aa37a1d09441a in term 2.
I20260709 13:06:25.932356 12786 catalog_manager.cc:5697] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a reported cstate change: term changed from 1 to 2, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a8ce4463a21a47afa02aa37a1d09441a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.952513 13039 raft_consensus.cc:1275] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:25.953616 13217 consensus_queue.cc:1048] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:25.960904 12889 raft_consensus.cc:1275] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:25.962182 13260 consensus_queue.cc:1048] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.059446 12963 raft_consensus.cc:1275] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.060843 13220 consensus_queue.cc:1048] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.068826 13039 raft_consensus.cc:1275] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.070103 13263 consensus_queue.cc:1048] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.098748 13039 raft_consensus.cc:1275] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.099946 13260 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.108028 12889 raft_consensus.cc:1275] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.109470 13174 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.186650 13039 raft_consensus.cc:1275] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.187845 13263 consensus_queue.cc:1048] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.190927 12963 raft_consensus.cc:1275] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.192431 13169 consensus_queue.cc:1048] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.257334 12889 raft_consensus.cc:1275] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.258733 13174 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.266170 13039 raft_consensus.cc:1275] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.267911 13217 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.332109 13039 raft_consensus.cc:1275] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.333465 13220 consensus_queue.cc:1048] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.344940 12963 raft_consensus.cc:1275] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer a8ce4463a21a47afa02aa37a1d09441a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.347906 13220 consensus_queue.cc:1048] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.353662 13039 raft_consensus.cc:1275] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.355195 13260 consensus_queue.cc:1048] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.363260 12889 raft_consensus.cc:1275] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:26.364733 13174 consensus_queue.cc:1048] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:26.574546 13174 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a2cdf08ef784d4593c11c42bf6c5d2a)
I20260709 13:06:26.575026 13174 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:26.576822 13174 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:26.577648 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a" is_pre_election: true
I20260709 13:06:26.577759 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:26.578358 13039 raft_consensus.cc:2468] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:26.578415 12889 raft_consensus.cc:2393] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 579916bf94474548a6de9b8ea1cffb4a in current term 2: Already voted for candidate a8ce4463a21a47afa02aa37a1d09441a in this term.
I20260709 13:06:26.579557 12929 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:26.580317 13174 raft_consensus.cc:2804] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:26.580677 13174 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a2cdf08ef784d4593c11c42bf6c5d2a)
I20260709 13:06:26.581015 13174 raft_consensus.cc:3060] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.586612 13174 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:26.588637 13174 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:26.589387 12889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:26.589600 13039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "579916bf94474548a6de9b8ea1cffb4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
I20260709 13:06:26.590169 12889 raft_consensus.cc:2393] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 579916bf94474548a6de9b8ea1cffb4a in current term 2: Already voted for candidate a8ce4463a21a47afa02aa37a1d09441a in this term.
I20260709 13:06:26.594393 13039 raft_consensus.cc:2468] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579916bf94474548a6de9b8ea1cffb4a in term 2.
I20260709 13:06:26.595463 12929 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: a8ce4463a21a47afa02aa37a1d09441a
I20260709 13:06:26.596163 13174 raft_consensus.cc:2804] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:26.596655 13174 raft_consensus.cc:697] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Leader. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:26.597390 13174 consensus_queue.cc:237] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:26.604933 12786 catalog_manager.cc:5697] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a reported cstate change: term changed from 1 to 2, leader changed from 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195) to 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "579916bf94474548a6de9b8ea1cffb4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.893249 13175 raft_consensus.cc:493] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.893775 13175 raft_consensus.cc:515] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:26.895783 13175 leader_election.cc:290] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447), 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195:38223)
I20260709 13:06:26.896549 12963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579916bf94474548a6de9b8ea1cffb4a" is_pre_election: true
I20260709 13:06:26.896868 13040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cadafed43a34bfcacbab799f708ce5a" candidate_uuid: "a8ce4463a21a47afa02aa37a1d09441a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" is_pre_election: true
I20260709 13:06:26.897790 13040 raft_consensus.cc:2410] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8ce4463a21a47afa02aa37a1d09441a for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:06:26.898761 12854 leader_election.cc:304] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ce4463a21a47afa02aa37a1d09441a; no voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a
I20260709 13:06:26.899461 13175 raft_consensus.cc:2749] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:27.009042 13040 raft_consensus.cc:1275] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:27.010597 13217 consensus_queue.cc:1048] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:27.020604 12890 raft_consensus.cc:1275] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 579916bf94474548a6de9b8ea1cffb4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:27.022271 13174 consensus_queue.cc:1048] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:28.537400 12788 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58626:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:28.558789 12788 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58626:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:28.624759   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:28.748853 13315 tablet.cc:1724] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.811939 13317 tablet.cc:1724] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.836465 13326 tablet.cc:1724] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.878974 13331 tablet.cc:1724] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.886116 13333 tablet.cc:1724] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.913456 13334 tablet.cc:1724] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.928076 13335 tablet.cc:1724] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.825599 13313 tablet.cc:1724] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.883292 13330 tablet.cc:1724] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.968190 13336 tablet.cc:1724] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:28.971184 13323 log.cc:927] Time spent T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
I20260709 13:06:28.979723 13338 tablet.cc:1724] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.985507 13326 tablet.cc:1724] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:28.989321 13341 tablet.cc:1724] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:28.957077 12929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38223, user_credentials={real_user=slave}} blocked reactor thread for 43705.8us
I20260709 13:06:28.993228 12966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:55514
I20260709 13:06:29.013721 12966 raft_consensus.cc:606] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Received request to transfer leadership to TS 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:29.017297 13317 tablet.cc:1724] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.013867 13315 tablet.cc:1724] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.021397 12966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:55514
I20260709 13:06:29.022109 12966 raft_consensus.cc:606] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Received request to transfer leadership to TS 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:29.023379   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:06:29.024096   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:29.003263 13003 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:38443, user_credentials={real_user=slave}} blocked reactor thread for 71391.5us
I20260709 13:06:29.029785 13333 tablet.cc:1724] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.037986 13313 tablet.cc:1724] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.038703 13330 tablet.cc:1724] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.040192 13334 tablet.cc:1724] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.020704 13331 tablet.cc:1724] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.045192 13326 tablet.cc:1724] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.049307 13335 tablet.cc:1724] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.100253 13336 tablet.cc:1724] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.101763 13316 raft_consensus.cc:993] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.102355 13316 raft_consensus.cc:1081] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Signalling peer 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.114559 13313 tablet.cc:1724] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.130918 13036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:39156
I20260709 13:06:29.131639 13036 raft_consensus.cc:493] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.132061 13036 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.129732 13334 tablet.cc:1724] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.152006 13330 tablet.cc:1724] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.152940 12887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f"
dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
mode: GRACEFUL
new_leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:43658
I20260709 13:06:29.153620 12887 raft_consensus.cc:606] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Received request to transfer leadership to TS 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:29.157488 12964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9"
dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
mode: GRACEFUL
new_leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:55514
I20260709 13:06:29.158198 12964 raft_consensus.cc:606] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Received request to transfer leadership to TS 0a2cdf08ef784d4593c11c42bf6c5d2a
I20260709 13:06:29.159446   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:29.160099   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:29.172143 13315 tablet.cc:1724] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:29.174476 13036 raft_consensus.cc:515] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.178347 12887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:29.178951 12887 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.186614 12887 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.188031 13036 leader_election.cc:290] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:29.188119 13003 leader_election.cc:304] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:29.189320 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d3d6fc2eb741909d8b21801de96033" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:29.190140 12966 raft_consensus.cc:3055] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:29.190518 13343 raft_consensus.cc:2804] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:29.190551 12966 raft_consensus.cc:740] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:29.191222 13343 raft_consensus.cc:697] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:29.191718 12966 consensus_queue.cc:260] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.192217 13343 consensus_queue.cc:237] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.193096 12966 raft_consensus.cc:3060] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.204622 12788 catalog_manager.cc:5697] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 2 to 3, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.209033 12966 raft_consensus.cc:2468] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.213236 12789 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: soft_deleted_table [id=9f60da3f54b04646a9446176049573be] alter complete (version 1)
I20260709 13:06:29.282709 13175 raft_consensus.cc:993] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: : Instructing follower 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.283322 13279 raft_consensus.cc:1081] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Signalling peer 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.284915 13036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:06:29.285449 13036 raft_consensus.cc:493] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.285790 13036 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.293386 13036 raft_consensus.cc:515] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.294989 13036 leader_election.cc:290] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:29.296067 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:29.295969 12887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "785b40ccad454ddaadb93567cb933a3f" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:29.296888 12966 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.296989 12887 raft_consensus.cc:3055] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:29.297426 12887 raft_consensus.cc:740] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ce4463a21a47afa02aa37a1d09441a, State: Running, Role: LEADER
I20260709 13:06:29.298079 12887 consensus_queue.cc:260] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.299027 12887 raft_consensus.cc:3060] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.302736 12966 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.304023 13006 leader_election.cc:304] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:29.304071 12887 raft_consensus.cc:2468] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.304991 13343 raft_consensus.cc:2804] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:29.305483 13343 raft_consensus.cc:697] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:29.306286 13343 consensus_queue.cc:237] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.314551 12786 catalog_manager.cc:5697] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 2 to 3, leader changed from a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193) to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.430049 13309 raft_consensus.cc:993] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.430701 13346 raft_consensus.cc:1081] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Signalling peer 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.432164 13036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:39156
I20260709 13:06:29.432608 13036 raft_consensus.cc:493] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.432869 13036 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.438061 13036 raft_consensus.cc:515] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.439972 13036 leader_election.cc:290] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:29.440723 12887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:29.440977 12966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b83673be2ef4d758c3fcae0cede98db" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:29.441421 12887 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.441624 12966 raft_consensus.cc:3055] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:29.441982 12966 raft_consensus.cc:740] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:29.442706 12966 consensus_queue.cc:260] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.444849 12966 raft_consensus.cc:3060] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.447623 12887 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.448781 13003 leader_election.cc:304] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, a8ce4463a21a47afa02aa37a1d09441a; no voters: 
I20260709 13:06:29.449582 13328 raft_consensus.cc:2804] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:29.450076 13328 raft_consensus.cc:697] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:29.450923 13328 consensus_queue.cc:237] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.452458 12966 raft_consensus.cc:2468] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 3.
I20260709 13:06:29.458377 12786 catalog_manager.cc:5697] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 2 to 3, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.629989 12887 raft_consensus.cc:1275] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:29.631430 13328 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:29.639710 13309 raft_consensus.cc:993] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: : Instructing follower 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.640419 13346 raft_consensus.cc:1081] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Signalling peer 0a2cdf08ef784d4593c11c42bf6c5d2a to start an election
I20260709 13:06:29.642391 12966 raft_consensus.cc:1275] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:29.645548 13036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9"
dest_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a"
 from {username='slave'} at 127.0.0.1:39156
I20260709 13:06:29.645787 13353 consensus_queue.cc:1048] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:29.646193 13036 raft_consensus.cc:493] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.646623 13036 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.655642 13036 raft_consensus.cc:515] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.658360 13036 leader_election.cc:290] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 2 election: Requested vote from peers a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443), 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447)
I20260709 13:06:29.659351 12887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ce4463a21a47afa02aa37a1d09441a"
I20260709 13:06:29.660322 12887 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.660387 12964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb56c49b42064eea93557f6c053db8b9" candidate_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "579916bf94474548a6de9b8ea1cffb4a"
I20260709 13:06:29.662976 12964 raft_consensus.cc:3055] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.663300 12964 raft_consensus.cc:740] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 579916bf94474548a6de9b8ea1cffb4a, State: Running, Role: LEADER
I20260709 13:06:29.664179 12964 consensus_queue.cc:260] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.665390 12964 raft_consensus.cc:3060] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.671941 12964 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 2.
I20260709 13:06:29.674441 12887 raft_consensus.cc:2468] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2cdf08ef784d4593c11c42bf6c5d2a in term 2.
I20260709 13:06:29.674734 13006 leader_election.cc:304] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2cdf08ef784d4593c11c42bf6c5d2a, 579916bf94474548a6de9b8ea1cffb4a; no voters: 
I20260709 13:06:29.675735 13354 raft_consensus.cc:2804] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:29.676297 13354 raft_consensus.cc:697] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 LEADER]: Becoming Leader. State: Replica: 0a2cdf08ef784d4593c11c42bf6c5d2a, State: Running, Role: LEADER
I20260709 13:06:29.677241 13354 consensus_queue.cc:237] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } }
I20260709 13:06:29.685820 12887 raft_consensus.cc:1275] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:29.685971 12788 catalog_manager.cc:5697] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a reported cstate change: term changed from 1 to 2, leader changed from 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194) to 0a2cdf08ef784d4593c11c42bf6c5d2a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2cdf08ef784d4593c11c42bf6c5d2a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38223 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.687144 13354 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:29.701833 12964 raft_consensus.cc:1275] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:29.703352 13354 consensus_queue.cc:1048] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:29.952808 12964 raft_consensus.cc:1275] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:29.954033 13354 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:29.961498 12888 raft_consensus.cc:1275] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:29.963146 13343 consensus_queue.cc:1048] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:30.152678 12964 raft_consensus.cc:1275] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:30.153848 13343 consensus_queue.cc:1048] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "579916bf94474548a6de9b8ea1cffb4a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:30.162900 12887 raft_consensus.cc:1275] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Refusing update from remote peer 0a2cdf08ef784d4593c11c42bf6c5d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:30.165004 13328 consensus_queue.cc:1048] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ce4463a21a47afa02aa37a1d09441a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:30.228992   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:30.235929   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:06:30.236449   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:30.241381   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:06:30.241757   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:31.274367   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:31.305171   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:31.306061   431 tablet_replica.cc:333] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.307081   431 raft_consensus.cc:2243] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.308156   431 raft_consensus.cc:2272] T f80854bf157f4a54b3c61baaad17db99 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.310165   431 tablet_replica.cc:333] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.310693   431 raft_consensus.cc:2243] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.311051   431 raft_consensus.cc:2272] T 58d3d6fc2eb741909d8b21801de96033 P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.313184   431 tablet_replica.cc:333] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.313760   431 raft_consensus.cc:2243] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.314179   431 raft_consensus.cc:2272] T 6b83673be2ef4d758c3fcae0cede98db P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.316085   431 tablet_replica.cc:333] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.316672   431 raft_consensus.cc:2243] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.317173   431 raft_consensus.cc:2272] T c52e2f491bf14241bcbfcfb6314b88ae P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.319089   431 tablet_replica.cc:333] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.319744   431 raft_consensus.cc:2243] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.320268   431 raft_consensus.cc:2272] T 7fb3b2593d934f60a42794b62a86f3ba P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.322223   431 tablet_replica.cc:333] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.322746   431 raft_consensus.cc:2243] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.323531   431 raft_consensus.cc:2272] T 63548b1cfe3b41aea9086ab46cf5df6c P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.326304   431 tablet_replica.cc:333] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.326924   431 raft_consensus.cc:2243] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.327762   431 raft_consensus.cc:2272] T 9325d6826fd14512839a4b2c93d2298e P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:31.329909 13003 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
I20260709 13:06:31.330515   431 tablet_replica.cc:333] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.331477   431 raft_consensus.cc:2243] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.331949   431 raft_consensus.cc:2272] T 785b40ccad454ddaadb93567cb933a3f P a8ce4463a21a47afa02aa37a1d09441a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:31.333644 13003 consensus_peers.cc:597] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.334625   431 tablet_replica.cc:333] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.335325   431 raft_consensus.cc:2243] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.335812   431 raft_consensus.cc:2272] T 29882fd52664468c848fd7e5c82b7907 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.338037   431 tablet_replica.cc:333] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.338735   431 raft_consensus.cc:2243] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.339228   431 raft_consensus.cc:2272] T fb56c49b42064eea93557f6c053db8b9 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.342115   431 tablet_replica.cc:333] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.342747   431 raft_consensus.cc:2243] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.343258   431 raft_consensus.cc:2272] T c7eea2e74a5e4380b6fef269d5030e4f P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.345489   431 tablet_replica.cc:333] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.346060   431 raft_consensus.cc:2243] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.346797   431 raft_consensus.cc:2272] T 596bd6b656f245809e62b2e35f3be4c5 P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.348778   431 tablet_replica.cc:333] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.349320   431 raft_consensus.cc:2243] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.349761   431 raft_consensus.cc:2272] T 1bbc272093324888a170e4c87badda6c P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.351743   431 tablet_replica.cc:333] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.352293   431 raft_consensus.cc:2243] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.352702   431 raft_consensus.cc:2272] T 52c0f24d250a4bcbae0d8e08f2bd813e P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.354539   431 tablet_replica.cc:333] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.355029   431 raft_consensus.cc:2243] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.355437   431 raft_consensus.cc:2272] T c8d4d0352f524065a1ce767e0081e2fc P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.357270   431 tablet_replica.cc:333] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.357785   431 raft_consensus.cc:2243] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.358199   431 raft_consensus.cc:2272] T 8cadafed43a34bfcacbab799f708ce5a P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.360034   431 tablet_replica.cc:333] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.360512   431 raft_consensus.cc:2243] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.361297   431 raft_consensus.cc:2272] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P a8ce4463a21a47afa02aa37a1d09441a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.363389   431 tablet_replica.cc:333] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a: stopping tablet replica
I20260709 13:06:31.363922   431 raft_consensus.cc:2243] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.364673   431 raft_consensus.cc:2272] T 83f3533d969441f28f11504f41dd42cb P a8ce4463a21a47afa02aa37a1d09441a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.405560   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:31.425074 12929 consensus_peers.cc:597] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.437345 12929 consensus_peers.cc:597] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.450254   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:31.460235 13006 consensus_peers.cc:597] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Couldn't send request to peer 579916bf94474548a6de9b8ea1cffb4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.469385 13003 consensus_peers.cc:597] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.481449 12929 consensus_peers.cc:597] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.487488   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:31.488523   431 tablet_replica.cc:333] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
W20260709 13:06:31.488597 13006 consensus_peers.cc:597] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Couldn't send request to peer 579916bf94474548a6de9b8ea1cffb4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.489259   431 raft_consensus.cc:2243] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.489787   431 raft_consensus.cc:2272] T 58d3d6fc2eb741909d8b21801de96033 P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.491750   431 tablet_replica.cc:333] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.492375   431 raft_consensus.cc:2243] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.492821   431 raft_consensus.cc:2272] T f80854bf157f4a54b3c61baaad17db99 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.494792   431 tablet_replica.cc:333] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.495342   431 raft_consensus.cc:2243] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.495750   431 raft_consensus.cc:2272] T c52e2f491bf14241bcbfcfb6314b88ae P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.497480   431 tablet_replica.cc:333] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.498061   431 raft_consensus.cc:2243] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.498811   431 raft_consensus.cc:2272] T 7fb3b2593d934f60a42794b62a86f3ba P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.500854   431 tablet_replica.cc:333] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.501444   431 raft_consensus.cc:2243] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.501857   431 raft_consensus.cc:2272] T 6b83673be2ef4d758c3fcae0cede98db P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.503721   431 tablet_replica.cc:333] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.504334   431 raft_consensus.cc:2243] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.504801   431 raft_consensus.cc:2272] T 785b40ccad454ddaadb93567cb933a3f P 579916bf94474548a6de9b8ea1cffb4a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.506649   431 tablet_replica.cc:333] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.507212   431 raft_consensus.cc:2243] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.508148   431 raft_consensus.cc:2272] T 29882fd52664468c848fd7e5c82b7907 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.510118   431 tablet_replica.cc:333] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.510727   431 raft_consensus.cc:2243] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.511126   431 raft_consensus.cc:2272] T 63548b1cfe3b41aea9086ab46cf5df6c P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.513015   431 tablet_replica.cc:333] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.513616   431 raft_consensus.cc:2243] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.514039   431 raft_consensus.cc:2272] T 9325d6826fd14512839a4b2c93d2298e P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.515847   431 tablet_replica.cc:333] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.516388   431 raft_consensus.cc:2243] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.516785   431 raft_consensus.cc:2272] T fb56c49b42064eea93557f6c053db8b9 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.518725   431 tablet_replica.cc:333] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.519233   431 raft_consensus.cc:2243] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.519990   431 raft_consensus.cc:2272] T c8d4d0352f524065a1ce767e0081e2fc P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.521652   431 tablet_replica.cc:333] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.522164   431 raft_consensus.cc:2243] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.522677   431 raft_consensus.cc:2272] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.524793   431 tablet_replica.cc:333] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.525329   431 raft_consensus.cc:2243] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.526033   431 raft_consensus.cc:2272] T 8cadafed43a34bfcacbab799f708ce5a P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.527879   431 tablet_replica.cc:333] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.528474   431 raft_consensus.cc:2243] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.529013   431 raft_consensus.cc:2272] T c7eea2e74a5e4380b6fef269d5030e4f P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.531549   431 tablet_replica.cc:333] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
W20260709 13:06:31.532217 13006 consensus_peers.cc:597] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Couldn't send request to peer 579916bf94474548a6de9b8ea1cffb4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.532234   431 raft_consensus.cc:2243] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.533275   431 raft_consensus.cc:2272] T 52c0f24d250a4bcbae0d8e08f2bd813e P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.535467   431 tablet_replica.cc:333] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.535992   431 raft_consensus.cc:2243] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.536468   431 raft_consensus.cc:2272] T 596bd6b656f245809e62b2e35f3be4c5 P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.538439   431 tablet_replica.cc:333] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.538911   431 raft_consensus.cc:2243] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.539711   431 raft_consensus.cc:2272] T 1bbc272093324888a170e4c87badda6c P 579916bf94474548a6de9b8ea1cffb4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.541669   431 tablet_replica.cc:333] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a: stopping tablet replica
I20260709 13:06:31.542114   431 raft_consensus.cc:2243] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.542569   431 raft_consensus.cc:2272] T 83f3533d969441f28f11504f41dd42cb P 579916bf94474548a6de9b8ea1cffb4a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:31.548162 13003 consensus_peers.cc:597] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.571382 13006 consensus_peers.cc:597] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Couldn't send request to peer 579916bf94474548a6de9b8ea1cffb4a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.582674 13003 consensus_peers.cc:597] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.584834   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:31.615729 13006 consensus_peers.cc:597] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer 579916bf94474548a6de9b8ea1cffb4a (127.0.107.194:43447): Couldn't send request to peer 579916bf94474548a6de9b8ea1cffb4a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.621402   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:31.635473 13003 consensus_peers.cc:597] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a -> Peer a8ce4463a21a47afa02aa37a1d09441a (127.0.107.193:38443): Couldn't send request to peer a8ce4463a21a47afa02aa37a1d09441a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.652563   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:31.653453   431 tablet_replica.cc:333] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.654253   431 raft_consensus.cc:2243] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.655035   431 raft_consensus.cc:2272] T 58d3d6fc2eb741909d8b21801de96033 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.657116   431 tablet_replica.cc:333] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.657735   431 raft_consensus.cc:2243] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.658469   431 raft_consensus.cc:2272] T c52e2f491bf14241bcbfcfb6314b88ae P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.660418   431 tablet_replica.cc:333] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.660979   431 raft_consensus.cc:2243] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.661412   431 raft_consensus.cc:2272] T 7fb3b2593d934f60a42794b62a86f3ba P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.663184   431 tablet_replica.cc:333] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.663720   431 raft_consensus.cc:2243] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.664222   431 raft_consensus.cc:2272] T f80854bf157f4a54b3c61baaad17db99 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.666057   431 tablet_replica.cc:333] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.666594   431 raft_consensus.cc:2243] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.667429   431 raft_consensus.cc:2272] T 6b83673be2ef4d758c3fcae0cede98db P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.669489   431 tablet_replica.cc:333] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.670038   431 raft_consensus.cc:2243] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.670481   431 raft_consensus.cc:2272] T 1bbc272093324888a170e4c87badda6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.672531   431 tablet_replica.cc:333] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.673069   431 raft_consensus.cc:2243] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.673483   431 raft_consensus.cc:2272] T 596bd6b656f245809e62b2e35f3be4c5 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.675266   431 tablet_replica.cc:333] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.675896   431 raft_consensus.cc:2243] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.676640   431 raft_consensus.cc:2272] T 785b40ccad454ddaadb93567cb933a3f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.678443   431 tablet_replica.cc:333] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.678943   431 raft_consensus.cc:2243] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.679388   431 raft_consensus.cc:2272] T 29882fd52664468c848fd7e5c82b7907 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.681350   431 tablet_replica.cc:333] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.681870   431 raft_consensus.cc:2243] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.682616   431 raft_consensus.cc:2272] T fb56c49b42064eea93557f6c053db8b9 P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.684697   431 tablet_replica.cc:333] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.685233   431 raft_consensus.cc:2243] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.685643   431 raft_consensus.cc:2272] T c8d4d0352f524065a1ce767e0081e2fc P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.687326   431 tablet_replica.cc:333] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.687867   431 raft_consensus.cc:2243] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.688282   431 raft_consensus.cc:2272] T 8cadafed43a34bfcacbab799f708ce5a P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.690084   431 tablet_replica.cc:333] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.690598   431 raft_consensus.cc:2243] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.691015   431 raft_consensus.cc:2272] T 4b1fdaa5a06b42abaf04c2e79bbf7d1d P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.693055   431 tablet_replica.cc:333] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.693584   431 raft_consensus.cc:2243] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.693992   431 raft_consensus.cc:2272] T 52c0f24d250a4bcbae0d8e08f2bd813e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.695910   431 tablet_replica.cc:333] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.696434   431 raft_consensus.cc:2243] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.696872   431 raft_consensus.cc:2272] T 83f3533d969441f28f11504f41dd42cb P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.698763   431 tablet_replica.cc:333] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.699265   431 raft_consensus.cc:2243] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.700136   431 raft_consensus.cc:2272] T c7eea2e74a5e4380b6fef269d5030e4f P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.702054   431 tablet_replica.cc:333] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.702549   431 raft_consensus.cc:2243] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.702972   431 raft_consensus.cc:2272] T 63548b1cfe3b41aea9086ab46cf5df6c P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.704950   431 tablet_replica.cc:333] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a: stopping tablet replica
I20260709 13:06:31.705448   431 raft_consensus.cc:2243] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:31.705852   431 raft_consensus.cc:2272] T 9325d6826fd14512839a4b2c93d2298e P 0a2cdf08ef784d4593c11c42bf6c5d2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.743567   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:31.777771   431 master.cc:562] Master@127.0.107.254:39485 shutting down...
I20260709 13:06:31.800017   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:31.800590   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:31.800932   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 48b59da8fab84ad4a5e3da5aaa583a17: stopping tablet replica
I20260709 13:06:31.822103   431 master.cc:584] Master@127.0.107.254:39485 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12741 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:06:31.874507   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:33307
I20260709 13:06:31.875732   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:31.881639 13370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:31.882203 13373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:31.882728 13371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:31.882784   431 server_base.cc:1061] running on GCE node
I20260709 13:06:31.884478   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:31.884665   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:31.884781   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602391884771 us; error 0 us; skew 500 ppm
I20260709 13:06:31.885218   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:31.887542   431 webserver.cc:533] Webserver started at http://127.0.107.254:42859/ using document root <none> and password file <none>
I20260709 13:06:31.888042   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:31.888199   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:31.888408   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:31.889423   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/master-0-root/instance:
uuid: "3b18de68dd994c20a9947a801bbec2c5"
format_stamp: "Formatted at 2026-07-09 13:06:31 on dist-test-slave-kgkf"
I20260709 13:06:31.893782   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:06:31.896943 13378 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:31.897680   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:31.897962   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/master-0-root
uuid: "3b18de68dd994c20a9947a801bbec2c5"
format_stamp: "Formatted at 2026-07-09 13:06:31 on dist-test-slave-kgkf"
I20260709 13:06:31.898202   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:31.909932   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:31.911212   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:31.948582   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:33307
I20260709 13:06:31.948693 13429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:33307 every 8 connection(s)
I20260709 13:06:31.952512 13430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:31.962782 13430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: Bootstrap starting.
I20260709 13:06:31.967432 13430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.971572 13430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: No bootstrap required, opened a new log
I20260709 13:06:31.973719 13430 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER }
I20260709 13:06:31.974076 13430 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.974258 13430 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b18de68dd994c20a9947a801bbec2c5, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.974833 13430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER }
I20260709 13:06:31.975248 13430 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:31.975431 13430 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:31.975646 13430 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.980209 13430 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER }
I20260709 13:06:31.980644 13430 leader_election.cc:304] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b18de68dd994c20a9947a801bbec2c5; no voters: 
I20260709 13:06:31.981706 13430 leader_election.cc:290] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:31.982064 13433 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.983458 13433 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3b18de68dd994c20a9947a801bbec2c5, State: Running, Role: LEADER
I20260709 13:06:31.984162 13433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER }
I20260709 13:06:31.984607 13430 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:31.988592 13434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b18de68dd994c20a9947a801bbec2c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER } }
I20260709 13:06:31.989320 13434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:31.990736 13435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b18de68dd994c20a9947a801bbec2c5. Latest consensus state: current_term: 1 leader_uuid: "3b18de68dd994c20a9947a801bbec2c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b18de68dd994c20a9947a801bbec2c5" member_type: VOTER } }
I20260709 13:06:31.991575 13435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:31.991688 13439 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:31.996747 13439 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:32.001463   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:32.005927 13439 catalog_manager.cc:1383] Generated new cluster ID: 373804d8603749c79e62b9793cc1ea9c
I20260709 13:06:32.006225 13439 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:32.022200 13439 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:32.023590 13439 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:32.038497 13439 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: Generated new TSK 0
I20260709 13:06:32.039135 13439 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:32.068125   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:32.074295 13451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:32.075199 13452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:32.076962 13454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:32.077471   431 server_base.cc:1061] running on GCE node
I20260709 13:06:32.078378   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:32.078583   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:32.078755   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602392078721 us; error 0 us; skew 500 ppm
I20260709 13:06:32.079281   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:32.081542   431 webserver.cc:533] Webserver started at http://127.0.107.193:38321/ using document root <none> and password file <none>
I20260709 13:06:32.082020   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:32.082253   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:32.082535   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:32.083599   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-0-root/instance:
uuid: "30f13adabb264ed19bc6cbdf1f873538"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.088215   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:06:32.091467 13459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.092290   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:32.092588   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-0-root
uuid: "30f13adabb264ed19bc6cbdf1f873538"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.092885   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:32.107522   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:32.108807   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:32.110312   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:32.112639   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:32.112860   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.113111   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:32.113308   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.152670   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:37729
I20260709 13:06:32.152801 13521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:37729 every 8 connection(s)
I20260709 13:06:32.157860   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:32.165103 13526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:32.166115 13527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:32.169261   431 server_base.cc:1061] running on GCE node
W20260709 13:06:32.169895 13529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:32.170668 13522 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33307
I20260709 13:06:32.171094 13522 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:32.171100   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:32.171496   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:32.171675   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602392171660 us; error 0 us; skew 500 ppm
I20260709 13:06:32.172026 13522 heartbeater.cc:507] Master 127.0.107.254:33307 requested a full tablet report, sending...
I20260709 13:06:32.172369   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:32.174144 13395 ts_manager.cc:194] Registered new tserver with Master: 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729)
I20260709 13:06:32.175117   431 webserver.cc:533] Webserver started at http://127.0.107.194:34127/ using document root <none> and password file <none>
I20260709 13:06:32.175649   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:32.175884   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:32.176151   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:32.176242 13395 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39080
I20260709 13:06:32.177323   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-1-root/instance:
uuid: "f1cfe6266c2c461598761f06c3f46e26"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.181792   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:32.184880 13534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.185653   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:32.185935   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-1-root
uuid: "f1cfe6266c2c461598761f06c3f46e26"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.186204   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:32.197480   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:32.198608   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:32.200130   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:32.202273   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:32.202490   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.202721   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:32.202885   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.242470   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:35013
I20260709 13:06:32.242583 13596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:35013 every 8 connection(s)
I20260709 13:06:32.247275   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:32.254346 13600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:32.256196 13601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:32.259438 13603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:32.260039 13597 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33307
I20260709 13:06:32.260571 13597 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:32.260826   431 server_base.cc:1061] running on GCE node
I20260709 13:06:32.261561 13597 heartbeater.cc:507] Master 127.0.107.254:33307 requested a full tablet report, sending...
I20260709 13:06:32.261727   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:32.262063   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:32.262245   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602392262224 us; error 0 us; skew 500 ppm
I20260709 13:06:32.262813   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:32.264215 13395 ts_manager.cc:194] Registered new tserver with Master: f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:32.265591   431 webserver.cc:533] Webserver started at http://127.0.107.195:33335/ using document root <none> and password file <none>
I20260709 13:06:32.266287   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:32.266189 13395 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39082
I20260709 13:06:32.266633   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:32.266974   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:32.268478   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-2-root/instance:
uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.272912   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:32.276223 13608 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.276984   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:32.277285   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-2-root
uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
format_stamp: "Formatted at 2026-07-09 13:06:32 on dist-test-slave-kgkf"
I20260709 13:06:32.277549   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskseTR8j/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601992069626-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:32.289489   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:32.290580   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:32.292001   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:32.294157   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:32.294364   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.294621   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:32.294819   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:32.336067   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:40043
I20260709 13:06:32.336164 13670 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:40043 every 8 connection(s)
I20260709 13:06:32.349646 13671 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33307
I20260709 13:06:32.350117 13671 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:32.350835 13671 heartbeater.cc:507] Master 127.0.107.254:33307 requested a full tablet report, sending...
I20260709 13:06:32.352743 13395 ts_manager.cc:194] Registered new tserver with Master: b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:32.352900   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013719927s
I20260709 13:06:32.354136 13395 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39094
I20260709 13:06:32.378800 13395 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39096:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:32.381491 13395 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:32.452450 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 3f8978ace07e4cb0a4dd24c207d6fac9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:32.454219 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f8978ace07e4cb0a4dd24c207d6fac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.455741 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 85f8f68254cc4ebea8476631126ffc2a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:32.458364 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f8f68254cc4ebea8476631126ffc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.460840 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4799af8d42433e9d10138bd79ebaa9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:32.462280 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4799af8d42433e9d10138bd79ebaa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.464943 13633 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa7b5859d487cbfad3f0090a3ad78 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:32.466384 13633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa7b5859d487cbfad3f0090a3ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.468986 13632 tablet_service.cc:1511] Processing CreateTablet for tablet da7a38f0ae5848ac846044718257dad7 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:32.470460 13632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7a38f0ae5848ac846044718257dad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.474452 13631 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff234e22c0845c4a8ad42671e104934 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:32.475975 13631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff234e22c0845c4a8ad42671e104934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.478664 13630 tablet_service.cc:1511] Processing CreateTablet for tablet b2b743418e294d1a9d3eaa20ea044b75 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:32.480254 13630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b743418e294d1a9d3eaa20ea044b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.481431 13629 tablet_service.cc:1511] Processing CreateTablet for tablet 145fa6a445994668a08b1d954a9c227d (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:32.482914 13629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 145fa6a445994668a08b1d954a9c227d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.493558 13562 tablet_service.cc:1511] Processing CreateTablet for tablet 3f8978ace07e4cb0a4dd24c207d6fac9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:32.495070 13562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f8978ace07e4cb0a4dd24c207d6fac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.497515 13561 tablet_service.cc:1511] Processing CreateTablet for tablet 85f8f68254cc4ebea8476631126ffc2a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:32.498965 13561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f8f68254cc4ebea8476631126ffc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.501386 13560 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4799af8d42433e9d10138bd79ebaa9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:32.501389 13628 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3b361c05214d7686b6a29ae887d92a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:32.504200 13560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4799af8d42433e9d10138bd79ebaa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.504758 13559 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa7b5859d487cbfad3f0090a3ad78 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:32.506091 13559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa7b5859d487cbfad3f0090a3ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.509260 13628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3b361c05214d7686b6a29ae887d92a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.524020 13484 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa7b5859d487cbfad3f0090a3ad78 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:32.524947 13558 tablet_service.cc:1511] Processing CreateTablet for tablet da7a38f0ae5848ac846044718257dad7 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:32.525640 13484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa7b5859d487cbfad3f0090a3ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.526340 13558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7a38f0ae5848ac846044718257dad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.528828 13486 tablet_service.cc:1511] Processing CreateTablet for tablet 85f8f68254cc4ebea8476631126ffc2a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:32.530226 13486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f8f68254cc4ebea8476631126ffc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.529968 13482 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff234e22c0845c4a8ad42671e104934 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:32.531296 13482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff234e22c0845c4a8ad42671e104934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.528834 13483 tablet_service.cc:1511] Processing CreateTablet for tablet da7a38f0ae5848ac846044718257dad7 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:32.533579 13487 tablet_service.cc:1511] Processing CreateTablet for tablet 3f8978ace07e4cb0a4dd24c207d6fac9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:32.535138 13557 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff234e22c0845c4a8ad42671e104934 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:32.536475 13557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff234e22c0845c4a8ad42671e104934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.538398 13485 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4799af8d42433e9d10138bd79ebaa9 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:32.542244 13556 tablet_service.cc:1511] Processing CreateTablet for tablet b2b743418e294d1a9d3eaa20ea044b75 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:32.544903 13556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b743418e294d1a9d3eaa20ea044b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.549057 13483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7a38f0ae5848ac846044718257dad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.551694 13555 tablet_service.cc:1511] Processing CreateTablet for tablet 145fa6a445994668a08b1d954a9c227d (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:32.552227 13487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f8978ace07e4cb0a4dd24c207d6fac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.553102 13555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 145fa6a445994668a08b1d954a9c227d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.556213 13485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4799af8d42433e9d10138bd79ebaa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.556465 13554 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3b361c05214d7686b6a29ae887d92a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:32.561012 13554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3b361c05214d7686b6a29ae887d92a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.561113 13481 tablet_service.cc:1511] Processing CreateTablet for tablet b2b743418e294d1a9d3eaa20ea044b75 (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:32.562505 13481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b743418e294d1a9d3eaa20ea044b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.563633 13480 tablet_service.cc:1511] Processing CreateTablet for tablet 145fa6a445994668a08b1d954a9c227d (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:32.565011 13480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 145fa6a445994668a08b1d954a9c227d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.569005 13479 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3b361c05214d7686b6a29ae887d92a (DEFAULT_TABLE table=test-workload [id=79f0c6dc097d4001b47a1271a79338f4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:32.570502 13479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3b361c05214d7686b6a29ae887d92a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:32.687415 13686 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:32.695084 13688 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:32.697551 13686 tablet_bootstrap.cc:654] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.692066 13687 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.719497 13687 tablet_bootstrap.cc:654] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.720801 13688 tablet_bootstrap.cc:654] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.722993 13686 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:32.723538 13686 ts_tablet_manager.cc:1403] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.008s
I20260709 13:06:32.727578 13686 raft_consensus.cc:359] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.728655 13686 raft_consensus.cc:385] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.729074 13686 raft_consensus.cc:740] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.730855 13687 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.730412 13686 consensus_queue.cc:260] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.731419 13687 ts_tablet_manager.cc:1403] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.009s
I20260709 13:06:32.737725 13671 heartbeater.cc:499] Master 127.0.107.254:33307 was elected leader, sending a full tablet report...
I20260709 13:06:32.738319 13686 ts_tablet_manager.cc:1434] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 13:06:32.738651 13688 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:32.738319 13687 raft_consensus.cc:359] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:32.739307 13688 ts_tablet_manager.cc:1403] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.009s
I20260709 13:06:32.739492 13687 raft_consensus.cc:385] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.739904 13687 raft_consensus.cc:740] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.739981 13686 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:32.740761 13687 consensus_queue.cc:260] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:32.742631 13688 raft_consensus.cc:359] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.743438 13688 raft_consensus.cc:385] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.743743 13688 raft_consensus.cc:740] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.744513 13688 consensus_queue.cc:260] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.749290 13686 tablet_bootstrap.cc:654] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.764189 13686 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:32.765021 13686 ts_tablet_manager.cc:1403] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.013s
I20260709 13:06:32.768946 13686 raft_consensus.cc:359] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.770010 13686 raft_consensus.cc:385] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.770449 13686 raft_consensus.cc:740] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.771733 13686 consensus_queue.cc:260] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.774770 13686 ts_tablet_manager.cc:1434] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:06:32.776197 13686 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:32.785332 13686 tablet_bootstrap.cc:654] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.793107 13597 heartbeater.cc:499] Master 127.0.107.254:33307 was elected leader, sending a full tablet report...
I20260709 13:06:32.793680 13687 ts_tablet_manager.cc:1434] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.062s	user 0.018s	sys 0.041s
I20260709 13:06:32.794798 13688 ts_tablet_manager.cc:1434] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.049s
I20260709 13:06:32.794790 13687 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.795061 13522 heartbeater.cc:499] Master 127.0.107.254:33307 was elected leader, sending a full tablet report...
I20260709 13:06:32.796011 13688 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:32.802402 13687 tablet_bootstrap.cc:654] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.802429 13688 tablet_bootstrap.cc:654] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.803984 13686 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:32.804666 13686 ts_tablet_manager.cc:1403] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.005s
I20260709 13:06:32.808236 13694 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.808540 13686 raft_consensus.cc:359] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.810046 13686 raft_consensus.cc:385] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.810544 13686 raft_consensus.cc:740] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.812048 13686 consensus_queue.cc:260] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.814188 13686 ts_tablet_manager.cc:1434] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:06:32.815335 13686 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:32.817121 13687 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.817734 13687 ts_tablet_manager.cc:1403] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.023s	user 0.005s	sys 0.013s
I20260709 13:06:32.809060 13694 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.821009 13687 raft_consensus.cc:359] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.821714 13687 raft_consensus.cc:385] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.822005 13687 raft_consensus.cc:740] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.822253 13686 tablet_bootstrap.cc:654] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.822804 13687 consensus_queue.cc:260] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.825114 13687 ts_tablet_manager.cc:1434] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:32.826392 13687 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.833727 13694 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:32.839924 13687 tablet_bootstrap.cc:654] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.853142 13687 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.853817 13687 ts_tablet_manager.cc:1403] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:06:32.857995 13687 raft_consensus.cc:359] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.858975 13687 raft_consensus.cc:385] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.859368 13687 raft_consensus.cc:740] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.860266 13693 raft_consensus.cc:493] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.860397 13687 consensus_queue.cc:260] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.861182 13693 raft_consensus.cc:515] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:32.867081 13692 raft_consensus.cc:493] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.867748 13692 raft_consensus.cc:515] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.870543 13687 ts_tablet_manager.cc:1434] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.007s
I20260709 13:06:32.872066 13687 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.873188 13692 leader_election.cc:290] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:32.880153 13687 tablet_bootstrap.cc:654] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.894013 13687 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.894747 13687 ts_tablet_manager.cc:1403] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.013s
I20260709 13:06:32.898986 13687 raft_consensus.cc:359] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.900081 13687 raft_consensus.cc:385] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.900506 13687 raft_consensus.cc:740] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.901585 13687 consensus_queue.cc:260] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.904758 13687 ts_tablet_manager.cc:1434] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.004s
I20260709 13:06:32.906231 13687 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.911943 13693 leader_election.cc:290] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043), 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729)
I20260709 13:06:32.914861 13687 tablet_bootstrap.cc:654] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.931384 13687 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.932124 13687 ts_tablet_manager.cc:1403] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:06:32.936658 13687 raft_consensus.cc:359] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.937731 13687 raft_consensus.cc:385] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.938110 13687 raft_consensus.cc:740] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.939095 13687 consensus_queue.cc:260] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.942322 13687 ts_tablet_manager.cc:1434] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:32.943800 13687 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.953243 13687 tablet_bootstrap.cc:654] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.966835 13687 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:32.967557 13687 ts_tablet_manager.cc:1403] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:06:32.968320 13692 raft_consensus.cc:493] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.969105 13692 raft_consensus.cc:515] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.971635 13687 raft_consensus.cc:359] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.972213 13692 leader_election.cc:290] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:32.972700 13687 raft_consensus.cc:385] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.973189 13687 raft_consensus.cc:740] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.974747 13687 consensus_queue.cc:260] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:32.979372 13687 ts_tablet_manager.cc:1434] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:06:32.981230 13687 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:32.990990 13687 tablet_bootstrap.cc:654] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.001216 13688 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.001819 13688 ts_tablet_manager.cc:1403] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.206s	user 0.076s	sys 0.115s
I20260709 13:06:33.004235 13687 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:33.005162 13687 ts_tablet_manager.cc:1403] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.009s
I20260709 13:06:33.005023 13688 raft_consensus.cc:359] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.005897 13688 raft_consensus.cc:385] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.006223 13688 raft_consensus.cc:740] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.007025 13688 consensus_queue.cc:260] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.009120 13687 raft_consensus.cc:359] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.009979 13687 raft_consensus.cc:385] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.010281 13687 raft_consensus.cc:740] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.010703 13688 ts_tablet_manager.cc:1434] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:33.011139 13687 consensus_queue.cc:260] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.011900 13688 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.013813 13687 ts_tablet_manager.cc:1434] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:06:33.014953 13687 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:33.019052 13688 tablet_bootstrap.cc:654] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.020417 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.021297 13687 tablet_bootstrap.cc:654] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:33.022238 13535 leader_election.cc:343] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.026207 13687 tablet_bootstrap.cc:492] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:33.026674 13687 ts_tablet_manager.cc:1403] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:33.029165 13687 raft_consensus.cc:359] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.029356 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.029870 13687 raft_consensus.cc:385] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.030189 13687 raft_consensus.cc:740] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.030922 13687 consensus_queue.cc:260] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.031328 13686 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:33.031940 13686 ts_tablet_manager.cc:1403] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.217s	user 0.057s	sys 0.125s
I20260709 13:06:33.033224 13687 ts_tablet_manager.cc:1434] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:33.034204 13687 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
W20260709 13:06:33.034547 13538 leader_election.cc:343] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.035286 13538 leader_election.cc:304] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:33.036463 13693 raft_consensus.cc:2749] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:33.034916 13686 raft_consensus.cc:359] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.037215 13686 raft_consensus.cc:385] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.037575 13686 raft_consensus.cc:740] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.038386 13686 consensus_queue.cc:260] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.040663 13686 ts_tablet_manager.cc:1434] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:06:33.041806 13686 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
W20260709 13:06:33.045507 13443 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:33.051504 13686 tablet_bootstrap.cc:654] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.052150 13687 tablet_bootstrap.cc:654] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.052171 13688 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.052901 13688 ts_tablet_manager.cc:1403] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.021s
I20260709 13:06:33.056046 13688 raft_consensus.cc:359] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.056797 13688 raft_consensus.cc:385] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.057101 13688 raft_consensus.cc:740] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.057905 13688 consensus_queue.cc:260] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.060369 13688 ts_tablet_manager.cc:1434] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:33.061450 13688 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.062476 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.064672 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.069804 13688 tablet_bootstrap.cc:654] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:33.075309 13609 leader_election.cc:343] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:33.076514 13609 leader_election.cc:343] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.077955 13693 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.078779 13693 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.081565 13693 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.082823 13688 tablet_bootstrap.cc:492] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.083441 13688 ts_tablet_manager.cc:1403] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 13:06:33.087001 13688 raft_consensus.cc:359] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.087911 13688 raft_consensus.cc:385] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.088212 13688 raft_consensus.cc:740] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.089066 13688 consensus_queue.cc:260] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.092828 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.093492 13688 ts_tablet_manager.cc:1434] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 13:06:33.093603 13497 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:33.094861 13688 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.101632 13688 tablet_bootstrap.cc:654] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.103893 13535 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.105016 13693 raft_consensus.cc:2804] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.105525 13693 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.105953 13693 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.113907 13688 tablet_bootstrap.cc:492] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.114599 13688 ts_tablet_manager.cc:1403] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:06:33.117621 13688 raft_consensus.cc:359] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.118404 13688 raft_consensus.cc:385] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.118707 13688 raft_consensus.cc:740] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.119686 13688 consensus_queue.cc:260] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.122263 13688 ts_tablet_manager.cc:1434] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:33.123286 13688 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.128207 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.128928 13646 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:33.129890 13688 tablet_bootstrap.cc:654] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.130302 13703 raft_consensus.cc:493] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.133953 13703 raft_consensus.cc:515] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.131515 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.136104 13572 raft_consensus.cc:2468] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:33.132784 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.137480 13571 raft_consensus.cc:2468] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:33.133039 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.138679 13646 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:33.140797 13703 leader_election.cc:290] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.141883 13463 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:33.143100 13694 raft_consensus.cc:2804] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.143571 13694 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.144038 13694 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.158777 13694 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.165726 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.167553 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b743418e294d1a9d3eaa20ea044b75" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.184927 13497 raft_consensus.cc:2468] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:33.172842 13612 leader_election.cc:304] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.187357 13612 leader_election.cc:304] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.175618 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b743418e294d1a9d3eaa20ea044b75" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.190258 13692 raft_consensus.cc:2804] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.190815 13692 raft_consensus.cc:493] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.191254 13692 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.200726 13535 leader_election.cc:304] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.202736 13703 raft_consensus.cc:2804] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.203246 13703 raft_consensus.cc:493] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.203701 13703 raft_consensus.cc:3060] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:33.212560 13538 leader_election.cc:343] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.223830 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.232982 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.233881 13646 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.245404 13646 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:33.247210 13463 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:33.248773 13696 raft_consensus.cc:2804] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.268364 13723 raft_consensus.cc:2804] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.269148 13723 raft_consensus.cc:493] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.269582 13723 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.272908 13725 raft_consensus.cc:493] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.273715 13725 raft_consensus.cc:515] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
W20260709 13:06:33.278560 13612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:35013, user_credentials={real_user=slave}} blocked reactor thread for 91317us
I20260709 13:06:33.279979 13694 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.283435 13696 raft_consensus.cc:697] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:33.284596 13687 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:33.285249 13687 ts_tablet_manager.cc:1403] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.251s	user 0.069s	sys 0.156s
I20260709 13:06:33.285502 13703 raft_consensus.cc:515] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.286530 13686 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:33.287420 13686 ts_tablet_manager.cc:1403] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.246s	user 0.079s	sys 0.144s
I20260709 13:06:33.284607 13696 consensus_queue.cc:237] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.288767 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b743418e294d1a9d3eaa20ea044b75" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.289371 13497 raft_consensus.cc:3060] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.291122 13687 raft_consensus.cc:359] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.293000 13687 raft_consensus.cc:385] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.293310 13687 raft_consensus.cc:740] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.291839 13686 raft_consensus.cc:359] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.294880 13686 raft_consensus.cc:385] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.295213 13686 raft_consensus.cc:740] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.296097 13686 consensus_queue.cc:260] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.294047 13687 consensus_queue.cc:260] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.182919 13693 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.299839 13686 ts_tablet_manager.cc:1434] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:33.300909 13686 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:33.311549 13571 raft_consensus.cc:2393] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30f13adabb264ed19bc6cbdf1f873538 in current term 1: Already voted for candidate f1cfe6266c2c461598761f06c3f46e26 in this term.
I20260709 13:06:33.314209 13686 tablet_bootstrap.cc:654] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.315408 13395 catalog_manager.cc:5697] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.321233 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.323172 13646 raft_consensus.cc:2393] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f1cfe6266c2c461598761f06c3f46e26 in current term 1: Already voted for candidate 30f13adabb264ed19bc6cbdf1f873538 in this term.
I20260709 13:06:33.321885 13703 leader_election.cc:290] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.323335 13725 leader_election.cc:290] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.324695 13725 raft_consensus.cc:493] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.324973 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.325863 13495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff234e22c0845c4a8ad42671e104934" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.310859 13692 raft_consensus.cc:515] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.326417 13497 raft_consensus.cc:2468] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:33.330850 13688 tablet_bootstrap.cc:492] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.331211 13692 leader_election.cc:290] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:33.319964 13693 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
W20260709 13:06:33.334000 13535 leader_election.cc:343] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.337400 13535 leader_election.cc:304] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.338778 13693 raft_consensus.cc:2804] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.334179 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.340227 13496 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.335090 13688 ts_tablet_manager.cc:1403] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.212s	user 0.075s	sys 0.111s
I20260709 13:06:33.339504 13693 raft_consensus.cc:697] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:33.343714 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b743418e294d1a9d3eaa20ea044b75" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.346189 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff234e22c0845c4a8ad42671e104934" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.346814 13646 raft_consensus.cc:2468] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:33.348421 13723 raft_consensus.cc:515] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.348716 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.349426 13571 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.351409 13723 leader_election.cc:290] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:33.353008 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.353657 13497 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.356451 13538 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:33.360477 13703 raft_consensus.cc:2749] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:33.361464 13497 raft_consensus.cc:2468] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.362620 13609 leader_election.cc:304] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:33.363443 13723 raft_consensus.cc:2804] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.354301 13688 raft_consensus.cc:359] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.364276 13723 raft_consensus.cc:697] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
W20260709 13:06:33.361941 13538 leader_election.cc:343] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.364434 13688 raft_consensus.cc:385] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.340847 13725 raft_consensus.cc:515] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.368736 13687 ts_tablet_manager.cc:1434] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.021s
I20260709 13:06:33.366338 13571 raft_consensus.cc:2468] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.370034 13693 consensus_queue.cc:237] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.371011 13686 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:33.371546 13686 ts_tablet_manager.cc:1403] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.009s
I20260709 13:06:33.371702 13703 raft_consensus.cc:493] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.372213 13703 raft_consensus.cc:515] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.367254 13688 raft_consensus.cc:740] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.374914 13703 leader_election.cc:290] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.376171 13495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.376272 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.376080 13688 consensus_queue.cc:260] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
W20260709 13:06:33.378006 13535 leader_election.cc:343] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.378274 13725 leader_election.cc:290] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
W20260709 13:06:33.378921 13535 leader_election.cc:343] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.379648 13688 ts_tablet_manager.cc:1434] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.003s
I20260709 13:06:33.380375 13496 raft_consensus.cc:2468] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.380764 13688 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.381585 13609 leader_election.cc:304] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:33.382611 13692 raft_consensus.cc:2804] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.383107 13692 raft_consensus.cc:697] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:33.384064 13692 consensus_queue.cc:237] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.387286 13688 tablet_bootstrap.cc:654] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.386813 13686 raft_consensus.cc:359] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.389070 13686 raft_consensus.cc:385] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.389501 13686 raft_consensus.cc:740] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.385308 13393 catalog_manager.cc:5697] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.390349 13723 consensus_queue.cc:237] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.390431 13686 consensus_queue.cc:260] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.394428 13686 ts_tablet_manager.cc:1434] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.001s
I20260709 13:06:33.394613 13688 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.395238 13688 ts_tablet_manager.cc:1403] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:06:33.396054 13538 leader_election.cc:304] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.396970 13725 raft_consensus.cc:2804] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.397315 13725 raft_consensus.cc:493] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.397607 13725 raft_consensus.cc:3060] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.395558 13686 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:33.398809 13688 raft_consensus.cc:359] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.399622 13688 raft_consensus.cc:385] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.399948 13688 raft_consensus.cc:740] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.401418 13688 consensus_queue.cc:260] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.404160 13688 ts_tablet_manager.cc:1434] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:06:33.405258 13688 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.406147 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.407753 13725 raft_consensus.cc:515] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.409446 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.410305 13646 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:33.411324 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff234e22c0845c4a8ad42671e104934" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.412140 13496 raft_consensus.cc:3060] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.412868 13686 tablet_bootstrap.cc:654] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.412726 13696 raft_consensus.cc:493] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.413725 13696 raft_consensus.cc:515] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.415779 13688 tablet_bootstrap.cc:654] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.417737 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f8f68254cc4ebea8476631126ffc2a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.418565 13572 raft_consensus.cc:2468] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:33.420529 13463 leader_election.cc:304] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.420634 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.421569 13571 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.421881 13694 raft_consensus.cc:2804] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.422394 13694 raft_consensus.cc:493] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.422766 13694 raft_consensus.cc:3060] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.425634 13496 raft_consensus.cc:2468] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:33.427276 13725 leader_election.cc:290] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.427541 13535 leader_election.cc:304] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.429236 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f8f68254cc4ebea8476631126ffc2a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
W20260709 13:06:33.430871 13463 leader_election.cc:343] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.426100 13696 leader_election.cc:290] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.437078 13694 raft_consensus.cc:515] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.437704 13571 raft_consensus.cc:2468] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.439363 13694 leader_election.cc:290] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.439956 13725 raft_consensus.cc:2804] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.447009 13725 raft_consensus.cc:697] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:33.448052 13686 tablet_bootstrap.cc:492] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:33.448724 13686 ts_tablet_manager.cc:1403] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.053s	user 0.007s	sys 0.007s
I20260709 13:06:33.448272 13725 consensus_queue.cc:237] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.450335 13688 tablet_bootstrap.cc:492] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.450968 13688 ts_tablet_manager.cc:1403] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.006s
I20260709 13:06:33.440310 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f8f68254cc4ebea8476631126ffc2a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.440739 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f8f68254cc4ebea8476631126ffc2a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.441601 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff234e22c0845c4a8ad42671e104934" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.452203 13686 raft_consensus.cc:359] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.453052 13645 raft_consensus.cc:3060] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.453158 13394 catalog_manager.cc:5697] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:33.454090 13688 raft_consensus.cc:359] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.454929 13688 raft_consensus.cc:385] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.455253 13688 raft_consensus.cc:740] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.456108 13688 consensus_queue.cc:260] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.458458 13688 ts_tablet_manager.cc:1434] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:06:33.453024 13686 raft_consensus.cc:385] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.459688 13688 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:33.460677 13696 raft_consensus.cc:493] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.461278 13696 raft_consensus.cc:515] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.464569 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.459084 13686 raft_consensus.cc:740] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.465296 13572 raft_consensus.cc:2468] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:33.466142 13688 tablet_bootstrap.cc:654] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.466569 13463 leader_election.cc:304] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.468815 13724 raft_consensus.cc:2804] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.469197 13724 raft_consensus.cc:493] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.469638 13724 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.469224 13686 consensus_queue.cc:260] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.472113 13646 raft_consensus.cc:3060] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.473063 13686 ts_tablet_manager.cc:1434] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:06:33.474474 13686 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:33.476085 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
W20260709 13:06:33.496443 13463 leader_election.cc:343] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.474490 13696 leader_election.cc:290] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.498641 13696 raft_consensus.cc:493] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.500571 13686 tablet_bootstrap.cc:654] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.513752 13571 raft_consensus.cc:3060] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.499285 13696 raft_consensus.cc:515] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.519282 13686 tablet_bootstrap.cc:492] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
W20260709 13:06:33.445892 13538 leader_election.cc:343] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:33.519892 13686 ts_tablet_manager.cc:1403] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.046s	user 0.023s	sys 0.005s
I20260709 13:06:33.520157 13538 leader_election.cc:304] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:33.521035 13693 raft_consensus.cc:2749] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:33.522234 13538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40043, user_credentials={real_user=slave}} blocked reactor thread for 76423.4us
I20260709 13:06:33.522722 13686 raft_consensus.cc:359] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.523034 13538 leader_election.cc:304] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.523665 13686 raft_consensus.cc:385] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.524029 13693 raft_consensus.cc:2804] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.524025 13686 raft_consensus.cc:740] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.524717 13693 raft_consensus.cc:493] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.525074 13693 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.525179 13686 consensus_queue.cc:260] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.527397 13686 ts_tablet_manager.cc:1434] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:33.527642 13724 raft_consensus.cc:515] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.528330 13686 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:33.530227 13724 leader_election.cc:290] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.530956 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.531579 13572 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.533100 13693 raft_consensus.cc:515] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.534288 13686 tablet_bootstrap.cc:654] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.507241 13645 raft_consensus.cc:2468] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:33.536746 13693 leader_election.cc:290] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.537771 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.538719 13496 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.540256 13686 tablet_bootstrap.cc:492] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:33.540913 13686 ts_tablet_manager.cc:1403] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:06:33.544471 13571 raft_consensus.cc:2468] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:33.543792 13686 raft_consensus.cc:359] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.545209 13686 raft_consensus.cc:385] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.545495 13686 raft_consensus.cc:740] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.546124 13463 leader_election.cc:304] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.546439 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.546221 13686 consensus_queue.cc:260] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.547322 13743 raft_consensus.cc:2804] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.547991 13743 raft_consensus.cc:697] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:33.548302 13572 raft_consensus.cc:2468] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:33.549312 13686 ts_tablet_manager.cc:1434] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:33.549628 13463 leader_election.cc:304] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.550359 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.550942 13645 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.551051 13644 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.552204 13696 leader_election.cc:290] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.552945 13496 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:33.553737 13646 raft_consensus.cc:2468] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:33.554051 13535 leader_election.cc:304] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.555677 13725 raft_consensus.cc:2804] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.557008 13395 catalog_manager.cc:5697] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.561424 13645 raft_consensus.cc:2468] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:33.562701 13725 raft_consensus.cc:697] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:33.562311 13393 catalog_manager.cc:5697] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:33.562024 13644 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:33.563671 13725 consensus_queue.cc:237] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.565900 13696 raft_consensus.cc:2804] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.566072 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:33.566648 13696 raft_consensus.cc:697] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:33.566972 13645 raft_consensus.cc:2393] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30f13adabb264ed19bc6cbdf1f873538 in current term 1: Already voted for candidate f1cfe6266c2c461598761f06c3f46e26 in this term.
I20260709 13:06:33.567597 13696 consensus_queue.cc:237] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.548964 13743 consensus_queue.cc:237] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.579231 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.580093 13688 tablet_bootstrap.cc:492] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:33.580834 13688 ts_tablet_manager.cc:1403] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.121s	user 0.033s	sys 0.035s
I20260709 13:06:33.584486 13688 raft_consensus.cc:359] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.584710 13463 leader_election.cc:304] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:33.585222 13688 raft_consensus.cc:385] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.585646 13694 raft_consensus.cc:2749] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:33.585556 13688 raft_consensus.cc:740] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.586779 13688 consensus_queue.cc:260] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.589222 13688 ts_tablet_manager.cc:1434] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:06:33.590222 13395 catalog_manager.cc:5697] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.594117 13393 catalog_manager.cc:5697] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.607496 13393 catalog_manager.cc:5697] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.615476 13723 raft_consensus.cc:493] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.616144 13723 raft_consensus.cc:515] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.617765 13723 leader_election.cc:290] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729)
I20260709 13:06:33.618595 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:33.618767 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:33.619359 13571 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:33.619365 13496 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:33.620676 13609 leader_election.cc:304] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:33.621407 13723 raft_consensus.cc:2804] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:33.621755 13723 raft_consensus.cc:493] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.622026 13723 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.627270 13723 raft_consensus.cc:515] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.628964 13723 leader_election.cc:290] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729)
I20260709 13:06:33.629928 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.630070 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.630544 13571 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.630640 13496 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.636024 13571 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.636270 13496 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:33.637259 13612 leader_election.cc:304] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.638089 13723 raft_consensus.cc:2804] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.638578 13723 raft_consensus.cc:697] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:33.639362 13723 consensus_queue.cc:237] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:33.646785 13393 catalog_manager.cc:5697] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.704986 13743 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.717219 13743 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.736744 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:57270
I20260709 13:06:33.737210 13645 raft_consensus.cc:606] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.752367 13645 raft_consensus.cc:3060] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.754280 13571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:51236
I20260709 13:06:33.754966 13571 raft_consensus.cc:606] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.760274 13725 consensus_queue.cc:1048] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.760845 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:57270
I20260709 13:06:33.761446 13645 raft_consensus.cc:606] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:33.779292 13725 consensus_queue.cc:1048] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.786152 13497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
mode: GRACEFUL
new_leader_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:06:33.786769 13497 raft_consensus.cc:606] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Received request to transfer leadership to TS f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:33.798197 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
mode: GRACEFUL
new_leader_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:57270
I20260709 13:06:33.798688 13645 raft_consensus.cc:606] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Received request to transfer leadership to TS f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:33.803201 13497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
mode: GRACEFUL
new_leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:06:33.803709 13497 raft_consensus.cc:606] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Received request to transfer leadership to TS b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:33.827066 13723 consensus_queue.cc:1048] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.831578 13723 consensus_queue.cc:1048] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.835085 13692 raft_consensus.cc:993] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:33.835644 13692 raft_consensus.cc:1081] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:33.842453 13497 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:06:33.843091 13497 raft_consensus.cc:493] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:33.843529 13497 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.843954 13727 consensus_queue.cc:1048] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:33.846346 13723 consensus_queue.cc:1048] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.847275 13693 consensus_queue.cc:1048] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.861723 13727 raft_consensus.cc:993] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: : Instructing follower f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:33.862175 13727 raft_consensus.cc:1081] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Signalling peer f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:33.863914 13572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:51228
I20260709 13:06:33.864540 13572 raft_consensus.cc:493] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:33.864935 13572 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.865792 13497 raft_consensus.cc:515] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.869102 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:33.869699 13571 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.870002 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa7b5859d487cbfad3f0090a3ad78" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.870764 13645 raft_consensus.cc:3055] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:33.871088 13645 raft_consensus.cc:740] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:33.871969 13645 consensus_queue.cc:260] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.873327 13645 raft_consensus.cc:3060] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.877046 13572 raft_consensus.cc:515] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.878842 13571 raft_consensus.cc:2468] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:33.879410 13572 leader_election.cc:290] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.881139 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:33.881705 13646 raft_consensus.cc:3055] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:33.882000 13646 raft_consensus.cc:740] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:33.882201 13693 consensus_queue.cc:1048] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:33.882853 13646 consensus_queue.cc:260] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.884511 13646 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.886675 13645 raft_consensus.cc:2468] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:33.888209 13463 leader_election.cc:304] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:33.889078 13743 raft_consensus.cc:2804] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:33.889552 13743 raft_consensus.cc:697] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:33.890411 13743 consensus_queue.cc:237] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.892818 13497 leader_election.cc:290] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:33.894892 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "145fa6a445994668a08b1d954a9c227d" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:33.895574 13496 raft_consensus.cc:3060] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:33.900183 13394 catalog_manager.cc:5697] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.903862 13496 raft_consensus.cc:2468] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:33.904771 13646 raft_consensus.cc:2468] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:33.905264 13535 leader_election.cc:304] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:33.906271 13693 raft_consensus.cc:2804] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:33.906751 13693 raft_consensus.cc:697] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:33.907513 13693 consensus_queue.cc:237] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:33.915040 13394 catalog_manager.cc:5697] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 1 to 2, leader changed from b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195) to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.979566 13743 consensus_queue.cc:1048] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.990000 13693 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.990772 13694 consensus_queue.cc:1048] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.002548 13703 raft_consensus.cc:993] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:34.002990 13703 raft_consensus.cc:1081] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:34.005131 13494 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44692
I20260709 13:06:34.005686 13494 raft_consensus.cc:493] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:34.006071 13494 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.007458 13725 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.013671 13494 raft_consensus.cc:515] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.014816 13496 raft_consensus.cc:1240] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Rejecting Update request from peer f1cfe6266c2c461598761f06c3f46e26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:34.016733 13494 leader_election.cc:290] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:34.016629 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:34.017611 13572 raft_consensus.cc:3055] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:34.018020 13572 raft_consensus.cc:740] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:34.017853 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:34.018852 13646 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.018842 13725 consensus_queue.cc:1059] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:34.020035 13572 consensus_queue.cc:260] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.021056 13572 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.025382 13646 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:34.026465 13463 leader_election.cc:304] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:34.026624 13572 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
W20260709 13:06:34.027163 13703 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:06:34.027320 13724 raft_consensus.cc:2804] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:34.027959 13724 raft_consensus.cc:697] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:34.028811 13724 consensus_queue.cc:237] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.034865 13395 catalog_manager.cc:5697] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.086551 13724 consensus_queue.cc:1048] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.100127 13694 consensus_queue.cc:1048] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.100332 13692 consensus_queue.cc:1048] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.115520 13694 raft_consensus.cc:993] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: : Instructing follower b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:34.116086 13694 raft_consensus.cc:1081] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Signalling peer b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:34.119293 13692 consensus_queue.cc:1048] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:34.120020 13646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:06:34.120711 13646 raft_consensus.cc:493] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:34.121101 13646 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.128988 13646 raft_consensus.cc:515] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.129884 13692 raft_consensus.cc:993] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:34.130443 13692 raft_consensus.cc:1081] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:34.132018 13646 leader_election.cc:290] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 2 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:34.132640 13496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:06:34.133325 13496 raft_consensus.cc:493] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:34.133356 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:34.133852 13496 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.134117 13571 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.135731 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3b361c05214d7686b6a29ae887d92a" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:34.136441 13494 raft_consensus.cc:3055] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:34.136778 13494 raft_consensus.cc:740] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:34.137578 13494 consensus_queue.cc:260] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.138928 13494 raft_consensus.cc:3060] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.141292 13571 raft_consensus.cc:2468] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 2.
I20260709 13:06:34.141338 13496 raft_consensus.cc:515] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:34.142906 13612 leader_election.cc:304] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:34.143828 13692 raft_consensus.cc:2804] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:34.144326 13692 raft_consensus.cc:697] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:34.145166 13692 consensus_queue.cc:237] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.145968 13494 raft_consensus.cc:2468] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 2.
I20260709 13:06:34.147714 13496 leader_election.cc:290] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:34.147933 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:34.148661 13646 raft_consensus.cc:3055] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:34.148983 13646 raft_consensus.cc:740] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:34.149104 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f8978ace07e4cb0a4dd24c207d6fac9" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:34.149895 13571 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.149842 13646 consensus_queue.cc:260] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:34.151259 13646 raft_consensus.cc:3060] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.154392 13394 catalog_manager.cc:5697] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 1 to 2, leader changed from 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193) to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.156725 13646 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:34.157796 13571 raft_consensus.cc:2468] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:34.158216 13463 leader_election.cc:304] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:34.159065 13694 raft_consensus.cc:2804] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:34.159524 13694 raft_consensus.cc:697] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:34.160460 13694 consensus_queue.cc:237] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } }
I20260709 13:06:34.168458 13394 catalog_manager.cc:5697] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.210023 13696 raft_consensus.cc:993] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: : Instructing follower f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:34.210541 13694 raft_consensus.cc:1081] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Signalling peer f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:34.212107 13571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:06:34.212692 13571 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:34.213075 13571 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.218359 13571 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.220583 13571 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:34.221042 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:34.221257 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:34.221778 13496 raft_consensus.cc:3055] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:34.221915 13646 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.222080 13496 raft_consensus.cc:740] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:34.222863 13496 consensus_queue.cc:260] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.223944 13496 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.228159 13646 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:34.229607 13538 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:34.229869 13496 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:34.230394 13703 raft_consensus.cc:2804] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:34.230856 13703 raft_consensus.cc:697] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:34.231709 13703 consensus_queue.cc:237] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:34.239147 13394 catalog_manager.cc:5697] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 1 to 2, leader changed from 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193) to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.338045 13496 raft_consensus.cc:1275] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.339390 13703 consensus_queue.cc:1048] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.347237 13646 raft_consensus.cc:1275] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.348855 13703 consensus_queue.cc:1048] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.439328 13571 raft_consensus.cc:1275] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.440562 13694 consensus_queue.cc:1048] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.447873 13646 raft_consensus.cc:1275] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.449331 13724 consensus_queue.cc:1048] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.517812 13644 raft_consensus.cc:1275] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.519220 13724 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.527873 13571 raft_consensus.cc:1275] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.529173 13724 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.555182 13644 raft_consensus.cc:1275] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.556571 13724 consensus_queue.cc:1048] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.565842 13571 raft_consensus.cc:1275] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.567533 13724 consensus_queue.cc:1048] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.570535 13496 raft_consensus.cc:1275] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.572235 13692 consensus_queue.cc:1048] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.583587 13571 raft_consensus.cc:1275] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.585073 13727 consensus_queue.cc:1048] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.649947 13496 raft_consensus.cc:1275] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.651157 13703 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.658291 13644 raft_consensus.cc:1275] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:34.659921 13725 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:36.836220 13393 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39100:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:36.839354 13393 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:36.893795 13487 tablet_service.cc:1511] Processing CreateTablet for tablet ff714cc8473746b09143748633e809d5 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:36.894752 13554 tablet_service.cc:1511] Processing CreateTablet for tablet ff714cc8473746b09143748633e809d5 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:36.895987 13487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff714cc8473746b09143748633e809d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.896544 13554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff714cc8473746b09143748633e809d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.897687 13628 tablet_service.cc:1511] Processing CreateTablet for tablet ff714cc8473746b09143748633e809d5 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:36.899417 13628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff714cc8473746b09143748633e809d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.899828 13561 tablet_service.cc:1511] Processing CreateTablet for tablet 1c77180ce4e54dc9af03b942d05de500 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:36.901232 13561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c77180ce4e54dc9af03b942d05de500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.904754 13633 tablet_service.cc:1511] Processing CreateTablet for tablet 1c77180ce4e54dc9af03b942d05de500 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:36.905138 13485 tablet_service.cc:1511] Processing CreateTablet for tablet 1c77180ce4e54dc9af03b942d05de500 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:36.906323 13633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c77180ce4e54dc9af03b942d05de500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.906661 13485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c77180ce4e54dc9af03b942d05de500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.910832 13630 tablet_service.cc:1511] Processing CreateTablet for tablet 50afd3199d014b0cad8eb33518b110b1 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:36.912223 13630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50afd3199d014b0cad8eb33518b110b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.916518 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 5f10046984174efd8efcf2b243a6e697 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:36.918041 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f10046984174efd8efcf2b243a6e697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.918191 13558 tablet_service.cc:1511] Processing CreateTablet for tablet 50afd3199d014b0cad8eb33518b110b1 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:36.919585 13558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50afd3199d014b0cad8eb33518b110b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.922169 13479 tablet_service.cc:1511] Processing CreateTablet for tablet 50afd3199d014b0cad8eb33518b110b1 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:36.922672 13636 tablet_service.cc:1511] Processing CreateTablet for tablet a9d719eb4edf4a47a60c2b41e3451928 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:36.923667 13479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50afd3199d014b0cad8eb33518b110b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.924168 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d719eb4edf4a47a60c2b41e3451928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.924998 13557 tablet_service.cc:1511] Processing CreateTablet for tablet 5f10046984174efd8efcf2b243a6e697 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:36.926409 13557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f10046984174efd8efcf2b243a6e697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.929319 13484 tablet_service.cc:1511] Processing CreateTablet for tablet 5f10046984174efd8efcf2b243a6e697 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:36.931031 13484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f10046984174efd8efcf2b243a6e697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.956791 13631 tablet_service.cc:1511] Processing CreateTablet for tablet fba4e19f161645c5aac66aaf74c76892 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:36.958369 13631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba4e19f161645c5aac66aaf74c76892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.964779 13632 tablet_service.cc:1511] Processing CreateTablet for tablet b47ff1aa0aea4ec4b8ef19898f505207 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:36.966284 13632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47ff1aa0aea4ec4b8ef19898f505207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.967267 13634 tablet_service.cc:1511] Processing CreateTablet for tablet d956e04308804f11a7f39cc63e7f708b (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:36.968658 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d956e04308804f11a7f39cc63e7f708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.972383 13560 tablet_service.cc:1511] Processing CreateTablet for tablet a9d719eb4edf4a47a60c2b41e3451928 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:36.973843 13560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d719eb4edf4a47a60c2b41e3451928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.974207 13629 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb727ef8748471281e26b5bcadf2bbe (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:36.975539 13629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb727ef8748471281e26b5bcadf2bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.977514 13562 tablet_service.cc:1511] Processing CreateTablet for tablet fba4e19f161645c5aac66aaf74c76892 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:36.978974 13562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba4e19f161645c5aac66aaf74c76892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:36.999280 13555 tablet_service.cc:1511] Processing CreateTablet for tablet b47ff1aa0aea4ec4b8ef19898f505207 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:37.001122 13555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47ff1aa0aea4ec4b8ef19898f505207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.004405 13559 tablet_service.cc:1511] Processing CreateTablet for tablet d956e04308804f11a7f39cc63e7f708b (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:37.005813 13559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d956e04308804f11a7f39cc63e7f708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.021308 13486 tablet_service.cc:1511] Processing CreateTablet for tablet a9d719eb4edf4a47a60c2b41e3451928 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:37.022833 13486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d719eb4edf4a47a60c2b41e3451928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.025678 13480 tablet_service.cc:1511] Processing CreateTablet for tablet fba4e19f161645c5aac66aaf74c76892 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:37.029039 13482 tablet_service.cc:1511] Processing CreateTablet for tablet b47ff1aa0aea4ec4b8ef19898f505207 (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:37.029412 13556 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb727ef8748471281e26b5bcadf2bbe (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:37.030609 13556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb727ef8748471281e26b5bcadf2bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.032155 13481 tablet_service.cc:1511] Processing CreateTablet for tablet d956e04308804f11a7f39cc63e7f708b (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:37.036191 13482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47ff1aa0aea4ec4b8ef19898f505207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.037997 13480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba4e19f161645c5aac66aaf74c76892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.039702 13481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d956e04308804f11a7f39cc63e7f708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.073881 13483 tablet_service.cc:1511] Processing CreateTablet for tablet 4eb727ef8748471281e26b5bcadf2bbe (DEFAULT_TABLE table=soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:37.075467 13483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb727ef8748471281e26b5bcadf2bbe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:37.112890 13443 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:37.124414 13382 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.195:40043, user_credentials={real_user=slave}} blocked reactor thread for 50323.5us
I20260709 13:06:37.157600 13789 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.184494 13787 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.190745 13788 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.193120 13787 tablet_bootstrap.cc:654] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.208700 13787 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.209240 13787 ts_tablet_manager.cc:1403] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.004s
I20260709 13:06:37.211491 13788 tablet_bootstrap.cc:654] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.212523 13789 tablet_bootstrap.cc:654] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.212325 13787 raft_consensus.cc:359] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.213352 13787 raft_consensus.cc:385] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.213711 13787 raft_consensus.cc:740] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.214551 13787 consensus_queue.cc:260] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.217784 13787 ts_tablet_manager.cc:1434] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:06:37.219045 13787 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.220758 13789 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.221344 13789 ts_tablet_manager.cc:1403] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.004s
I20260709 13:06:37.224514 13789 raft_consensus.cc:359] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.225344 13789 raft_consensus.cc:385] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.225694 13789 raft_consensus.cc:740] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.226188 13788 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.226742 13788 ts_tablet_manager.cc:1403] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.000s
I20260709 13:06:37.226922 13787 tablet_bootstrap.cc:654] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.226560 13789 consensus_queue.cc:260] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.230069 13788 raft_consensus.cc:359] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.231043 13788 raft_consensus.cc:385] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.231447 13788 raft_consensus.cc:740] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.232131 13789 ts_tablet_manager.cc:1434] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:06:37.232390 13788 consensus_queue.cc:260] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.233582 13789 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.235946 13788 ts_tablet_manager.cc:1434] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:06:37.237483 13788 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.248075 13788 tablet_bootstrap.cc:654] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.257287 13789 tablet_bootstrap.cc:654] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.264380 13787 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.264936 13787 ts_tablet_manager.cc:1403] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.025s
I20260709 13:06:37.267900 13787 raft_consensus.cc:359] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.268635 13787 raft_consensus.cc:385] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.268932 13787 raft_consensus.cc:740] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.269662 13787 consensus_queue.cc:260] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.272089 13787 ts_tablet_manager.cc:1434] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:37.272470 13788 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.273062 13788 ts_tablet_manager.cc:1403] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.009s
I20260709 13:06:37.273591 13787 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.276165 13788 raft_consensus.cc:359] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.276965 13788 raft_consensus.cc:385] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.277329 13788 raft_consensus.cc:740] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.278160 13788 consensus_queue.cc:260] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.280962 13788 ts_tablet_manager.cc:1434] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 13:06:37.281961 13788 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.282414 13787 tablet_bootstrap.cc:654] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.282470 13789 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.283205 13789 ts_tablet_manager.cc:1403] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.013s
I20260709 13:06:37.286242 13789 raft_consensus.cc:359] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.286971 13789 raft_consensus.cc:385] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.287261 13789 raft_consensus.cc:740] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.287984 13789 consensus_queue.cc:260] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.290063 13788 tablet_bootstrap.cc:654] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.290586 13789 ts_tablet_manager.cc:1434] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:06:37.291595 13789 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.297171 13787 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.297593 13788 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.297691 13787 ts_tablet_manager.cc:1403] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.012s
I20260709 13:06:37.298118 13788 ts_tablet_manager.cc:1403] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.001s
I20260709 13:06:37.298269 13789 tablet_bootstrap.cc:654] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.301211 13787 raft_consensus.cc:359] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.302086 13787 raft_consensus.cc:385] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.302459 13787 raft_consensus.cc:740] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.303382 13787 consensus_queue.cc:260] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.303946 13788 raft_consensus.cc:359] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
W20260709 13:06:37.305092 13598 tablet.cc:2406] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:37.306761 13787 ts_tablet_manager.cc:1434] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:37.308193 13787 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.304689 13788 raft_consensus.cc:385] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.314855 13788 raft_consensus.cc:740] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.315670 13788 consensus_queue.cc:260] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.316974 13787 tablet_bootstrap.cc:654] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.319589 13788 ts_tablet_manager.cc:1434] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:06:37.320924 13788 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.329210 13788 tablet_bootstrap.cc:654] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.356242 13789 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.356863 13788 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.356864 13789 ts_tablet_manager.cc:1403] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.042s
I20260709 13:06:37.357357 13787 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.357729 13788 ts_tablet_manager.cc:1403] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.007s
I20260709 13:06:37.357997 13787 ts_tablet_manager.cc:1403] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.027s
I20260709 13:06:37.360630 13788 raft_consensus.cc:359] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.360865 13789 raft_consensus.cc:359] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.361058 13787 raft_consensus.cc:359] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.361541 13789 raft_consensus.cc:385] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.361342 13776 raft_consensus.cc:493] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.361913 13787 raft_consensus.cc:385] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.361490 13788 raft_consensus.cc:385] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.362175 13789 raft_consensus.cc:740] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.362321 13787 raft_consensus.cc:740] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.362397 13788 raft_consensus.cc:740] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.362345 13776 raft_consensus.cc:515] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.363236 13789 consensus_queue.cc:260] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.363554 13787 consensus_queue.cc:260] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.363971 13788 consensus_queue.cc:260] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.366648 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.368270 13789 ts_tablet_manager.cc:1434] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:06:37.369316 13789 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.372550 13788 ts_tablet_manager.cc:1434] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:06:37.373786 13788 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
W20260709 13:06:37.374897 13609 leader_election.cc:343] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.378715 13789 tablet_bootstrap.cc:654] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.380421 13788 tablet_bootstrap.cc:654] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.384691 13776 leader_election.cc:290] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.388453 13803 raft_consensus.cc:493] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.389559 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
W20260709 13:06:37.396162 13612 leader_election.cc:343] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.396068 13803 raft_consensus.cc:515] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.396735 13612 leader_election.cc:304] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:37.399041 13776 raft_consensus.cc:2749] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:37.402515 13789 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.403203 13789 ts_tablet_manager.cc:1403] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.005s
I20260709 13:06:37.402982 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.403970 13496 raft_consensus.cc:2468] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:37.404954 13788 tablet_bootstrap.cc:492] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.405143 13609 leader_election.cc:304] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.405880 13788 ts_tablet_manager.cc:1403] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.001s
I20260709 13:06:37.406240 13776 raft_consensus.cc:2804] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.406664 13776 raft_consensus.cc:493] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.406985 13776 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.408262 13789 raft_consensus.cc:359] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.409116 13789 raft_consensus.cc:385] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.409070 13788 raft_consensus.cc:359] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.410030 13788 raft_consensus.cc:385] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.410346 13788 raft_consensus.cc:740] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.411195 13788 consensus_queue.cc:260] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.409653 13789 raft_consensus.cc:740] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.413007 13789 consensus_queue.cc:260] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.413576 13788 ts_tablet_manager.cc:1434] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:06:37.420547 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.416205 13776 raft_consensus.cc:515] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
W20260709 13:06:37.422062 13612 leader_election.cc:343] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.423408 13776 leader_election.cc:290] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.424072 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.424674 13496 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.431667 13776 raft_consensus.cc:493] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.432377 13776 raft_consensus.cc:515] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.435007 13776 leader_election.cc:290] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.435876 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.436805 13494 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:37.439302 13572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.439610 13788 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.440331 13572 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:37.442395 13609 leader_election.cc:304] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.443559 13776 raft_consensus.cc:2804] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.444006 13776 raft_consensus.cc:493] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.444413 13776 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.444563 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
W20260709 13:06:37.446238 13612 leader_election.cc:343] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.447127 13696 raft_consensus.cc:493] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.447896 13693 raft_consensus.cc:493] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.447639 13696 raft_consensus.cc:515] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.448381 13693 raft_consensus.cc:515] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.449968 13696 leader_election.cc:290] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.450513 13693 leader_election.cc:290] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.452164 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.452977 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47ff1aa0aea4ec4b8ef19898f505207" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.453768 13788 tablet_bootstrap.cc:654] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.452099 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47ff1aa0aea4ec4b8ef19898f505207" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
W20260709 13:06:37.454233 13463 leader_election.cc:343] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:37.454749 13538 leader_election.cc:343] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.454726 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.454684 13494 raft_consensus.cc:2468] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
W20260709 13:06:37.456452 13463 leader_election.cc:343] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.456390 13776 raft_consensus.cc:515] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.456751 13535 leader_election.cc:304] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.457108 13463 leader_election.cc:304] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:37.457669 13496 raft_consensus.cc:2468] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:37.457986 13693 raft_consensus.cc:2804] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.458308 13696 raft_consensus.cc:2749] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:37.458459 13693 raft_consensus.cc:493] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.422866 13803 leader_election.cc:290] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.458962 13693 raft_consensus.cc:3060] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.461690 13609 leader_election.cc:304] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:37.462404 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.463020 13496 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.463028 13787 ts_tablet_manager.cc:1434] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.105s	user 0.010s	sys 0.045s
I20260709 13:06:37.464273 13787 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.468181 13802 raft_consensus.cc:2804] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.468896 13693 raft_consensus.cc:515] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.471349 13496 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:37.471872 13802 raft_consensus.cc:697] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:37.472067 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47ff1aa0aea4ec4b8ef19898f505207" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.472760 13494 raft_consensus.cc:3060] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.472856 13802 consensus_queue.cc:237] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.478024 13789 ts_tablet_manager.cc:1434] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.073s	user 0.020s	sys 0.016s
I20260709 13:06:37.478130 13787 tablet_bootstrap.cc:654] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.480449 13789 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.484962 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47ff1aa0aea4ec4b8ef19898f505207" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.487187 13609 leader_election.cc:304] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.484007 13693 leader_election.cc:290] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.486001 13776 leader_election.cc:290] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.488952 13776 raft_consensus.cc:2804] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:37.489763 13538 leader_election.cc:343] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.486624 13806 raft_consensus.cc:493] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.491191 13806 raft_consensus.cc:515] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.491667 13776 raft_consensus.cc:697] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:37.492508 13494 raft_consensus.cc:2468] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:37.492710 13776 consensus_queue.cc:237] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.493162 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:37.493800 13571 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.494509 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.492770 13395 catalog_manager.cc:5697] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.497063 13535 leader_election.cc:304] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.498215 13693 raft_consensus.cc:2804] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.498716 13693 raft_consensus.cc:697] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:37.499624 13693 consensus_queue.cc:237] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.500066 13497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
W20260709 13:06:37.501427 13538 leader_election.cc:343] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.501937 13806 leader_election.cc:290] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
W20260709 13:06:37.503908 13535 leader_election.cc:343] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.505990 13535 leader_election.cc:304] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.506939 13808 raft_consensus.cc:2749] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:37.510299 13789 tablet_bootstrap.cc:654] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.513025 13788 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.513850 13788 ts_tablet_manager.cc:1403] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.007s
I20260709 13:06:37.516515 13571 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:37.521471 13788 raft_consensus.cc:359] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.522541 13788 raft_consensus.cc:385] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.522831 13788 raft_consensus.cc:740] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.523862 13788 consensus_queue.cc:260] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.530112 13787 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.531247 13787 ts_tablet_manager.cc:1403] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.011s
I20260709 13:06:37.530267 13394 catalog_manager.cc:5697] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.533597 13788 ts_tablet_manager.cc:1434] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:06:37.534792 13788 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.534979 13789 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.535566 13789 ts_tablet_manager.cc:1403] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.004s
I20260709 13:06:37.535149 13787 raft_consensus.cc:359] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.540892 13789 raft_consensus.cc:359] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.541669 13789 raft_consensus.cc:385] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.541996 13789 raft_consensus.cc:740] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.541726 13787 raft_consensus.cc:385] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.542899 13787 raft_consensus.cc:740] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.542896 13789 consensus_queue.cc:260] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.544708 13787 consensus_queue.cc:260] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.546082 13789 ts_tablet_manager.cc:1434] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:06:37.547353 13789 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.547820 13788 tablet_bootstrap.cc:654] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.548650 13787 ts_tablet_manager.cc:1434] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:06:37.549822 13787 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.555635 13789 tablet_bootstrap.cc:654] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.558292 13787 tablet_bootstrap.cc:654] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.562901 13788 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.563623 13788 ts_tablet_manager.cc:1403] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.007s
I20260709 13:06:37.566474 13776 raft_consensus.cc:493] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.570722 13787 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.570652 13776 raft_consensus.cc:515] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.571316 13787 ts_tablet_manager.cc:1403] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 13:06:37.573127 13776 leader_election.cc:290] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.574420 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.574661 13787 raft_consensus.cc:359] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.575523 13787 raft_consensus.cc:385] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.575512 13496 raft_consensus.cc:2468] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 0.
I20260709 13:06:37.576100 13787 raft_consensus.cc:740] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.576952 13788 raft_consensus.cc:359] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.576968 13787 consensus_queue.cc:260] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.579041 13609 leader_election.cc:304] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.580853 13776 raft_consensus.cc:2804] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.577847 13788 raft_consensus.cc:385] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.581357 13776 raft_consensus.cc:493] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.581808 13788 raft_consensus.cc:740] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.582012 13776 raft_consensus.cc:3060] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.582633 13788 consensus_queue.cc:260] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.577739 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.588229 13696 raft_consensus.cc:493] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.588773 13696 raft_consensus.cc:515] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
W20260709 13:06:37.589649 13612 leader_election.cc:343] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.590418 13787 ts_tablet_manager.cc:1434] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:06:37.590878 13696 leader_election.cc:290] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.591548 13787 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.592583 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.593171 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.593420 13571 raft_consensus.cc:2468] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
W20260709 13:06:37.594719 13463 leader_election.cc:343] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.595530 13463 leader_election.cc:304] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.596972 13696 raft_consensus.cc:2804] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.597386 13696 raft_consensus.cc:493] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.597759 13696 raft_consensus.cc:3060] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.599469 13787 tablet_bootstrap.cc:654] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.587927 13788 ts_tablet_manager.cc:1434] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:37.605047 13788 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.606233 13776 raft_consensus.cc:515] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.613752 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.614562 13496 raft_consensus.cc:3060] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.621084 13696 raft_consensus.cc:515] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.626645 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:37.627328 13571 raft_consensus.cc:3060] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.628417 13789 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.629061 13789 ts_tablet_manager.cc:1403] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.082s	user 0.039s	sys 0.035s
I20260709 13:06:37.632397 13789 raft_consensus.cc:359] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.633210 13789 raft_consensus.cc:385] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.633539 13789 raft_consensus.cc:740] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.634409 13789 consensus_queue.cc:260] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.635401 13799 raft_consensus.cc:493] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.635269 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.636227 13571 raft_consensus.cc:2468] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:37.636031 13799 raft_consensus.cc:515] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.637005 13789 ts_tablet_manager.cc:1434] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
W20260709 13:06:37.637061 13463 leader_election.cc:343] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.638031 13463 leader_election.cc:304] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.638235 13789 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.638314 13799 leader_election.cc:290] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.638991 13799 raft_consensus.cc:2804] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.644588 13788 tablet_bootstrap.cc:654] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.646006 13789 tablet_bootstrap.cc:654] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.648759 13394 catalog_manager.cc:5697] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.639539 13799 raft_consensus.cc:697] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:37.653007 13799 consensus_queue.cc:237] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.654371 13776 leader_election.cc:290] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.637238 13696 leader_election.cc:290] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.655534 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
W20260709 13:06:37.657126 13612 leader_election.cc:343] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.658062 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.658737 13644 raft_consensus.cc:2468] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:37.659803 13463 leader_election.cc:304] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.660151 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.660641 13696 raft_consensus.cc:2804] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.661491 13696 raft_consensus.cc:493] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.661512 13789 tablet_bootstrap.cc:492] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.662061 13696 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.662446 13789 ts_tablet_manager.cc:1403] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:06:37.662940 13787 tablet_bootstrap.cc:492] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.663534 13787 ts_tablet_manager.cc:1403] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.025s
W20260709 13:06:37.663066 13463 leader_election.cc:343] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:37.666816 13789 raft_consensus.cc:359] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.684370 13789 raft_consensus.cc:385] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.684799 13789 raft_consensus.cc:740] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.685633 13789 consensus_queue.cc:260] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.676082 13696 raft_consensus.cc:515] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.651449 13496 raft_consensus.cc:2468] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 1.
I20260709 13:06:37.686700 13787 raft_consensus.cc:359] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.687604 13787 raft_consensus.cc:385] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.687940 13787 raft_consensus.cc:740] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.688598 13609 leader_election.cc:304] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:37.688678 13789 ts_tablet_manager.cc:1434] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.011s
I20260709 13:06:37.688860 13787 consensus_queue.cc:260] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.689638 13788 tablet_bootstrap.cc:492] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.689882 13776 raft_consensus.cc:2804] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.690099 13789 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.690172 13788 ts_tablet_manager.cc:1403] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.024s
I20260709 13:06:37.690390 13776 raft_consensus.cc:697] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:37.693357 13788 raft_consensus.cc:359] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.694044 13788 raft_consensus.cc:385] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.694445 13788 raft_consensus.cc:740] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.693923 13776 consensus_queue.cc:237] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.695161 13788 consensus_queue.cc:260] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.697407 13788 ts_tablet_manager.cc:1434] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:37.697899 13789 tablet_bootstrap.cc:654] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.698446 13788 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: Bootstrap starting.
I20260709 13:06:37.705144 13788 tablet_bootstrap.cc:654] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.707458 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:37.708117 13571 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.709789 13394 catalog_manager.cc:5697] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.716806 13788 tablet_bootstrap.cc:492] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: No bootstrap required, opened a new log
I20260709 13:06:37.717401 13788 ts_tablet_manager.cc:1403] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 13:06:37.717219 13787 ts_tablet_manager.cc:1434] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:06:37.719456 13787 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.716991 13571 raft_consensus.cc:2468] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:37.720366 13788 raft_consensus.cc:359] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.720661 13463 leader_election.cc:304] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.721112 13788 raft_consensus.cc:385] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.721390 13788 raft_consensus.cc:740] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.721580 13799 raft_consensus.cc:2804] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.722121 13788 consensus_queue.cc:260] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.723537 13799 raft_consensus.cc:697] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:37.725068 13788 ts_tablet_manager.cc:1434] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:37.728670 13799 consensus_queue.cc:237] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.733995 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.734766 13644 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.736305 13696 leader_election.cc:290] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.748623 13644 raft_consensus.cc:2468] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:37.752133 13693 raft_consensus.cc:493] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.751886 13395 catalog_manager.cc:5697] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.752900 13693 raft_consensus.cc:515] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.757277 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.757673 13787 tablet_bootstrap.cc:654] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.758097 13496 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:37.759500 13535 leader_election.cc:304] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.760406 13806 raft_consensus.cc:2804] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.760802 13806 raft_consensus.cc:493] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.761129 13806 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.763276 13789 tablet_bootstrap.cc:492] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.763831 13789 ts_tablet_manager.cc:1403] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.074s	user 0.039s	sys 0.019s
I20260709 13:06:37.765731 13808 raft_consensus.cc:493] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.766429 13808 raft_consensus.cc:515] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.767055 13789 raft_consensus.cc:359] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.767750 13789 raft_consensus.cc:385] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.768045 13789 raft_consensus.cc:740] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.768738 13693 leader_election.cc:290] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.769379 13808 leader_election.cc:290] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.769397 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.770452 13644 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:37.770452 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.771166 13646 raft_consensus.cc:2468] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:37.771917 13789 consensus_queue.cc:260] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.773056 13538 leader_election.cc:304] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.774514 13789 ts_tablet_manager.cc:1434] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:37.775240 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.775535 13789 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: Bootstrap starting.
I20260709 13:06:37.777390 13806 raft_consensus.cc:515] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.778079 13496 raft_consensus.cc:2468] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 0.
I20260709 13:06:37.780105 13808 raft_consensus.cc:2804] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.780575 13808 raft_consensus.cc:493] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.780928 13808 raft_consensus.cc:3060] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.782238 13789 tablet_bootstrap.cc:654] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.782294 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.783453 13496 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.791031 13806 leader_election.cc:290] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.791397 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.792048 13496 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:37.792309 13646 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.793663 13535 leader_election.cc:304] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.794730 13806 raft_consensus.cc:2804] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.795271 13806 raft_consensus.cc:697] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:37.796193 13806 consensus_queue.cc:237] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.788252 13394 catalog_manager.cc:5697] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.807397 13808 raft_consensus.cc:515] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.811544 13789 tablet_bootstrap.cc:492] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: No bootstrap required, opened a new log
I20260709 13:06:37.812000 13787 tablet_bootstrap.cc:492] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.812124 13789 ts_tablet_manager.cc:1403] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.018s
I20260709 13:06:37.812541 13787 ts_tablet_manager.cc:1403] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.093s	user 0.026s	sys 0.024s
I20260709 13:06:37.807835 13646 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:37.814904 13789 raft_consensus.cc:359] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.815629 13789 raft_consensus.cc:385] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.816306 13789 raft_consensus.cc:740] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.815543 13787 raft_consensus.cc:359] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.817175 13787 raft_consensus.cc:385] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.817062 13789 consensus_queue.cc:260] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.817548 13787 raft_consensus.cc:740] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.818427 13787 consensus_queue.cc:260] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.819151 13789 ts_tablet_manager.cc:1434] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:06:37.821333 13787 ts_tablet_manager.cc:1434] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:37.821741 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:37.822315 13787 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: Bootstrap starting.
I20260709 13:06:37.822283 13496 raft_consensus.cc:3060] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.829242 13696 raft_consensus.cc:493] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.829847 13696 raft_consensus.cc:515] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.831374 13496 raft_consensus.cc:2468] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:37.831876 13696 leader_election.cc:290] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.832510 13535 leader_election.cc:304] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.832897 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d719eb4edf4a47a60c2b41e3451928" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.833523 13646 raft_consensus.cc:3060] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.833716 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.834364 13644 raft_consensus.cc:2468] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:37.834318 13806 raft_consensus.cc:2804] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.834154 13395 catalog_manager.cc:5697] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.835131 13806 raft_consensus.cc:697] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:37.835881 13463 leader_election.cc:304] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:37.836061 13806 consensus_queue.cc:237] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.833982 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.839777 13696 raft_consensus.cc:2804] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:37.829269 13787 tablet_bootstrap.cc:654] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.839973 13571 raft_consensus.cc:2468] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 0.
I20260709 13:06:37.840883 13696 raft_consensus.cc:493] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.841291 13696 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.839877 13808 leader_election.cc:290] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.845198 13646 raft_consensus.cc:2468] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 1.
I20260709 13:06:37.853552 13696 raft_consensus.cc:515] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.856853 13787 tablet_bootstrap.cc:492] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: No bootstrap required, opened a new log
I20260709 13:06:37.856982 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:37.857640 13571 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.857684 13696 leader_election.cc:290] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.857683 13787 ts_tablet_manager.cc:1403] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.007s
I20260709 13:06:37.859086 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:37.859827 13646 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.861068 13787 raft_consensus.cc:359] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.861779 13787 raft_consensus.cc:385] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.862033 13787 raft_consensus.cc:740] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.862826 13787 consensus_queue.cc:260] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.862613 13394 catalog_manager.cc:5697] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.864708 13571 raft_consensus.cc:2468] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:37.865039 13787 ts_tablet_manager.cc:1434] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:37.866467 13463 leader_election.cc:304] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:37.867419 13696 raft_consensus.cc:2804] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.867926 13696 raft_consensus.cc:697] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:37.868775 13696 consensus_queue.cc:237] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.873944 13646 raft_consensus.cc:2468] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 1.
I20260709 13:06:37.881129 13394 catalog_manager.cc:5697] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:37.901499 13646 raft_consensus.cc:3060] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.908170 13806 consensus_queue.cc:1048] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:37.920936 13693 consensus_queue.cc:1048] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:37.972170 13776 raft_consensus.cc:493] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.972781 13776 raft_consensus.cc:515] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.974618 13776 leader_election.cc:290] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:37.975365 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.975656 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50afd3199d014b0cad8eb33518b110b1" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cfe6266c2c461598761f06c3f46e26" is_pre_election: true
I20260709 13:06:37.976425 13571 raft_consensus.cc:2393] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in current term 1: Already voted for candidate 30f13adabb264ed19bc6cbdf1f873538 in this term.
I20260709 13:06:37.977219 13612 leader_election.cc:304] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:37.977815 13776 raft_consensus.cc:3060] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.980376 13693 raft_consensus.cc:493] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.980930 13693 raft_consensus.cc:515] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:37.984357 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:37.985229 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eb727ef8748471281e26b5bcadf2bbe" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:37.984867 13693 leader_election.cc:290] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:37.985739 13776 raft_consensus.cc:2749] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:37.986078 13494 raft_consensus.cc:2393] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1cfe6266c2c461598761f06c3f46e26 in current term 1: Already voted for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in this term.
I20260709 13:06:37.987411 13535 leader_election.cc:304] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:37.989774 13693 raft_consensus.cc:3060] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.994041 13571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff714cc8473746b09143748633e809d5"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:51236
I20260709 13:06:37.994995 13571 raft_consensus.cc:606] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:37.998564 13693 raft_consensus.cc:2749] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.001096 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:57270
I20260709 13:06:38.001649 13646 raft_consensus.cc:606] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:38.009529 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
mode: GRACEFUL
new_leader_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:57270
I20260709 13:06:38.010066 13646 raft_consensus.cc:606] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Received request to transfer leadership to TS 30f13adabb264ed19bc6cbdf1f873538
I20260709 13:06:38.014981 13802 consensus_queue.cc:1048] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.018139 13494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
mode: GRACEFUL
new_leader_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:06:38.018813 13494 raft_consensus.cc:606] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Received request to transfer leadership to TS f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:38.021852 13776 raft_consensus.cc:993] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.022578 13802 raft_consensus.cc:1081] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.024834 13494 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:06:38.025799 13494 raft_consensus.cc:493] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:38.026196 13494 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.028649 13843 consensus_queue.cc:1048] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.033814 13496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
mode: GRACEFUL
new_leader_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:06:38.033690 13494 raft_consensus.cc:515] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.034518 13496 raft_consensus.cc:606] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Received request to transfer leadership to TS f1cfe6266c2c461598761f06c3f46e26
I20260709 13:06:38.040499 13494 leader_election.cc:290] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.041728 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:38.041916 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c77180ce4e54dc9af03b942d05de500" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:38.042492 13571 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.042822 13646 raft_consensus.cc:3055] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:38.043126 13646 raft_consensus.cc:740] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:38.043889 13646 consensus_queue.cc:260] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.045056 13646 raft_consensus.cc:3060] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.049297 13571 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.050465 13463 leader_election.cc:304] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:38.051317 13696 raft_consensus.cc:2804] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:38.051517 13646 raft_consensus.cc:2468] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.051853 13696 raft_consensus.cc:697] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:38.052803 13696 consensus_queue.cc:237] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.059585 13395 catalog_manager.cc:5697] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.080847 13806 raft_consensus.cc:493] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.081352 13806 raft_consensus.cc:515] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.083604 13806 leader_election.cc:290] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.084059 13494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f13adabb264ed19bc6cbdf1f873538" is_pre_election: true
I20260709 13:06:38.084623 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" is_pre_election: true
I20260709 13:06:38.085062 13494 raft_consensus.cc:2393] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1cfe6266c2c461598761f06c3f46e26 in current term 1: Already voted for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in this term.
I20260709 13:06:38.086431 13535 leader_election.cc:304] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cfe6266c2c461598761f06c3f46e26; no voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:38.087343 13806 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.087201 13843 consensus_queue.cc:1048] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.094789 13806 raft_consensus.cc:2749] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.097659 13843 consensus_queue.cc:1048] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.099452 13776 raft_consensus.cc:993] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.099892 13776 raft_consensus.cc:1081] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.107417 13496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:06:38.108206 13496 raft_consensus.cc:493] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:38.108562 13496 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.114874 13496 raft_consensus.cc:515] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.123104 13843 consensus_queue.cc:1048] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.125579 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:38.126161 13571 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.133556 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f10046984174efd8efcf2b243a6e697" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:38.135262 13799 consensus_queue.cc:1048] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.134253 13646 raft_consensus.cc:3055] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:38.137531 13646 raft_consensus.cc:740] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:38.138410 13646 consensus_queue.cc:260] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.139941 13646 raft_consensus.cc:3060] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.146438 13496 leader_election.cc:290] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.151261 13646 raft_consensus.cc:2468] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.152894 13776 consensus_queue.cc:1048] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.153475 13463 leader_election.cc:304] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:38.155373 13822 raft_consensus.cc:2804] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:38.155877 13822 raft_consensus.cc:697] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:38.156701 13822 consensus_queue.cc:237] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.161279 13822 raft_consensus.cc:993] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: : Instructing follower f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:38.161725 13799 raft_consensus.cc:1081] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Signalling peer f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:38.163791 13569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:06:38.164402 13569 raft_consensus.cc:493] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:38.164803 13569 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.174532 13571 raft_consensus.cc:2468] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.175808 13394 catalog_manager.cc:5697] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.176910 13799 consensus_queue.cc:1048] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.178714 13569 raft_consensus.cc:515] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.187426 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:38.188179 13496 raft_consensus.cc:3055] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:38.188459 13496 raft_consensus.cc:740] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:38.189743 13496 consensus_queue.cc:260] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.190791 13496 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.192117 13806 consensus_queue.cc:1048] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:38.192780 13569 leader_election.cc:290] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.195297 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba4e19f161645c5aac66aaf74c76892" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:38.195981 13644 raft_consensus.cc:3060] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.202430 13799 consensus_queue.cc:1048] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:38.205116 13644 raft_consensus.cc:2468] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:38.208117 13496 raft_consensus.cc:2468] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:38.210244 13535 leader_election.cc:304] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: 
I20260709 13:06:38.211329 13806 raft_consensus.cc:2804] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:38.211855 13806 raft_consensus.cc:697] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:38.212761 13806 consensus_queue.cc:237] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.218688 13693 raft_consensus.cc:993] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: : Instructing follower 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.219946 13808 raft_consensus.cc:1081] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Signalling peer 30f13adabb264ed19bc6cbdf1f873538 to start an election
I20260709 13:06:38.221858 13808 consensus_queue.cc:1048] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.222998 13494 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff714cc8473746b09143748633e809d5"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
 from {username='slave'} at 127.0.0.1:44692
I20260709 13:06:38.223739 13494 raft_consensus.cc:493] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:38.224179 13494 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.231385 13494 raft_consensus.cc:515] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.232475 13496 raft_consensus.cc:1240] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Rejecting Update request from peer f1cfe6266c2c461598761f06c3f46e26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:38.233539 13494 leader_election.cc:290] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Requested vote from peers f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.234557 13806 consensus_queue.cc:1059] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.237607 13693 raft_consensus.cc:3055] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:38.237944 13693 raft_consensus.cc:740] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:38.238751 13693 consensus_queue.cc:260] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.241245 13569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:38.243256 13393 catalog_manager.cc:5697] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 1 to 2, leader changed from 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193) to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.244248 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff714cc8473746b09143748633e809d5" candidate_uuid: "30f13adabb264ed19bc6cbdf1f873538" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:38.245010 13646 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.256599 13696 consensus_queue.cc:1048] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:38.260195 13646 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.263638 13693 raft_consensus.cc:3060] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:38.262682 13538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:40043, user_credentials={real_user=slave}} blocked reactor thread for 43666.3us
I20260709 13:06:38.263567 13463 leader_election.cc:304] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:38.268262 13696 raft_consensus.cc:2804] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:38.269297 13696 raft_consensus.cc:697] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Leader. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:38.270081 13696 consensus_queue.cc:237] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.277688 13569 raft_consensus.cc:2468] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f13adabb264ed19bc6cbdf1f873538 in term 2.
I20260709 13:06:38.278689 13693 consensus_queue.cc:1048] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.281875 13395 catalog_manager.cc:5697] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 reported cstate change: term changed from 1 to 2, leader changed from f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194) to 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "30f13adabb264ed19bc6cbdf1f873538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.291904 13806 consensus_queue.cc:1048] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.306306 13696 consensus_queue.cc:1048] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.315296 13696 raft_consensus.cc:993] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: : Instructing follower f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:38.315780 13696 raft_consensus.cc:1081] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Signalling peer f1cfe6266c2c461598761f06c3f46e26 to start an election
I20260709 13:06:38.317328 13571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:06:38.317873 13571 raft_consensus.cc:493] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:38.318231 13571 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.323936 13799 consensus_queue.cc:1048] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.325497 13571 raft_consensus.cc:515] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.331794 13496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:38.332445 13496 raft_consensus.cc:3055] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:38.332504 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d956e04308804f11a7f39cc63e7f708b" candidate_uuid: "f1cfe6266c2c461598761f06c3f46e26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
I20260709 13:06:38.331898 13571 leader_election.cc:290] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195:40043)
I20260709 13:06:38.332805 13496 raft_consensus.cc:740] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:38.340399 13496 consensus_queue.cc:260] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.341490 13496 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.347551 13496 raft_consensus.cc:2468] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1cfe6266c2c461598761f06c3f46e26 in term 2.
I20260709 13:06:38.348492 13535 leader_election.cc:304] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, f1cfe6266c2c461598761f06c3f46e26; no voters: b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:38.349225 13806 raft_consensus.cc:2804] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:38.349655 13806 raft_consensus.cc:697] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Becoming Leader. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:38.350409 13806 consensus_queue.cc:237] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:38.357873 13394 catalog_manager.cc:5697] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 reported cstate change: term changed from 1 to 2, leader changed from 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193) to f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f1cfe6266c2c461598761f06c3f46e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.511015 13571 raft_consensus.cc:1275] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.512269 13829 consensus_queue.cc:1048] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.519603 13646 raft_consensus.cc:1275] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.521195 13799 consensus_queue.cc:1048] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.592226 13571 raft_consensus.cc:1275] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.593603 13696 consensus_queue.cc:1048] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.601384 13646 raft_consensus.cc:1275] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.602761 13799 consensus_queue.cc:1048] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.628408 13496 raft_consensus.cc:1275] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.629627 13693 consensus_queue.cc:1048] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.637704 13646 raft_consensus.cc:1275] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.639873 13693 consensus_queue.cc:1048] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:38.732017 13571 raft_consensus.cc:1275] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.733362 13799 consensus_queue.cc:1048] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.741055 13646 raft_consensus.cc:1275] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer 30f13adabb264ed19bc6cbdf1f873538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.742359 13799 consensus_queue.cc:1048] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:38.814656 13496 raft_consensus.cc:1275] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.815711 13693 consensus_queue.cc:1048] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.823365 13646 raft_consensus.cc:3060] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:38.830678 13646 raft_consensus.cc:1275] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Refusing update from remote peer f1cfe6266c2c461598761f06c3f46e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:38.832118 13693 consensus_queue.cc:1048] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:41.041667 13395 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39100:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:41.062805 13395 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39100:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:41.120229   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:41.206163 13910 tablet.cc:1724] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.274796 13900 tablet.cc:1724] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:41.308938 13523 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:a9d719eb4edf4a47a60c2b41e3451928)
I20260709 13:06:41.285524 13914 tablet.cc:1724] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:41.315757 13912 log.cc:927] Time spent T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:06:41.307814 13882 log.cc:927] Time spent T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:06:41.326588 13900 tablet.cc:1724] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.337356 13913 tablet.cc:1724] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.304500 13898 tablet.cc:1724] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:41.387454 13523 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:a9d719eb4edf4a47a60c2b41e3451928)
W20260709 13:06:41.388825 13523 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:a9d719eb4edf4a47a60c2b41e3451928)
W20260709 13:06:41.391789 13609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37729, user_credentials={real_user=slave}} blocked reactor thread for 82512.6us
I20260709 13:06:41.437268 13913 tablet.cc:1724] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.446059 13910 tablet.cc:1724] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.458668 13495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9"
dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
mode: GRACEFUL
new_leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:38444
I20260709 13:06:41.459444 13495 raft_consensus.cc:606] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Received request to transfer leadership to TS b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:41.409617 13921 tablet.cc:1724] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.486678 13898 tablet.cc:1724] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.488175 13928 tablet.cc:1724] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:41.501081 13915 log.cc:927] Time spent T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:06:41.503525 13916 log.cc:927] Time spent T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:06:41.513820 13927 tablet.cc:1724] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.515481 13898 tablet.cc:1724] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.506489 13917 tablet.cc:1724] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.503630 13920 tablet.cc:1724] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.507861 13925 tablet.cc:1724] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.521330 13914 tablet.cc:1724] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.530121 13922 tablet.cc:1724] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.536834 13900 tablet.cc:1724] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.537767 13925 tablet.cc:1724] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.530213 13913 tablet.cc:1724] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.548298 13910 tablet.cc:1724] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.553277 13925 tablet.cc:1724] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.562781 13928 tablet.cc:1724] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.547267 13917 tablet.cc:1724] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.593935 13928 tablet.cc:1724] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.597237 13927 tablet.cc:1724] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:41.601754 13909 raft_consensus.cc:993] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: : Instructing follower b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:41.605733 13909 raft_consensus.cc:1081] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Signalling peer b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:41.607673 13643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:06:41.608297 13643 raft_consensus.cc:493] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:41.608685 13643 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.611985 13571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7"
dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
mode: GRACEFUL
new_leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:57200
I20260709 13:06:41.612610 13571 raft_consensus.cc:606] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Received request to transfer leadership to TS b0b59fdb97fa4f4fb61f9da00a32bba7
I20260709 13:06:41.618237   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:41.618860   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:41.620860 13643 raft_consensus.cc:515] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:41.623482 13495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:41.623740 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4799af8d42433e9d10138bd79ebaa9" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:41.624131 13495 raft_consensus.cc:3055] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:41.624395 13571 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.624480 13495 raft_consensus.cc:740] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30f13adabb264ed19bc6cbdf1f873538, State: Running, Role: LEADER
I20260709 13:06:41.625381 13495 consensus_queue.cc:260] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:41.626471 13495 raft_consensus.cc:3060] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.634398 13495 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 3.
I20260709 13:06:41.635447 13571 raft_consensus.cc:2468] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 3.
I20260709 13:06:41.635843 13609 leader_election.cc:304] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:41.636931 13643 leader_election.cc:290] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 3 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:41.636960 13930 raft_consensus.cc:2804] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:41.637588 13930 raft_consensus.cc:697] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:41.638619 13930 consensus_queue.cc:237] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:41.644155 13395 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: soft_deleted_table [id=9bb778b5c3914ac689d451568faedb88] alter complete (version 1)
I20260709 13:06:41.651914 13395 catalog_manager.cc:5697] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 2 to 3, leader changed from 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193) to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:41.973330 13876 raft_consensus.cc:993] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: : Instructing follower b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:41.973887 13861 raft_consensus.cc:1081] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Signalling peer b0b59fdb97fa4f4fb61f9da00a32bba7 to start an election
I20260709 13:06:41.975306 13643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7"
dest_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7"
 from {username='slave'} at 127.0.0.1:57250
I20260709 13:06:41.975739 13643 raft_consensus.cc:493] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:41.976076 13643 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.981698 13643 raft_consensus.cc:515] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:41.983254 13643 leader_election.cc:290] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 3 election: Requested vote from peers 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729), f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013)
I20260709 13:06:41.984073 13495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30f13adabb264ed19bc6cbdf1f873538"
I20260709 13:06:41.984370 13571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7a38f0ae5848ac846044718257dad7" candidate_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f1cfe6266c2c461598761f06c3f46e26"
I20260709 13:06:41.984728 13495 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.984907 13571 raft_consensus.cc:3055] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:41.985253 13571 raft_consensus.cc:740] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1cfe6266c2c461598761f06c3f46e26, State: Running, Role: LEADER
I20260709 13:06:41.986063 13571 consensus_queue.cc:260] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:41.987336 13571 raft_consensus.cc:3060] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.991823 13495 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 3.
I20260709 13:06:41.993207 13609 leader_election.cc:304] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f13adabb264ed19bc6cbdf1f873538, b0b59fdb97fa4f4fb61f9da00a32bba7; no voters: 
I20260709 13:06:41.993889 13571 raft_consensus.cc:2468] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0b59fdb97fa4f4fb61f9da00a32bba7 in term 3.
I20260709 13:06:41.994004 13930 raft_consensus.cc:2804] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:41.994788 13930 raft_consensus.cc:697] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 LEADER]: Becoming Leader. State: Replica: b0b59fdb97fa4f4fb61f9da00a32bba7, State: Running, Role: LEADER
I20260709 13:06:41.995564 13930 consensus_queue.cc:237] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } }
I20260709 13:06:42.004772 13395 catalog_manager.cc:5697] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 reported cstate change: term changed from 2 to 3, leader changed from f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194) to b0b59fdb97fa4f4fb61f9da00a32bba7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b59fdb97fa4f4fb61f9da00a32bba7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40043 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:42.068344 13571 raft_consensus.cc:1275] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:42.069638 13930 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:42.083573 13495 raft_consensus.cc:1275] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:42.085201 13930 consensus_queue.cc:1048] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:42.547516 13571 raft_consensus.cc:1275] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:42.549026 13930 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cfe6266c2c461598761f06c3f46e26" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:42.558094 13495 raft_consensus.cc:1275] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Refusing update from remote peer b0b59fdb97fa4f4fb61f9da00a32bba7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:42.559748 13919 consensus_queue.cc:1048] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f13adabb264ed19bc6cbdf1f873538" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:42.678004   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:06:42.691536 13609 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:06:42.696092 13609 consensus_peers.cc:597] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.705008 13535 consensus_peers.cc:597] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.713769   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:42.714597   431 tablet_replica.cc:333] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.715255   431 raft_consensus.cc:2243] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.715734   431 raft_consensus.cc:2272] T a9d719eb4edf4a47a60c2b41e3451928 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.717634   431 tablet_replica.cc:333] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.718209   431 raft_consensus.cc:2243] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.718632   431 raft_consensus.cc:2272] T d956e04308804f11a7f39cc63e7f708b P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.720409   431 tablet_replica.cc:333] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.720955   431 raft_consensus.cc:2243] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.721396   431 raft_consensus.cc:2272] T fba4e19f161645c5aac66aaf74c76892 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.723142   431 tablet_replica.cc:333] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.723698   431 raft_consensus.cc:2243] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.724522   431 raft_consensus.cc:2272] T 1c77180ce4e54dc9af03b942d05de500 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.726330   431 tablet_replica.cc:333] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.726882   431 raft_consensus.cc:2243] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.727629   431 raft_consensus.cc:2272] T 5f10046984174efd8efcf2b243a6e697 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.729598   431 tablet_replica.cc:333] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.730155   431 raft_consensus.cc:2243] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.730934   431 raft_consensus.cc:2272] T 3f8978ace07e4cb0a4dd24c207d6fac9 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.732913   431 tablet_replica.cc:333] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.733445   431 raft_consensus.cc:2243] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.733873   431 raft_consensus.cc:2272] T 7c3b361c05214d7686b6a29ae887d92a P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.735699   431 tablet_replica.cc:333] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.736289   431 raft_consensus.cc:2243] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.736704   431 raft_consensus.cc:2272] T b47ff1aa0aea4ec4b8ef19898f505207 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.738380   431 tablet_replica.cc:333] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.738888   431 raft_consensus.cc:2243] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.739331   431 raft_consensus.cc:2272] T 145fa6a445994668a08b1d954a9c227d P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.741380   431 tablet_replica.cc:333] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.741904   431 raft_consensus.cc:2243] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.742632   431 raft_consensus.cc:2272] T 50afd3199d014b0cad8eb33518b110b1 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.744614   431 tablet_replica.cc:333] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.745139   431 raft_consensus.cc:2243] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.745548   431 raft_consensus.cc:2272] T 4eb727ef8748471281e26b5bcadf2bbe P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.747334   431 tablet_replica.cc:333] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.747907   431 raft_consensus.cc:2243] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.748692   431 raft_consensus.cc:2272] T 85f8f68254cc4ebea8476631126ffc2a P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.750547   431 tablet_replica.cc:333] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.751060   431 raft_consensus.cc:2243] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.751864   431 raft_consensus.cc:2272] T ff714cc8473746b09143748633e809d5 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.753588   431 tablet_replica.cc:333] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.754145   431 raft_consensus.cc:2243] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.754570   431 raft_consensus.cc:2272] T b2b743418e294d1a9d3eaa20ea044b75 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.756544   431 tablet_replica.cc:333] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.757063   431 raft_consensus.cc:2243] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.757836   431 raft_consensus.cc:2272] T eb6fa7b5859d487cbfad3f0090a3ad78 P 30f13adabb264ed19bc6cbdf1f873538 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.759783   431 tablet_replica.cc:333] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.760289   431 raft_consensus.cc:2243] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.760697   431 raft_consensus.cc:2272] T 3c4799af8d42433e9d10138bd79ebaa9 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.763690   431 tablet_replica.cc:333] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.764266   431 raft_consensus.cc:2243] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.764664   431 raft_consensus.cc:2272] T 0ff234e22c0845c4a8ad42671e104934 P 30f13adabb264ed19bc6cbdf1f873538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.766677   431 tablet_replica.cc:333] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538: stopping tablet replica
I20260709 13:06:42.767135   431 raft_consensus.cc:2243] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.767554   431 raft_consensus.cc:2272] T da7a38f0ae5848ac846044718257dad7 P 30f13adabb264ed19bc6cbdf1f873538 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.807993   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:06:42.840405   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:42.865355 13535 consensus_peers.cc:597] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.872946 13535 consensus_peers.cc:597] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:42.873291 13535 consensus_peers.cc:597] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.877554   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:42.878337   431 tablet_replica.cc:333] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.878963   431 raft_consensus.cc:2243] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.879439   431 raft_consensus.cc:2272] T 5f10046984174efd8efcf2b243a6e697 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.881376   431 tablet_replica.cc:333] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.881935   431 raft_consensus.cc:2243] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.882676   431 raft_consensus.cc:2272] T fba4e19f161645c5aac66aaf74c76892 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.884549   431 tablet_replica.cc:333] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.885103   431 raft_consensus.cc:2243] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.885526   431 raft_consensus.cc:2272] T 50afd3199d014b0cad8eb33518b110b1 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.887244   431 tablet_replica.cc:333] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.887816   431 raft_consensus.cc:2243] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.888677   431 raft_consensus.cc:2272] T a9d719eb4edf4a47a60c2b41e3451928 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.890416   431 tablet_replica.cc:333] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.890941   431 raft_consensus.cc:2243] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.891396   431 raft_consensus.cc:2272] T 3f8978ace07e4cb0a4dd24c207d6fac9 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.893276   431 tablet_replica.cc:333] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.893812   431 raft_consensus.cc:2243] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.894232   431 raft_consensus.cc:2272] T 7c3b361c05214d7686b6a29ae887d92a P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.896188   431 tablet_replica.cc:333] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.896679   431 raft_consensus.cc:2243] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.897084   431 raft_consensus.cc:2272] T 4eb727ef8748471281e26b5bcadf2bbe P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.898726   431 tablet_replica.cc:333] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.899246   431 raft_consensus.cc:2243] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.899672   431 raft_consensus.cc:2272] T ff714cc8473746b09143748633e809d5 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.901422   431 tablet_replica.cc:333] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.901943   431 raft_consensus.cc:2243] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.902364   431 raft_consensus.cc:2272] T 85f8f68254cc4ebea8476631126ffc2a P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.904709   431 tablet_replica.cc:333] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.905227   431 raft_consensus.cc:2243] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.905642   431 raft_consensus.cc:2272] T 1c77180ce4e54dc9af03b942d05de500 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.907487   431 tablet_replica.cc:333] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.908064   431 raft_consensus.cc:2243] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:42.908509 13609 consensus_peers.cc:597] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 -> Peer 30f13adabb264ed19bc6cbdf1f873538 (127.0.107.193:37729): Couldn't send request to peer 30f13adabb264ed19bc6cbdf1f873538. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.908885   431 raft_consensus.cc:2272] T b47ff1aa0aea4ec4b8ef19898f505207 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.910725   431 tablet_replica.cc:333] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.911249   431 raft_consensus.cc:2243] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.912039   431 raft_consensus.cc:2272] T 145fa6a445994668a08b1d954a9c227d P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.914613   431 tablet_replica.cc:333] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.915259   431 raft_consensus.cc:2243] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.916199   431 raft_consensus.cc:2272] T d956e04308804f11a7f39cc63e7f708b P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:42.916391 13612 consensus_peers.cc:597] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 -> Peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Couldn't send request to peer f1cfe6266c2c461598761f06c3f46e26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.918053   431 tablet_replica.cc:333] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.918691   431 raft_consensus.cc:2243] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.919473   431 raft_consensus.cc:2272] T b2b743418e294d1a9d3eaa20ea044b75 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.921617   431 tablet_replica.cc:333] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.922207   431 raft_consensus.cc:2243] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.922712   431 raft_consensus.cc:2272] T eb6fa7b5859d487cbfad3f0090a3ad78 P f1cfe6266c2c461598761f06c3f46e26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.924963   431 tablet_replica.cc:333] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.925411   431 raft_consensus.cc:2243] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.925745   431 raft_consensus.cc:2272] T da7a38f0ae5848ac846044718257dad7 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.928742   431 tablet_replica.cc:333] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.929183   431 raft_consensus.cc:2243] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:42.929876   431 raft_consensus.cc:2272] T 0ff234e22c0845c4a8ad42671e104934 P f1cfe6266c2c461598761f06c3f46e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:42.931842   431 tablet_replica.cc:333] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26: stopping tablet replica
I20260709 13:06:42.932236   431 raft_consensus.cc:2243] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:42.932614   431 raft_consensus.cc:2272] T 3c4799af8d42433e9d10138bd79ebaa9 P f1cfe6266c2c461598761f06c3f46e26 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:42.946712 13612 consensus_peers.cc:597] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 -> Peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Couldn't send request to peer f1cfe6266c2c461598761f06c3f46e26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.964426   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:42.965744 13612 consensus_peers.cc:597] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 -> Peer f1cfe6266c2c461598761f06c3f46e26 (127.0.107.194:35013): Couldn't send request to peer f1cfe6266c2c461598761f06c3f46e26. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.993901   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:43.022269   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:43.022914   431 tablet_replica.cc:333] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.023506   431 raft_consensus.cc:2243] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.024051   431 raft_consensus.cc:2272] T 50afd3199d014b0cad8eb33518b110b1 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.025923   431 tablet_replica.cc:333] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.026482   431 raft_consensus.cc:2243] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.026929   431 raft_consensus.cc:2272] T fba4e19f161645c5aac66aaf74c76892 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.028788   431 tablet_replica.cc:333] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.029381   431 raft_consensus.cc:2243] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.029793   431 raft_consensus.cc:2272] T d956e04308804f11a7f39cc63e7f708b P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.031459   431 tablet_replica.cc:333] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.032043   431 raft_consensus.cc:2243] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.032464   431 raft_consensus.cc:2272] T 1c77180ce4e54dc9af03b942d05de500 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.034116   431 tablet_replica.cc:333] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.034642   431 raft_consensus.cc:2243] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.035046   431 raft_consensus.cc:2272] T a9d719eb4edf4a47a60c2b41e3451928 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.036847   431 tablet_replica.cc:333] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.037410   431 raft_consensus.cc:2243] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.037838   431 raft_consensus.cc:2272] T 3f8978ace07e4cb0a4dd24c207d6fac9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.039719   431 tablet_replica.cc:333] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.040271   431 raft_consensus.cc:2243] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.041049   431 raft_consensus.cc:2272] T 7c3b361c05214d7686b6a29ae887d92a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.042954   431 tablet_replica.cc:333] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.043494   431 raft_consensus.cc:2243] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.043946   431 raft_consensus.cc:2272] T ff714cc8473746b09143748633e809d5 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.045583   431 tablet_replica.cc:333] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.046090   431 raft_consensus.cc:2243] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.046880   431 raft_consensus.cc:2272] T 4eb727ef8748471281e26b5bcadf2bbe P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.048717   431 tablet_replica.cc:333] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.049213   431 raft_consensus.cc:2243] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.049618   431 raft_consensus.cc:2272] T 85f8f68254cc4ebea8476631126ffc2a P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.051508   431 tablet_replica.cc:333] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.052093   431 raft_consensus.cc:2243] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.052526   431 raft_consensus.cc:2272] T 5f10046984174efd8efcf2b243a6e697 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.054207   431 tablet_replica.cc:333] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.054733   431 raft_consensus.cc:2243] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.055171   431 raft_consensus.cc:2272] T eb6fa7b5859d487cbfad3f0090a3ad78 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.057156   431 tablet_replica.cc:333] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.057790   431 raft_consensus.cc:2243] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.058219   431 raft_consensus.cc:2272] T b2b743418e294d1a9d3eaa20ea044b75 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.060182   431 tablet_replica.cc:333] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.060761   431 raft_consensus.cc:2243] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.061198   431 raft_consensus.cc:2272] T 0ff234e22c0845c4a8ad42671e104934 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.063067   431 tablet_replica.cc:333] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.063565   431 raft_consensus.cc:2243] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.064350   431 raft_consensus.cc:2272] T 3c4799af8d42433e9d10138bd79ebaa9 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.066849   431 tablet_replica.cc:333] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.067358   431 raft_consensus.cc:2243] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.068188   431 raft_consensus.cc:2272] T da7a38f0ae5848ac846044718257dad7 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.070320   431 tablet_replica.cc:333] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.070801   431 raft_consensus.cc:2243] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.071224   431 raft_consensus.cc:2272] T b47ff1aa0aea4ec4b8ef19898f505207 P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.072992   431 tablet_replica.cc:333] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7: stopping tablet replica
I20260709 13:06:43.073450   431 raft_consensus.cc:2243] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.073891   431 raft_consensus.cc:2272] T 145fa6a445994668a08b1d954a9c227d P b0b59fdb97fa4f4fb61f9da00a32bba7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.103225   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:43.134742   431 master.cc:562] Master@127.0.107.254:33307 shutting down...
I20260709 13:06:43.156399   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.156945   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.157238   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3b18de68dd994c20a9947a801bbec2c5: stopping tablet replica
I20260709 13:06:43.178181   431 master.cc:584] Master@127.0.107.254:33307 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11360 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24101 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (410996 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:06:43.238935   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 145 messages since previous log ~1 seconds ago
I20260709 13:06:43.239236   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 147 messages since previous log ~1 seconds ago
I20260709 13:06:43.239522   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 147 messages since previous log ~1 seconds ago
I20260709 13:06:43.239868   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:06:43.240106   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:06:43.240401   431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 2 messages since previous log ~264 seconds ago
I20260709 13:06:43.240693   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 2 messages since previous log ~264 seconds ago
I20260709 13:06:43.240978   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 6 messages since previous log ~187 seconds ago
I20260709 13:06:43.241322   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20260709 13:06:43.241779   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 25 messages since previous log ~5 seconds ago
